Job Description
Job Description:
We are seeking an experienced Data Engineer with strong expertise in Databricks to join our team in Boston, MA. The ideal candidate will have 8+ years of experience in designing, implementing, and optimizing data solutions, including building robust ETL pipelines and advanced analytics platforms. You will work closely with cross-functional teams to manage and process large-scale datasets using Databricks and cloud technologies.
Key Responsibilities:
Design, develop, and maintain ETL pipelines for data ingestion, processing, and analytics using Databricks.
Collaborate with data scientists, analysts, and business stakeholders to deliver end-to-end data solutions.
Optimize data workflows and performance, ensuring scalability and efficiency.
Develop, test, and deploy SQL and Python scripts for data processing and transformation.
Work with cloud platforms, primarily Azure or AWS, to manage large-scale datasets.
Implement data governance and security best practices.
Perform data validation, cleansing, and quality assurance to ensure accurate results.
Troubleshoot and resolve data-related issues.
Qualifications:
8+ years of hands-on experience in data engineering, with expertise in Databricks.
Proficiency in Apache Spark, PySpark, and Scala for large-scale data processing.
Strong experience with SQL for querying and manipulating databases.
Proven experience with cloud platforms like Azure, AWS, or GCP.
Experience with Azure Data Lake, Azure Synapse, or similar tools is a plus.
Familiarity with data warehousing concepts and ETL processes.
Excellent problem-solving skills and ability to work in a collaborative environment.
Strong communication skills and ability to work with cross-functional teams
Job Tags
Similar Jobs
Donato Technologies Inc
...solutions using the optimal combination of technologies. Job Summary experience building/implementing data pipelines using Databricks or similar cloud database. Experience in DLT Expert level knowledge of using SQL to write complex, highly-optimized...
David's Bridal
At Davids Bridal, we empower our customers and our employees to stay true to their dreams and find the one, whether that means the wedding dress that matches her personal styleor the career thats a perfect fit. Were looking for leaders with diverse backgrounds and...
University of California Los Angeles
...Open-rank Faculty Position in Computational and Behavioral Neuroscience in the UCLA Department of Psychology University of California Los Angeles Requisition Number: JPF09740 Open-rank Faculty Position in Computational and Behavioral Neuroscience in the UCLA...
XPRTAS Talent Acquisition
...GENERAL FUNCTION The operation of a Flexographic printing press. The operator will possess a combination of knowledge skill and abilities to carry out work activities related to the operation of the flexographic printing press. SPECIFIC ESSENTIAL RESPONSIBILITIES...
University of Michigan - Ann Arbor
...international cuisines, and sustainability throughout all dining operations. You will report to the Michigan Dining Marketing Manager... .... ~ Experience with video production techniques, including camera operation, lighting, and sound recording. ~ Experience shooting...