Job Description
Role: Databricks Engineer
Job Location: Remote
Job Type: Full Time
Required Skills/Qualifications:
- 10-18 years of relevant experience
- Bachelor's and/or master's degree in computer science or equivalent experience.
- Strong communication, analytical and problem-solving skills with a high attention to detail.
Desired Experience:
- Cloud Platforms (broad knowledge of basic utilities and in-depth knowledge of more than one of the cloud platforms)
- Implement scalable and sustainable data engineering solutions using tools such as Databricks, Azure, Apache Spark, and Python. The data pipelines must be created, maintained, and optimized as workloads move from development to production for specific use cases.
- Architecture Design Experience for Cloud and Non-cloud platforms
- Expertise with various ETL technologies and familiar with ETL tools
- Scrum Management experience with medium and large-scale projects
- Experience to whiteboard enterprise level architectures
- Must have extensive Range of knowledge of cloud/on premise tools and architectures
- Experience to implement large scale hybrid cloud platform & applications
- Knowledge of one or more scripting language
- Experience with CI/CD
- Ability to set and lead the technical vision while balancing business drivers
- Understand Azure Synapse Analytics architecture and component integrations
- Execute Change Tasks including but not limited to security provisioning schema creation DB role creation DDL execution data restoration etc.
- Review workload and provide recommendations for performance optimization and operational efficiency
- Proactively adjust capacity based on current utilization and upcoming usage guestimate projections
- Document and advise developers and end-users about best practices and housekeeping
- Work on incidents / onboarding issues related to Pivoting to Cloud from on-prem datastores.
- Build performing data with Table Distribution and Index types Partition tables using partitioning strategy
- Creating and optimizing star schema models in a Dedicated SQL Pool
Responsibilities:
- Engage with stakeholders for expansion of business and present Technical Architectures Responsible for creating Project Roadmaps and Case Studies
- Delivers software solutions by
- Understanding requirements
- Understanding deadlines
- Understanding systems flow, data usage, and work processes
- Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
- Follow the Agile development methodology; think strategically and execute methodically
- Provides information to client and leadership by collecting, analyzing, and summarizing development and service issues
- Accomplishes engineering and organization mission by completing related results as needed and on time with high quality
- Develop and manage capacity and growth projection forecasts of the environment within budgets
- Write technical responses to RFP's & RFI's
- Documents and demonstrates solutions by developing documentation in the forms of BRDs, TDDs, User Manuals, Support manuals etc.
- End to End Owner for the entire engagement
Job Tags
Full time, Remote job,