Description
AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 7-10 years of prior experience in data engineering, with a strong background in Databricks. This role offers an exciting opportunity to work on diverse projects, collaborating with cross-functional teams to design, build, and optimize data pipelines and infrastructure.
Location : Mumbai/Bengaluru/Hyderabad/ Gurgaon (Hybrid)
Responsibilities
- Designing, building, and operating scalable on-premises or cloud data architecture
- Analyzing business requirements and translating them into technical specifications
- Design, develop, and implement data engineering solutions using Databricks on cloud platforms (AWS, Azure)
- Design, develop, and maintain scalable data pipelines and ETL processes
- Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
- Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness
- Implement data governance and security best practices to ensure compliance and data integrity
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring
- Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Overall 7+ years of prior experience in data engineering, with a focus on designing and building data pipelines
- 4+ years experience in Databricks and Spark in at least two end-to-end data engineering implementations
- Comprehensive understanding of the Databricks products and ecosystem
- Strong programming skills in languages such as Python, Scala, or SQL.
- Experience with data modeling, ETL processes, and data warehousing concepts.
- Familiarity with containerization technologies orchestration tools is a plus.
(ref:hirist.tech)
