Job Title: Data Engineer Databricks & Data Integrations
Location - Chennai / Indore
Experience - 36 years
Employment Type - Full-time
About the Role
We are looking for a skilled Data Engineer to build, manage, and optimize scalable data pipelines using Databricks. This role will play a key role in data source identification, data mapping, data quality assessment, and system integration, ensuring reliable, high-quality data for analytics and application use.
The ideal candidate will have strong hands-on experience with Databricks, data modeling, and integrations, and will collaborate closely with backend, analytics, and product teams.
Key Responsibilities :
Data Source Identification & Quality Assessment
- Identify, analyze, and document available data sources, formats, and access methods.
- Assess data quality across completeness, accuracy, consistency, and reliability.
- Proactively identify data gaps, risks, and opportunities for improvement.
Data Mapping & Integration
- Define and maintain comprehensive data mapping between source systems and Databricks tables.
- Design and implement scalable ETL/ELT pipelines using Databricks and Apache Spark.
- Manage end-to-end data integrations with internal systems and third-party platforms.
- Ensure smooth, reliable, and maintainable data flows across systems.
Databricks & Data Modeling
- Develop and optimize Databricks workloads using Spark and Delta Lake.
- Design efficient data models optimized for performance, analytics, and API consumption.
- Manage schema evolution, versioning, and data lineage.
- Monitor pipeline performance and optimize for cost and scalability.
Data Quality, Validation & Governance
- Implement data validation, reconciliation, and error-handling mechanisms.
- Ensure data accuracy, integrity, and consistency across pipelines.
- Follow best practices for data security, governance, and access control.
Collaboration & Documentation
- Work closely with backend, analytics, and product teams to align data solutions with business needs
- Create and maintain clear technical documentation for data mappings, pipelines, and integrations.
- Participate in design and architecture discussions.
Required Skills & Qualifications:
Technical Skills
- Strong hands-on experience with Databricks and Apache Spark.
- Proficiency in Python and SQL.
- Proven experience in data mapping, transformation, and data modeling.
- Experience integrating data from APIs, databases, and cloud storage.
- Solid understanding of ETL/ELT concepts and data warehousing principles.
Cloud & Platform Experience
- Experience working with AWS, Azure, or GCP(any one of them).
- Familiarity with Delta Lake, Parquet, or similar data formats.
- Understanding of secure data access and role-based permissions.
Good to Have
- Experience supporting data for analytics dashboards or data-driven applications.
- Exposure to streaming or near-real-time data pipelines.
- Experience in startup or fast-paced product development environments.
Soft Skills
- Strong analytical and problem-solving skills.
- Ability to work effectively with cross-functional teams.
- Clear communication and documentation skills.
- Strong ownership mindset with attention to detail.
Why Join Us
- Opportunity to work on impactful data platforms and integrations.
- Hands-on role with ownership over data pipelines and models.
- Collaborative and growth-oriented work culture.
How to Apply / Contact
Sending their resume to: [Confidential Information]
Mentioning the subject line: Application Data Engineer (Databricks).
