Responsibilities :

Key Responsibilities:

Lead end-to-end implementation of data ingestion and orchestration workflows using ADF, including scheduling, dependency management, parameterization, and error handling.

Design and develop scalable data processing pipelines in Databricks using Spark-based transformations for batch and incremental loads.

Build and optimize ELT/ETL patterns integrating Snowflake as a target/source, ensuring performance, reliability, and cost efficiency.

Define data pipeline standards (naming, modularity, reusability) and enforce engineering best practices across the team.

Implement monitoring, alerting, and operational runbooks for production pipelines drive incident triage and root-cause analysis.

Collaborate with stakeholders to translate requirements into technical designs, estimates, and delivery plans manage risks and dependencies.

Conduct code reviews, mentor engineers, and guide technical decisions to ensure maintainable and secure solutions.

Improve pipeline performance through tuning, partitioning strategies, and efficient data layout/processing approaches. Minimum Qualifications:

BTECH, MTECH, MCA, or MSC.

7â€9 years of overall experience with strong hands-on expertise in ADF and Databricks for building production-grade data pipelines.

Proven experience designing and supporting reliable ETL/ELT workflows, including scheduling, retries, and failure recovery patterns.

Strong SQL skills and experience working with large datasets and data quality considerations.

Experience collaborating with cross-functional teams and leading technical delivery with ownership mindset.

Additional Responsibilities:

Preferred Qualifications:

Strong experience integrating and optimizing workloads with Snowflake, including loading strategies and performance tuning.

Experience implementing medallion/lakehouse-style architectures and scalable data modeling patterns for analytics consumption.

Familiarity with CI/CD practices for data engineering workflows and automated testing approaches for pipelines.

Experience with production observability practices (pipeline metrics, logging, alerting) and operational excellence.

Demonstrated ability to mentor team members, drive design discussions, and influence engineering standards across projects.

Technical and Professional Requirements:

Technology- Cloud Integration- Azure Data Factory (ADF) Technology- Data Engineering- Databricks Technology- Data on Cloud- Snowflake

ADF (Azure Data Factory) + Databricks+ Snowflake