Senior Data Engineer
Job Description
Role: Data Engineer – Databricks (Pipeline/CI/CD)
Experience: 5+ Years
Notice Period: Immediate Joiners Only
Job Summary
We are looking for an experienced Data Engineer with strong expertise in Databricks, data pipeline development, and CI/CD practices to design, develop, and maintain scalable data engineering solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks.
- Build and optimize ETL/ELT workflows for efficient data processing.
- Implement CI/CD pipelines for automated testing, deployment, and release management.
- Develop and maintain data integration workflows and transformation processes.
- Monitor pipeline performance, troubleshoot issues, and optimize data processing.
- Collaborate with cross-functional teams to deliver reliable data solutions.
Required Skills
- 5+ years of relevant experience in Data Engineering.
- Strong hands-on experience with Databricks.
- Experience in data pipeline development and orchestration.
- Practical knowledge of CI/CD tools and deployment practices.
- Strong SQL and Python skills.
- Experience with cloud-based data platforms is preferred.
More Info
Key Skills
About Company
Skills:
Github, Pyspark, Pandas, Numpy, Gcp, Docker, Databricks, Azure, Python, AWS, scikit-learn, Great Expectations, Kedro
Skills:
S3, Github, Pyspark, Lambda, Ec2, Kinesis, Docker, Shell scripting, Python, AWS, Scala, Redshift, Sql, Git, Kubernetes, GenAI, Airflow, AI coding assistants, LLM tools, RAG pipelines, LangChain, prompt engineering, dbt, Glue, LlamaIndex
Skills:
snowflake , SQL Server, Microsoft Azure, Python, Sql, ETL processes
Skills:
snowflake , Data Visualization, Python, Sql, Etl, ELT, business intelligence, dbt
Skills:
Pyspark, Sql, Azure ML, Azure Synapse, Azure Data Factory, Databricks, Azure DevOps, FiveTran, GCP Cloud Composer, Vertex AI, GCP BigQuery, GCP Cloud Run, GCP DLP