Senior Data Engineer
Experience: 7 12 Years
Location: Hyderabad
Employment Type: Full-Time
About The Role
We are seeking a highly skilled
Senior Data Engineer to join our Data Science and Engineering team. The ideal candidate will have strong expertise in building scalable data platforms, developing robust ETL/ELT pipelines, and enabling machine learning and analytics workloads through reliable, production-grade data infrastructure.
This role requires hands-on experience with modern data engineering technologies including
DBT, Databricks, PySpark, Airflow, SQL, and Lakehouse architectures.
Key Responsibilities
- Design, develop, and maintain scalable DBT models for analytics, reporting, and machine learning use cases.
- Build and optimize Databricks and PySpark pipelines for processing large-scale structured and unstructured data.
- Develop and manage workflow orchestration using Airflow (Prefect experience is a plus).
- Create and maintain Delta Lake/Iceberg tables and Lakehouse architectures.
- Collaborate with Data Scientists and ML Engineers to productionize feature engineering and machine learning data pipelines.
- Improve performance, scalability, reliability, and cost efficiency of Spark and DBT workloads.
- Implement data quality checks, testing frameworks, monitoring, alerting, and documentation standards.
- Participate in code reviews and contribute to data engineering best practices and architecture discussions.
- Work with cloud-native data platforms and distributed data processing systems.
Required Skills
- Strong hands-on experience in SQL, DBT, Databricks, PySpark, and Airflow.
- Experience building and maintaining large-scale ETL/ELT pipelines.
- Strong understanding of data modeling, data warehousing, and Lakehouse architectures.
- Experience with Delta Lake and/or Apache Iceberg.
- Knowledge of distributed data processing frameworks and performance optimization techniques.
- Experience with Infrastructure as Code (IaC) tools such as Terraform, AWS CDK, or Pulumi.
- Familiarity with cloud platforms such as AWS, Azure, or GCP.
- Strong problem-solving, communication, and stakeholder management skills.
Preferred Skills
- Experience with Prefect orchestration.
- Exposure to machine learning feature stores and MLOps workflows.
- Experience with Kafka, Spark Streaming, or real-time data processing.
- Knowledge of CI/CD, Git, and DevOps practices.
Keywords for Search
DBT, Databricks, PySpark, Airflow, SQL, Delta Lake, Apache Iceberg, Terraform, Data Engineering, ETL, ELT, Lakehouse, Data Modeling, Spark, AWS, Azure, Feature Engineering, Data Warehouse
