Work Location: Hyderabad (Hybrid Model)
Prestige Skytech, Financial District
Experience: 7 - 12 years
Mandatory Skills: Databricks, Snowflake,Pyspark, SQL
Job Summary
We are seeking a highly skilled Senior Data Engineer/Lead with strong expertise in PySpark, Databricks, Snowflake, DBT, and advanced SQL. The ideal candidate should have hands-on experience building enterprise-scale batch and real-time data pipelines, optimizing high-volume workloads, and designing modern data architectures.
The candidate should possess strong knowledge of Data Warehousing, Data Vault, Data Mesh, Lakehouse, Data Lake, and Microsoft Fabric architectures and should be capable of designing scalable enterprise solutions.
Key Responsibilities
- Design and develop scalable batch and streaming data pipelines using PySpark and Databricks
- Build and optimize enterprise data models and transformation pipelines using Snowflake and DBT
- Develop advanced SQL scripts, stored procedures, and performance-optimized queries
- Design and implement real-time streaming architectures
- Build integrations using APIs, event-driven architectures, and cloud-native services
- Optimize workloads for performance, scalability, and cost efficiency
- Design enterprise data platforms using Lakehouse, Data Mesh, Data Vault, and Fabric principles
- Work with cross-functional teams to build scalable analytics and data engineering solutions
- Implement CI/CD and DevOps practices for data engineering pipelines
- Provide technical leadership and mentor engineering teams
Required Skills
- Strong hands-on experience with PySpark
- Expertise in Databricks
- Strong Snowflake experience
- Strong DBT knowledge
- Advanced SQL expertise
- Strong experience with Stored Procedures and complex SQL optimization
- Hands-on experience with streaming frameworks and real-time processing
- Experience with REST APIs and enterprise integrations
- Strong understanding of distributed data processing
- Working knowledge of AWS and/or Azure
- Excellent analytical and debugging skills
Strong Knowledge Of
- Data Warehousing
- Data Vault Modeling
- Data Mesh
- Data Lake & Lakehouse Architecture
- Microsoft Fabric
- Enterprise Data Architecture Patterns
Good to Have
- Kafka/Event Hub/Kinesis
- Airflow
- Terraform
- Unity Catalog
- Delta Live Tables
- CI/CD pipelines
Skills: sql,pyspark,dbt,data vault,aws,databricks,snowflake
