Data Engineer Lead to design, build, and manage scalable data infrastructure and pipelines for enterprise-grade data platforms across cloud and big data ecosystems.
The Ideal Candidate Will Lead a Team Of Data Engineers And Drive End-to-end Data Architecture, Ensuring High-performance, Secure, And Reliable Data Solutions That Support Analytics, AI/ML, And Business Intelligence
- Lead design and development of scalable data pipelines and data engineering solutions.
- Architect and implement end-to-end data platforms (batch and real-time processing).
- Build and optimize ETL/ELT workflows using modern data engineering tools.
- Develop robust data models for analytics, reporting, and machine learning use cases.
- Work with big data frameworks like Hadoop, Spark, and distributed computing systems.
- Design cloud-native data solutions on AWS / Azure / GCP.
- Implement data governance, data quality, and data security standards.
- Collaborate with Data Scientists, Analysts, and Business stakeholders.
- Manage streaming data pipelines using Kafka / real-time ingestion tools.
- Monitor, troubleshoot, and optimize data systems for performance and scalability.
- Lead and mentor a team of data engineers.
- Ensure documentation and adherence to best engineering practices.
Exp (Years)
Required Skills
- Strong experience in Python, SQL, and distributed data systems.
- Expertise in Apache Spark, Hadoop, Kafka, Airflow, Databricks.
- Experience with Machine Learning data pipelines.
- Knowledge of DevOps tools (Docker, Kubernetes, CI/CD).
- Familiarity with BI tools (Power BI / Tableau).
Education
- B.E / B.Tech / M.Tech / MCA / M.Sc (Computer Science / IT / Data Science / Statistics)
(ref:hirist.tech)
