Senior Data Engineer

LEAD DATA ENGINEER

Location: Hyderabad

Role: Permanent

Mode: WFO

JOB RESPONSIBILITIES:

• Tracks the various Machine learning projects and their data needs.
• Tracks and improves Kanban process of product maintenance
• Drives complex technical discussions both within company and outside data partners
• Actively Contributes to the design of machine learning solutions by having a deep understanding of how the data is used and how new sources of data can be introduced
• Advocates for investments in tools and technologies to streamline data workflows and reduce technical debt
• Continuously explores and adopts emerging technologies and methodologies in data engineering and machine learning
• Develops and maintains scalable data pipelines to support machine learning models and analytics
• Collaborates with data scientists to ensure efficient data processing and model deployment
• Ensures data quality, integrity, and security across all stages of the data pipeline
• Implements monitoring and alerting systems to detect anomalies in data processing and model performance
• Enhances data versioning, data lineage, and reproducibility practices to improve model
• transparency and auditing

.

QUALIFICATION

• 5+ years of experience in data engineering or related fields, with a strong focus on building scalable data pipelines to support machine learning workflows.
• Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or other relevant fields.
• Specific experience in Kafka needed . Snowflake and data bricks would be huge plus.
• Proven expertise in designing, implementing, and maintaining large-scale, high-performance data architectures and ETL processes managing 1TB a day.
• Strong knowledge of database management systems (SQL and NoSQL), distributed data processing (e.g., Hadoop, Spark), and cloud platforms (AWS, GCP, Azure).
• Experience working closely with data scientists and machine learning engineers to optimize data flows for model training and real-time inference with latency requirements.
• Hands-on experience with data wrangling, data preprocessing, and feature engineering to ensure clean, high-quality data for machine learning models.
• Solid understanding of data governance, security protocols, and compliance requirements (e.g., GDPR, HIPAA) to ensure data privacy and integrity.

Preferred

• Experience in data pipelines and analytics for video-game development
• Experience in Advertising industry
• Experience in online businesses where transactions happen without human intervention.