Data Engineer

Role Expectations:

• Architect and lead the development of complex, scalable data pipelines using technologies like Scala, Spark, Python, and Airflow.
• Set technical standards and best practices across data engineering teams. Oversee AWS-based data infrastructure (S3, Redshift, Glue, EMR), ensuring reliability and performance at scale. Lead real-time streaming solutions (Kafka, Kinesis, Snowplow), optimizing for low-latency and high throughput.
• Mentor and guide teams on advanced system design, data observability, and best practices. Drive DataOps initiatives, focusing on data quality, lineage, and governance.
• Collaborate with cross-functional teams to align technical architecture with business goals. Lead the integration of ML platforms like SageMaker for advanced analytics.
• Champion operational excellence through incident resolution and proactive monitoring.

Qualifications:

• 8+ years of hands-on experience in data engineering, with expertise in Scala, Spark, Python, and Airflow. Mastery in cloud data infrastructure (AWS Redshift, Glue, Athena, S3).
• Expertise in real-time streaming technologies (Kafka, Kinesis, Snowplow). Proven ability in DataOps, focusing on data observability, quality, and governance.
• Strong leadership and mentoring skills with a track record of guiding technical teams.
• Deep experience with large-scale distributed systems and Java, Vertx, Spring Boot is a plus.
• Familiarity with machine learning platforms (e.g., SageMaker) is preferred.