Build data pipelines for batch and real-time processing with strong data quality. Ensure systems are performant, reliable, scalable, and cost-efficient
Model data based on how it will be accessed and the capabilities of underlying systems. Ensure data is accessible, stored in appropriate formats, and integrates seamlessly with analytics and machine learning platforms
Write design proposals that explain system architecture, decision trade-offs, and long-term considerations. Maintain documentation for data pipelines and datasets, including operational runbooks and user guides
Analyze data to understand business patterns and extract insights. Develop applications that present findings in clear, actionable formats
Monitor data pipelines to meet service level agreements. Diagnose and resolve production issues promptly
Collaborate with Data Scientists, Product Managers, and DevOps teams to understand requirements and deliver effective data solutions. Contribute to technical discussions on system design and data governance
Mentor junior engineers in developing their technical skills. Foster a collaborative environment focused on continuous learning and quality delivery. Ensure adherence to coding standards and efficient resource utilization
Job Qualifications
Technical Skills:
5+ years of experience in Data Engineering, with a proven track record of designing, building, and maintaining large-scale data pipelines
Strong experience in big data processing frameworks like Apache Spark and table formats like Apache Iceberg
Proficient in SQL and experience with data modeling for analytical databases like Redshift/BigQuery/Snowflake/StarRocks
Experience with AWS services (e.g., Redshift, Athena, Glue, S3, EMR) or other similar systems from other Cloud Providers
Proficient in one high-level language like Java, Scala or Python
Experience with Machine Learning frameworks like SageMaker or MLFlow will be an advantage
Soft Skills
Strong problem-solving abilities with large-scale distributed systemsidentifying issues, optimizing performance, and resolving complex technical challenges
Effective communication skills across all levels, translating technical concepts for both engineering teams and business stakeholders
A sense of ownership and accountability, with a track record of delivering results and following through on commitments
Curiosity about the business context, asking the right questions to understand real problems before building solutions
