Data Engineer

DATA ENGINEER

Hyderabad · Full-Time · 3–5 Years Experience

About Deccan AI

Deccan AI delivers pristine, human-curated training data and evaluation pipelines to the world's leading frontier AI labs and enterprises, including Google DeepMind and Snowflake. Backed by a $25M Series A, headquartered in San Francisco with a large operations hub in Hyderabad, we are building the foundational data infrastructure that powers the next generation of AI. Data quality is not an afterthought here. It is the core system.

The Role

We are looking for a Data Engineer with 3 to 5 years of experience to join our Hyderabad team. You will design, build, and maintain the pipelines that ingest, validate, and deliver training data to frontier model teams. This is a high-ownership role. You will not be a cog in a large platform team. You will shape how Deccan's data infrastructure is built.

What You'll Do

• Build robust, fault-tolerant pipelines that ingest contributor outputs at scale and deliver clean, validated datasets to ML engineering teams
• Develop and maintain Airflow DAGs, Flink streaming jobs, and Spark batch pipelines supporting annotation workflows, RLHF data processing, and SFT dataset preparation
• Build data quality frameworks including schema validation, statistical checks, and anomaly detection, ensuring every dataset meets Deccan's standard before it reaches a client
• Own the full data lifecycle: ingestion, versioning, transformation, quality gating, and delivery to downstream consumers
• Work closely with ML engineers, QA teams, and PMs to translate annotation requirements into reliable pipeline logic
• Maintain data infrastructure on AWS or GCP covering storage, orchestration, access controls, and monitoring
• Contribute to Helix, our real-time evaluation platform, building streaming pipelines that capture production traces and trigger quality alerts
• Handle multi-modal data types: text, audio, video, images, code, and documents across concurrent projects

What We're Looking For

Non-negotiable:

• Strong hands-on experience with pipeline orchestration using Airflow, Prefect, or similar
• Proficiency in Apache Spark or Flink for batch and streaming processing at scale
• Solid Python and SQL, production-grade code, not just scripts
• AWS or GCP experience covering S3/GCS, Lambda, IAM, and data lake patterns
• Genuine data quality mindset; you have built validation frameworks and cared about what goes downstream

Strong differentiators:

• Apache Kafka or event-streaming experience
• dbt for data transformation and testing
• Exposure to annotation platforms or AI training data pipelines
• Understanding of RLHF or SFT data structures
• Multi-modal data pipeline experience
• Docker and basic Kubernetes comfort

Who you are:

• You own problems end-to-end
• You care about quality as a philosophy, not a checklist
• You communicate clearly across functions; ML engineers, QA leads, and PMs all need to understand what your pipelines do and why
• You are comfortable with ambiguity; client projects evolve and workflows change
• You have 2 or more years of meaningful tenure at at least one organisation

What Deccan Offers

• Competitive compensation benchmarked to top-tier AI startups in Hyderabad
• Direct exposure to frontier AI development; your pipelines feed models at Google DeepMind and beyond
• A seat at the table early enough to shape how infrastructure is built
• Hyderabad-based role with global reach, working with teams across India and the US
• A culture where quality is genuinely valued, not just talked about