About YAL
YAL is building AI-first products across voice, language, communication, discovery, and
intelligent user experiences. Our systems combine data, AI, NLP, speech, product engineering,
and real-time analytics to create scalable solutions for users and businesses.
Role Overview
We are looking for a hands-on, builder-mentality Data Engineer with 5–8 years of experience to
lead our zero-to-one data platform initiatives. This role focuses heavily on production
application data engineering—ensuring seamless, low-latency data flow between our live
transactional systems, analytical warehouses, and AI models. You will work across multiple data
domains, including product usage data, live user behavior, voice and text datasets, ML training
data, and real-time event streams.
Key Responsibilities
- Architect 0-to-1 Systems: Design, build, and scale our foundational data infrastructure
and pipelines from scratch, making critical early-stage technology choices.
- Production Data Engineering: Manage high-throughput production application data
streams. Implement Change Data Capture (CDC) and ensure seamless, low-latency
synchronization between transactional databases (OLTP) and analytical systems
(OLAP).
- AI/ML Data Infrastructure: Build robust data infrastructure to support AI/ML model
training, feature engineering, evaluation, experimentation, and production monitoring.
- End-to-End Pipelines: Develop reliable ETL/ELT workflows for data ingestion,
transformation, validation, storage, and consumption across structured, semi-structured,
and unstructured data.
- Real-Time & Batch Processing: Build systems for real-time event streaming and batch
data processing based on live product requirements.
- Integration & Collaboration: Work with backend and platform teams to integrate data
pipelines with APIs, internal tools, dashboards, and live production services.
- Data Governance & Quality: Establish the first principles of data quality, consistency,
observability, lineage, governance, and security across all pipelines.
- Cross-Functional Leadership: Collaborate with product and engineering teams to
convert ambiguous business requirements into resilient, production-ready data solutions.
Required Skills
- 5–8 years of experience in data engineering, backend engineering, or data platform
engineering, with a proven track record of 0-to-1 setups in a fast-paced or startup
environment.
- Strong programming fundamentals with deep hands-on experience in Python and
SQL.
- Production Application Data: Extensive experience working with live application
databases (e.g., PostgreSQL, MongoDB) and implementing event-driven architectures
and CDC (Change Data Capture) pipelines.
- Data Modeling & Architecture: Strong understanding of data warehousing, lakehouse
architecture, and distributed systems.
- Cloud & Tools: Experience with cloud platforms (AWS, GCP, or Azure) and hands-on
expertise with processing frameworks/tools such as Airflow, Spark, Kafka, Flink, or dbt.
- Engineering Best Practices: Good understanding of APIs, backend systems, data
quality checks, monitoring, logging, alerting, and pipeline reliability.
Good to Have
- Experience with ML dataset formats, transcription pipelines, speech analytics, or
conversational AI systems.
- Experience instrumenting product telemetry, tracking live user behavior events, and
modeling scalable schemas to power product analytics and map user journeys.
- Experience with data annotation workflows, feedback loops, or human-in-the-loop
systems.
- Hands-on exposure to feature engineering, including building centralized feature stores
from scratch and optimizing real-time data pipelines for machine learning models.
