A
Senior Data Engineer
A
- Posted 13 hours ago
- Be among the first 10 applicants
Job Description
Looking for a Senior Data Engineer with 4+ years of experience to build and scale data pipelines powering AI training datasets.
Key requirements:
- Strong Python & SQL
- Hands-on AWS – S3, Athena, Glue, DynamoDB, Lake Formation
- Airflow / workflow orchestration
- Apache Hudi or Delta Lake
- Strong experience with large-scale audio/video or multimodal data pipelines
- Understanding of GPU workloads, WebDataset / Parquet / sharded data
- Experience with data ingestion, processing and quality validation
This is a great opportunity for someone who enjoys working at the intersection of Data Engineering, AI infrastructure and multimodal data.
More Info
Key Skills
About Company
Skills:
Java, BigQuery, Hadoop, C++, Scala, Kafka, Hive, MLops, Spark, Databricks, Python, Kubernetes, Airflow, Go, HDFS, Delta Lake, CI/CD
Skills:
snowflake , Spark, Databricks, Sql, Python, Airflow, Microsoft Fabric, Azure data services
Skills:
composer , Google Cloud Platform (GCP), Github, BigQuery, Pyspark, Dataproc, Sql, ELT, Cloud Storage, Python, Etl, Pub Sub, Cloud Run, data fusion, CI/CD
Skills:
bigtable , Pyspark, PostgreSQL, Kafka, Avro, Cloud Storage, Terraform, Docker, Elasticsearch, Python, Spark SQL, Java, BigQuery, Google Cloud Platform, Scala, Apache Spark, Dataproc, Sql, Oop, DataFlow, Kubernetes, Airflow, Pub Sub, Cloud Composer, Parquet
Skills:
Pyspark, Sql, Terraform, Gitlab, Databricks, Azure cloud services, Python, Azure DevOps, MERGE INTO, GitHub Actions, DataOps, Delta Lake, CI/CD