Data Engineer
Search Jobs
Search by job, company or skills
Search by job, company or skills
About the Role: We are looking for a hands-on Data Engineer to join our team and take full ownership of scraping pipelines and data quality. You'll be working on data from 60+ websites involving PDFs, processed via OCR and stored in MySQL/PostgreSQL. You'll build robust, self-healing pipelines and fix common data issues (missing fields, duplication, formatting errors).
Responsibilities:
Own and optimize Airflow scraping DAGs for 60+ sites
Implement validation checks, retry logic, and error alerts
Build pre-processing routines to clean OCR'd text
Create data normalization and deduplication workflows
Maintain data integrity across MySQL and PostgreSQL
Collaborate with ML team for downstream AI use cases
Requirements:
2–5 years of experience in Python-based data engineering
Experience with Airflow, Pandas, OCR (Tesseract or AWS Textract)
Solid SQL and schema design skills (MySQL/PostgreSQL)
Familiarity with CSV processing and data pipelines
Bonus: Experience with scraping using Scrapy or Selenium
Location: Delhi (in-office only)
Salary Range : 50-80k/Month
Job ID: 126889713
Skills:
data engineering , Data Transformation, Power Bi, Data Modeling, Sql, ELT, Git, Microsoft Azure, Python, Etl, Azure DevOps, table maintenance, data quality checks, OneLake, data pipelines, schema validation, Spark notebooks, cloud data platforms, ADF pipelines, Lakehouse tables, Microsoft Fabric
Skills:
Strong SQL including analytical functions and query optimisation, Fluent use of coding agents and LLM tooling, Testing lineage and governance, ETL and ELT batch and streaming in Python, Production ownership of data systems
Skills:
BigQuery, Hadoop, Pyspark, Data Warehousing, Bash, Sparksql, Dataproc, Sql, Cloud Storage, Hive, Gcp, Iam, DataFlow, Python, Airflow, DataFrame, Pub Sub
Skills:
Java, Hive, Hadoop, Scala, Spark, Nodejs, Emr, Python, Sql, AI code generation tools
Skills:
BigQuery, Logging, Tableau, Sql, Gcp, App Engine, Python, GCP IAM, Looker, Monitoring, Production Support, AI LLM applications, Spanner, batch streaming pipelines
