Search by job, company or skills

zorba ai

Big Data Engineer(AWS, Python,Airflow)

zorba ai
Hyderabad, India
3-5 Years
Save
new job description bg glownew job description bg glownew job description bg svg
  • Posted 2 months ago
  • Over 50 applicants

Job Description

Job Description – AWS / Python / Airflow / Big Data Engineer

Role: AWS / Python / Airflow / Big Data Engineer

Experience: 3–5 Years

Job Overview

We are seeking two skilled AWS/Python/Airflow/Big Data Engineers with 3–5 years of hands-on experience to join our data engineering team. The ideal candidates will be responsible for designing, developing, and maintaining scalable data pipelines and workflows in a cloud-based AWS environment. This role requires strong expertise in Python, Apache Airflow, AWS services, and Big Data technologies.

Key Responsibilities

  • Design, develop, and maintain ETL and data pipelines using Apache Airflow.
  • Build and manage workflow orchestration (DAGs) for reliable data processing.
  • Implement scalable and cost-efficient solutions on AWS Cloud, leveraging:
    • S3, EC2, Lambda, EMR, Glue, Redshift
  • Write efficient, reusable, and maintainable Python code for data processing and automation.
  • Work with Big Data frameworks such as Apache Spark and Hadoop for large-scale data processing.
  • Optimize data workflows for performance, scalability, and cost efficiency.
  • Collaborate with cross-functional teams (data analysts, product, DevOps) to understand data requirements and deliver solutions.
  • Ensure data quality, reliability, security, and monitoring across all pipelines and workflows.
  • Troubleshoot and resolve data pipeline failures and performance bottlenecks.
Required Qualifications

Technical Skills

  • AWS Cloud Services: Hands-on experience with core AWS services used in data engineering.
  • Python Programming: Strong coding skills with experience using libraries such as Pandas, PySpark, and automation scripts.
  • Apache Airflow: Proven experience building and managing Airflow DAGs and workflows.
  • Big Data Technologies: Familiarity with Spark, Hadoop, or similar distributed data processing frameworks.
  • Data Modeling & ETL: Solid understanding of ETL processes, data transformations, and pipeline design.
  • Databases & SQL: Working knowledge of SQL and relational databases.

Soft Skills

  • Strong problem-solving and debugging abilities.
  • Excellent communication and collaboration skills.
  • Ability to work independently and in a fast-paced team environment.

Preferred Qualifications

  • Experience with CI/CD pipelines and DevOps practices.
  • Familiarity with containerization technologies such as Docker and Kubernetes.
  • Exposure to cloud monitoring, logging, and performance optimization tools.

Skills: python,pipelines,big data,cloud,aws,airflow

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Data Engineering
Employment Type:
Full time

About Company

zorba ai

Job ID: 136621525

Jobs by Skill - IT
Jobs by Skill - Non IT
International Jobs
Last Updated: 12-05-2026 02:01:40 PM
Homejobs in Hyderabad / Secunderabad, TelanganaBig Data Engineer(AWS, Python,Airflow)

Similar Jobs

Senior Python Developer

interscripts, inc.
Hyderabad, India

Skills:

Github, PostgreSQL, Microservices, Nosql, Encryption, Celery, Docker, Agile, Flask, Scrum, Python, AWS, Oauth, Pytest, SQL Server, Jwt, Sql, Django, Rabbitmq, Git, Gcp, MongoDB, FastAPI, Restful Apis, Asynchronous programming, Azure, Kubernetes, async await, unittest, form.io, GitHub Actions

Senior AI QA Engineer: Java + Python

luxoft india
Hyderabad, India

Skills:

graph databases , Sparql, Sql, Docker, Gitlab, Selenium, Api Automation, Postman, Python, Azure DevOps, Graph DB, RAG LLM QA, Cypher, HttpClient, Iceberg, GitHub Actions, SHACL validation, RESTSHARP, Java test automation

Lead Software Engineer (Python, GenAI, LLM)

EPAM
E
Hyderabad, India

Skills:

Tensorflow, Nlp, Pytorch, Python Programming, Keras, in-context learning, T5, Langchain, LLMs, MLflow, chain of thought, retrieval augmented generation, prompt engineering principles, BERT, GPT-3, tree of thought

Remote Python Engineer (FastAPI)

Turing
Hyderabad, India

Skills:

Sql, Tensorflow, Numpy, Pandas, Pytorch, FastAPI, Rest Apis, Python, API lifecycle management, Stable Baselines, reinforcement learning, Gymnasium, OpenAI Gym, async programming

Python FSD

Virtusa
V
Hyderabad, India

Skills:

PostgreSQL, Redux, AWS Athena, Css3, Jenkins, Django, Git, Pandas, MySQL, Html5, Reactjs, Flask, FastAPI, Restful Apis, Python, AWS, JavaScript ES6, Context API, microservices architecture, SQL databases