AWS, Pyspark

6-12 Years

Posted 9 hours ago
Be among the first 10 applicants

Job Description

Skill: Pyspark, AWS

Exp: 6 to 12 years

Location: Pune

Job Summary

We are seeking an experienced AWS PySpark Developer to design, develop, and optimize large-scale data processing pipelines in the cloud. The ideal candidate will have strong expertise in PySpark, AWS services, and ETL workflows, with a focus on building scalable, high-performance data solutions for analytics and reporting.

Key Responsibilities

Design, develop, and maintain ETL pipelines using PySpark on AWS.
Work with large datasets to perform data cleansing, transformation, and aggregation.
Optimize Spark jobs for performance and cost efficiency.
Integrate data from multiple sources including AWS S3, RDS, Redshift, DynamoDB, and APIs.
Implement data quality checks and ensure compliance with governance policies.
Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
Deploy and monitor jobs using AWS Glue, EMR, Step Functions, or Lambda.
Troubleshoot and resolve issues in production data pipelines.
Maintain documentation for data flows, transformations, and architecture.

Required Skills & Qualifications

3–7 years of experience in data engineering or big data development.
Strong proficiency in PySpark and Spark SQL.
Hands-on experience with AWS services such as:

S3, Glue, EMR, Lambda, Athena, Redshift, IAM

Solid understanding of ETL concepts and data warehousing.
Proficiency in Python for scripting and automation.
Experience with version control (Git) and CI/CD pipelines.
Strong problem-solving skills and ability to work in an agile environment.

Preferred Skills

Experience with AWS Glue Studio and Glue Catalog.
Knowledge of Airflow or other orchestration tools.
Familiarity with streaming data using Kafka or Kinesis.
Exposure to Terraform or CloudFormation for infrastructure as code.
Understanding of data security and encryption in AWS.

More Info

Job Type:

Industry:

Function:

Employment Type:

About Company

Cognizant Consulting

Job ID: 151729367

Jobs by Skill - IT

Jobs by Skill - Non IT

International Jobs

Last Updated: 29-07-2026 05:53:10 PM

Homejobs in ChennaiAWS, Pyspark

Similar Jobs

Java Backend Developer with AWS

Cognizant Consulting

6-8 yrs

Chennai, India

Skills:

Java, Oracle Pl Sql, Docker, Spring Boot, Multithreading, AWS, Microservices

Walk in Drive :: Java + AWS :: 1-Aug-26 :: Chennai

Tata Consultancy Services

5-8 yrs

Chennai, India

Skills:

Oauth, S3, RDS, PostgreSQL, Dynamodb, Kafka, Spring Boot, Microservices, Lambda, Cloudwatch, Ec2, JUnit, Docker, Core Java, MySQL, Sqs, Iam, MongoDB, Kubernetes, Api Gateway, AWS, REST API design