AWS Data Engineer (5-8 Years), Pune/ Hyderabad

Dear Professionals

Greetings from Tata consultancy Services,

Job Title AWS Data Engineer with Pyspark

Experiernce: 5-10 Years

Location: Only Pune/ Hyderabad

Mode of Work : Work from Office

Mode of Interview: Virtual

Key Responsibilities

• Data Pipeline Development: Design, build, and optimize scalable batch and near-real-time ETL/ELT pipelines using PySpark and AWS data services.
• Cloud Ingestion & Processing: Process massive datasets from diverse sources including structured databases, APIs, and raw flat files.
• Workload Migration: Migrate legacy on-premises workloads (such as Ab Initio, Informatica, or SSIS) into modern cloud-based data lakes.
• Pipeline Orchestration: Develop, monitor, and schedule complex workflows using Apache Airflow or AWS Step Functions.
• Performance Tuning: Debug and optimize Spark code using the Spark UI, managing memory bottlenecks, skewness, and partitioning issues.
• Data Warehousing & Architecture: Model data structures utilizing Star/Snowflake schemas, Data Lakehouses, or Data Vault patterns.
• Collaborative Delivery: Participate in Agile/Scrum processes, code reviews, and provide technical guidance to junior team members

Technical Skill Requirements

Must-Have Skills

• Core Language: Advanced Python programming and solid SQL querying skills (CTEs, Window functions, complex joins).
• Big Data Framework: Deep understanding of Apache Spark architecture and hand-on PySpark scripting.
• AWS Core Analytics Stack:
• AWS Glue (ETL Jobs, Crawlers, and Catalog management)
• Amazon EMR (Cluster provisioning and configurations)
• Amazon S3 (Data storage strategies, partitioning, Parquet/ORC file optimization)
• Amazon Redshift / Snowflake (Data warehousing and query optimization)
• AWS Lambda & Step Functions (Serverless execution and workflow automation)
• Version Control: Git / GitHub Actions.

If you are Interested in the above opportunity kindly share your updated resume to [Confidential Information] immediately with the details below (Mandatory)

Name:

Contact No.

Email id:

Skillset:

Total exp:

Relevant Exp:

Fulltime highest qualification (Year of completion with percentage scored):

Current organization details (Payroll company):

Current CTC:

Expected CTC:

Notice period:

Current location:

Preffered Location

Any gaps between your education or career (If yes pls specify the duration):

Will you be able to join within 30/45 days (Yes/NO)