T
AWS Data Engineer (Pyspark)_Indore,Pune,Hyderabad,,_6 to 9 yrs
Search Jobs
Search by job, company or skills
Search by job, company or skills
Key Responsibilities
ETL & Pipeline Development: Design and optimize scalable ETL batch pipelines in AWS for high performance and reliability.
Orchestration: Manage data workflows using tools like AWS Glue, MWAA, or Step Functions.
Data Processing: Develop large-scale processing jobs using PySpark while ensuring data quality and integrity.
Infrastructure as Code (IaC): Automate and manage AWS infrastructure using Terraform.
Containerization: Deploy and manage applications using Amazon EKS and Docker.
Required Skills & Qualifications
AWS Services: Hands-on expertise with Glue, S3, IAM, KMS, SNS, Athena, Lambda, SQS, CloudWatch, and EC2.
Programming: Proficiency in PySpark for complex data transformations.
Migration: Proven experience moving data from on-premises systems to AWS.
DevOps Tools: Skilled in Terraform for IaC and Docker/Containers for application packaging.
Job ID: 153636919
Skills:
containerization , Microservices, Angular, React, Docker, Python, AWS, Ml, full stack development, Vue, Gcp, Restful Apis, Azure, Kubernetes, Etl, LangGraph, data lakes, data engineering concepts, backend, security best practices, AI ML model integration, LangChain, LLMs, cloud platforms, event-driven architectures, Ai, agent frameworks, Semantic Kernel, compliance in AI applications, real-time processing
Skills:
Docker, Prometheus, Grafana, Kubernetes, Linux Server Administration
Skills:
Kibana, Prometheus, Grafana, Python Scripting, ARM templates, Docker, Terraform, Splunk, Kubernetes, Linux Bash, GitHub Actions, CI CD tools, Azure services
Skills:
Data Warehouse, Pyspark, ELT, Etl, data models, data lakes, data pipelines, Spark jobs, CI CD
Skills:
Java, Spring Boot, Microservices, Angular, Devops, React, Mockito, JUnit, Docker, Swagger, Openshift, Gitlab, Postman, Restful Apis, Kubernetes, AWS
