Data Engineer (AWS+Pyspark)
- Posted 8 hours ago
- Be among the first 10 applicants
Job Description
DRequired Skills
JOB DESCRIPTION
JOB DESCRIPTION
- 4–7 years of hands-on experience as a Data Engineer, designing, building, and supporting data pipelines on AWS.
- Strong programming skills in Python and PySpark, with hands-on experience in distributed data processing and big data frameworks.
- Strong SQL expertise, including complex query writing, performance optimization, and working with large datasets across relational and analytical databases.
- Proven experience working with AWS core services, including Lambda, RDS, CloudWatch, CloudTrail, SNS, and SQS.
- Hands-on experience with AWS data and analytics services such as: IAM, EMR, AWS Glue, Lambda, Lake Formation, RDS, DynamoDB, and related services.
- Strong expertise in IAM management, including role design, policy creation, permission boundaries, and enforcing least-privilege access.
- Experience in building and maintaining end-to-end data ingestion, transformation, and processing pipelines.
- Solid understanding of monitoring, logging, and alerting, leveraging CloudWatch and CloudTrail.
- Excellent debugging, troubleshooting, and root cause analysis skills across data pipelines and AWS services.
More Info
Key Skills
About Company
Skills:
Pyspark, S3, Aws Services, Emr, Python, Sql, AWS, GenAI, Airflow, Glue
Skills:
Git, Sql, AWS, Gitlab, Bitbucket, Python, Jenkins, Pyspark, CodePipeline, Palantir, CodeBuild
Skills:
statistical data analysis , Databricks, Spring Boot, Emr, Java, AWS Glue, Data Warehousing, Hadoop, Pyspark, Kafka, Python, Kubernetes, AWS EKS, Flask, Spark, ETL data pipelines, Flink, Data lakes, Data frameworks, Data engineering concepts, Real-time data processing, NoSQL databases