Job Summary
We are seeking a seasoned, AWS-Certified Data Engineer to design, build, and maintain cloud-native, scalable data pipelines that power real-time analytics and enterprise decision-making. This role plays a critical part in enabling secure, high-quality data solutions while driving innovation in modern data engineering practices across the organization.
The ideal candidate will collaborate closely with data scientists, architects, ETL developers, DevOps teams, and business stakeholders to deliver robust, automated, and production-ready data platforms on AWS.
Key Responsibilities
- Design, develop, test, deploy, orchestrate, monitor, and troubleshoot cloud-based data pipelines and automation workflows following security and best practices.
- Collaborate with data scientists, architects, ETL developers, and business teams to capture, transform, and integrate data from internal systems, external sources, and data warehouses.
- Build automated, scalable, and test-driven data pipelines supporting batch and real-time data processing.
- Lead research, experimentation, and evaluation of batch and streaming data technologies to assess business impact and suitability.
- Advocate and enable the adoption of emerging data engineering technologies and modern DataOps practices.
- Own end-to-end delivery of technical solutions including design, development, testing, and production deployment.
- Ensure data integrity, accuracy, security, and governance across enterprise data assets.
- Maintain high data quality standards for Data Services, Analytics, and Master Data Management (MDM).
- Apply software engineering best practices such as Git-based version control, CI/CD pipelines, and release management.
- Partner with DevOps engineers and cloud architects to improve DataOps tools, frameworks, and AWS CI/CD pipelines.
- Define, document, and continuously improve data engineering processes and standards.
- Mentor and guide ETL developers transitioning to cloud-native data engineering roles.
Basic Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 10+ years of experience in application development (excluding internships).
- 4+ years of hands-on experience with big data technologies.
- 4+ years of experience working with cloud platforms (AWS preferred; Azure or GCP acceptable).
- Strong programming skills in Python, SQL, Scala, or Java (4+ years).
- Experience with distributed computing technologies such as Hadoop, Hive, Spark, Kafka, EMR, MapReduce, MySQL, or similar (4+ years).
- Hands-on experience with real-time data processing and streaming applications (4+ years).
- Experience with NoSQL databases such as MongoDB or Cassandra (4+ years).
- Strong data warehousing experience (Amazon Redshift preferred) 4+ years.
- Advanced SQL development, tuning, and performance optimization 8+ years.
- Strong experience with PySpark 4+ years.
- Experience with workflow orchestration tools such as Airflow, Kubeflow, AWS Step Functions, or Luigi 3+ years.
- Proficiency in UNIX/Linux environments, including shell scripting 6+ years.
- Extensive experience working in Agile engineering environments 8+ years.
Preferred Qualifications
- AWS Certification (AWS Certified Data Analytics, Solutions Architect, or equivalent).
- 3+ years of experience supporting Machine Learning workflows.
- Experience building Data-as-a-Service (DaaS) platforms.
- Experience designing and deploying RESTful APIs and data services.
- Strong understanding of data security, governance, and compliance frameworks.
What We Offer
- Opportunity to work on enterprise-scale, cloud-native data platforms.
- Exposure to cutting-edge AWS and real-time data technologies.
- Collaborative, innovation-driven work environment.
- Competitive compensation and growth opportunities.
If interested, please share your resume at [Confidential Information]
Feel free to call 8108521337
