DevOps Engineer

Job Description – Data Platform Engineer (AWS | Domino Data Lab | Spark ML)

Job Title: Data Platform Engineer

Experience: 5–9 Years

Location: Chennai / Bengaluru / Hyderabad (Hybrid)

Employment Type: Full-Time

Job Summary

We are seeking an experienced Data Platform Engineer with expertise in Amazon Web Services (AWS), Containerization & Container Orchestration, Data Platform Operations, Domino Data Lab, Git, and Spark ML. The ideal candidate will be responsible for building, maintaining, and optimizing scalable data science platforms while enabling machine learning workloads in a cloud-native environment.

Mandatory Skills

• Amazon Web Services (AWS)
• Containerization (Docker)
• Container Orchestration (Kubernetes/EKS/OpenShift)
• Data Platform Operations
• Domino Data Lab
• Git/GitHub/GitLab
• Apache Spark ML (Spark MLlib)

Roles & Responsibilities

• Design, deploy, and manage enterprise-grade data platforms on AWS.
• Administer and support Domino Data Lab environments for Data Scientists and ML Engineers.
• Build and maintain containerized applications using Docker and orchestrate workloads using Kubernetes or EKS.
• Monitor, troubleshoot, and optimize data platform infrastructure for performance, scalability, and availability.
• Implement CI/CD pipelines and version control practices using Git.
• Develop and optimize machine learning pipelines using Apache Spark ML.
• Collaborate with Data Scientists to deploy, monitor, and operationalize ML models.
• Ensure platform reliability through monitoring, automation, patching, and incident management.
• Implement security best practices, IAM policies, and cloud governance.
• Automate infrastructure provisioning and platform operations wherever possible.
• Participate in platform upgrades, migrations, and disaster recovery planning.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 5–9 years of experience in Data Platform Engineering or Cloud Platform Operations.
• Strong hands-on experience with AWS services.
• Experience administering or supporting Domino Data Lab platforms.
• Expertise in Docker and Kubernetes (or Amazon EKS/OpenShift).
• Strong understanding of Git workflows and version control.
• Experience building ML pipelines using Apache Spark ML.
• Good knowledge of Linux administration and scripting (Python/Bash).
• Strong analytical, troubleshooting, and communication skills.

Preferred Skills

• Infrastructure as Code (Terraform or CloudFormation)
• AWS SageMaker
• Apache Airflow
• Jenkins or GitHub Actions
• Monitoring tools such as Prometheus, Grafana, or CloudWatch
• Data engineering frameworks and ETL pipelines
• MLOps concepts and model lifecycle management

Key Competencies

• Cloud Infrastructure Management
• Data Platform Operations
• MLOps Enablement
• Container Platform Administration
• Performance Optimization
• Automation & DevOps
• Incident & Problem Management
• Cross-functional Collaboration