Devops - Senior Engineer

Job Description

Kubernetes & Helm Management
. Design, implement, and maintain Kubernetes clusters across multiple environments (development, staging, production)
. Develop, customize, and deploy Helm charts for Dremio and other products and services
. Troubleshoot and optimize Kubernetes deployments, including resource allocation, scaling, and performance tuning
. Implement and maintain security best practices for Kubernetes workloads and cluster configurations
. Monitor cluster health, resource utilization, and application performance
Dremio Platform Management
. Deploy, configure, and manage Dremio clusters on Kubernetes using Helm charts
. Optimize Dremio cluster performance through proper resource allocation, scaling strategies, and cluster tuning
. Monitor Dremio cluster health, query performance, and resource utilization
. Troubleshoot Dremio-specific issues including connectivity, performance bottlenecks, and data source integrations
CI/CD Pipeline Development
. Design and implement comprehensive CI/CD pipelines using GitHub Actions for automated testing, building, and deployment
. Integrate Helios workflows into deployment automation processes, particularly for Dremio deployments
. Establish automated testing frameworks and quality gates within deployment pipelines
. Implement infrastructure as code practices using tools like Terraform or similar
. Create and maintain deployment strategies including blue-green, canary, and rolling deployments for Dremio clusters
Automation & Process Improvement
. Develop automation scripts and tools to streamline Dremio deployment processes and reduce manual intervention
. Implement monitoring and alerting solutions for Dremio clusters, deployment pipelines, and application health
. Create comprehensive documentation for Dremio deployment procedures, upgrade processes, and troubleshooting guides
. Collaborate with data engineering teams to establish best practices for Dremio configuration and optimization
Required Qualifications
Technical Skills
. 3+ years of hands-on experience with Kubernetes in production environments
. Strong proficiency in creating, customizing, and managing Helm charts
. Extensive experience with GitHub Actions for CI/CD pipeline development
. Working knowledge of Helios and its integration with deployment workflows
. Experience with Dremio or similar data lake engines (Trino, Presto) is highly preferred
. Strong scripting skills in Bash, Python
. Experience with monitoring tools (Prometheus, Grafana)
Soft Skills
. Strong problem-solving and troubleshooting abilities, particularly with distributed systems
. Excellent communication skills and ability to work collaboratively with data engineering and analytics teams
. Experience in agile development environments
. Ability to work independently and manage multiple priorities effectively
. Experience with capacity planning and performance optimization

Mandatory Competencies