Platform Engineer

Key Responsibilities:

• Perform Cloudera platform maintenance and administration.
• Manage and maintain Kafka clusters and Databricks environments.
• Design and support Iceberg-based Lakehouse architecture.
• Work within cloud environments with a strong understanding of infrastructure fundamentals.
• Implement and manage Azure Synapse analytics solutions.
• Develop and maintain automation scripts using Ansible, Terraform, Bash, and Python.
• Write and troubleshoot PySpark code and Spark-based applications.
• Perform advanced troubleshooting and performance tuning of Spark jobs.
• Manage containerized environments using Kubernetes.
• Optimize SQL queries and perform performance tuning for Hive, Impala, and Trino queries.
• Maintain Cloud PostgreSQL servers and manage replication setups.
• Support CI/CD pipelines using Jenkins and GitHub.
• Administer IDMC and Ataccama Data Quality tools (MDM and Profiling) as an added advantage.