Cloud Data Engineer

Job Title: Cloud Data Engineer - Location - Bangalore - (Hybrid - 3Days a Week)

Looking for Immediate Joiners / Serving Notice Peroid.

Experience

• 5 7 years of relevant experience in Data Engineering, Big Data, or Cloud Data Engineering.
• Strong hands-on experience with Hadoop, Hive, HDFS, and PySpark is mandatory.
  
  

Key Responsibilities & Technical Skills

• Develop, maintain, and optimize scalable data engineering solutions using Python and Apache Spark/PySpark.
• Design and build reusable, scalable data ingestion and data transformation frameworks for large-scale data processing.
• Work extensively with the Hadoop ecosystem, including Hadoop, Hive, HDFS, and Impala, preferably in Cloudera-based environments.
• Demonstrate strong understanding of HDFS architecture and internals, including:
• Data storage architecture
• Partition management
• Compaction strategies
• Resource utilization
• Cluster performance optimization
• Troubleshoot complex platform, storage, compute, and cluster-level issues, perform detailed root cause analysis, and implement performance improvements.
• Work with a variety of database technologies and object storage platforms, ensuring optimized connectivity, data access, and processing patterns.
• Apply strong knowledge of distributed data processing, Spark optimization, and data platform engineering best practices.
• Develop and support applications using Kubernetes, including:
• Containerized Spark workloads
• Pod management
• Application scaling
• Troubleshooting
• Operational support
• Build, package, and deploy Spark workloads using JFrog-managed container images and automated CI/CD pipelines.
• Leverage AI Agents, Generative AI, and LLM-powered development tools to improve engineering productivity and accelerate software delivery.
• Collaborate with engineering and platform teams to develop reliable, scalable, and high-performance data solutions.

Mandatory Skills

• Hadoop
• Hive
• HDFS
• PySpark / Apache Spark
• Python
• Distributed data processing
• Spark optimization
  
  

Good to Have / Added Advantage

• Cloudera ecosystem
• Impala
• Kubernetes
• Containerized Spark workloads
• JFrog
• CI/CD pipelines
• Object storage platforms
• Scala development
• Generative AI / AI Agents / LLM-powered development tools
  
  

Preferred Profile

Candidates should have 5 7 years of hands-on experience in data engineering or big data platforms, with strong practical expertise in Hadoop, Hive, HDFS, and PySpark. Experience in Cloudera, Kubernetes, containerized Spark, CI/CD, and platform performance optimization will be an added advantage.