Data Engineer

Job Title: Data Engineer

Duration: 6 months

Location: Bangalore

Key Responsibilities

• Design, develop, and maintain scalable Big Data solutions using Hadoop ecosystem technologies such as HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Flume, and Cassandra.
• Write, optimize, and tune complex Java, Hive, and MapReduce jobs for high-performance data processing.
• Lead backend/distributed data systems initiatives while remaining hands-on with development activities.
• Manage and mentor a team of Data Engineers, including full-time employees and third-party/vendor resources.
• Oversee business intelligence initiatives and ensure delivery of high-quality dashboards and analytics solutions.
• Analyze and validate source data integrity to ensure data quality and consistency.
• Support deployment, auditing, and monitoring of data models and attributes for accuracy and performance.
• Develop and maintain stream-processing pipelines using Spark Streaming, Storm, or similar technologies.
• Design and build dimensional data models to improve data accessibility, efficiency, and reporting capabilities.
• Write advanced SQL queries and perform SQL performance tuning and optimization.
• Troubleshoot and resolve complex networking, software, and data-related issues.
• Collaborate with cross-functional teams to prioritize projects and support organizational goals.
• Build and maintain strong working relationships with stakeholders, business teams, and technical teams.

Required Qualifications

• 7–10+ years of experience as a Big Data Engineer or Data Engineer.
• Strong hands-on expertise in Hadoop/Cloudera ecosystem technologies.
• Extensive experience with Apache Spark and distributed data processing frameworks.
• Proficiency in Big Data querying tools such as Hive, Pig, and Impala.
• Strong programming and scripting skills in Java, Python, Scala, Linux scripting, C++, Ruby, PHP, or R.
• Experience with relational and NoSQL databases including PostgreSQL and Cassandra.
• Strong knowledge of distributed systems and backend data architecture.
• Expertise in advanced SQL development and performance tuning.
• Experience with stream-processing systems such as Spark Streaming or Apache Storm.
• Ability to lead technical deliverables and manage project execution independently.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent communication, interpersonal, planning, and organizational skills.
• Ability to work collaboratively in a team-oriented environment.

Preferred Certifications

• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Cloudera Certified Associate (CCA) Data Analyst / Data Engineer
• Databricks Certified Data Engineer Associate or Professional
• Apache Software Foundation Hadoop Certification
• Amazon Web Services Certified Data Analytics – Specialty
• Google Cloud Professional Data Engineer
• Certifications in Spark, Big Data, SQL, or Cloud Data Engineering platforms are highly desirable.