Job Summary
We are looking for an experienced
Big Data Developer with strong expertise in
Hadoop ecosystem, Spark, Scala, and SQL to design and build scalable data processing systems. The ideal candidate should have hands-on experience working with large datasets, distributed computing, and modern data architectures.
Key Responsibilities
- Design, develop, and maintain big data pipelines using Hadoop and Spark
- Build scalable data processing frameworks using Scala and Spark
- Develop and optimize Hive queries and SQL scripts for large datasets
- Work with distributed storage systems like HDFS
- Perform data ingestion, transformation, and processing (ETL/ELT)
- Optimize performance of Spark jobs and Hadoop workloads
- Collaborate with data engineers, analysts, and business teams
- Ensure data quality, reliability, and scalability
- Troubleshoot and resolve performance bottlenecks
Required Skills (Must-Have)- Strong hands-on experience with:
- Hadoop ecosystem (HDFS, Hive, YARN)
- Apache Spark (Core, SQL, DataFrames)
- Scala programming
- Advanced SQL
- Experience in big data processing and distributed systems
- Knowledge of data warehousing concepts
- Strong problem-solving and debugging skills
Good-to-Have Skills
- Experience with Apache Kafka / streaming frameworks
- Exposure to cloud platforms (AWS / Azure / GCP)
- Familiarity with Airflow or workflow orchestration tools
- Knowledge of Python / PySpark
- Understanding of data lake architectures
Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
- 5–8 years of relevant experience in Big Data technologies
Skills: hadoop,big data,spark,sql,scala
