Big Data Developer

JD

Responsibilities:

• Design and Development: Design, develop, and maintain Hadoop applications, including data ingestion, processing, and storage.
• Data Processing: Develop and optimize Hadoop MapReduce jobs, Hive queries, and Pig scripts for data processing and analysis.
• Data Management: Manage and maintain Hadoop clusters, including configuration, monitoring, and troubleshooting.
• Collaboration: Work with cross-functional teams to identify and prioritize project requirements, provide technical guidance, and ensure data quality.
• Innovation: Stay up to date with industry trends, new technologies, and best practices in big data and Hadoop.

Required Skills:

• Hadoop Ecosystem: In-depth knowledge of Hadoop, HDFS, MapReduce, Hive, Pig, HBase, and Spark.
• Programming: Strong programming skills in Java, Python, or Scala.
• Big Data Technologies: Experience with big data technologies, including NoSQL databases, data warehousing, and ETL tools.
• Data Processing: Experience with data processing frameworks, including Hadoop, Spark, and Flink.
• Very good conceptual understanding of Multithreading, distributed computing concepts of Pyspark
• Communication: Excellent communication and collaboration skills.

Preferred Skills:

• Cloud Experience: Experience with cloud-based big data platforms, such as AWS EMR, Azure HDInsight, or Google Cloud Dataproc.
• Machine Learning: Experience with machine learning and deep learning frameworks, such as TensorFlow, PyTorch, or Scikit-learn.

Certifications: Relevant certifications, such as Cloudera Certified Professional