Big Data Engineer
- Posted 5 hours ago
- Be among the first 10 applicants
Job Description
- Utilize strong software engineering skills in Java, Python, Scala, and Apache Spark to analyze complex systems and design scalable data solutions.
- Design, develop, and maintain scalable Big Data applications and distributed data processing solutions.
- Integrate new data sources, tools, and technologies into existing Big Data ecosystems.
- Implement scalable, reliable, and fault-tolerant distributed data ingestion and replication strategies.
- Work with large datasets using Apache Spark and various components of the Hadoop ecosystem.
- Collaborate with cross-functional teams to design, develop, test, and deploy data tools supporting business and product requirements.
- Provide technical guidance and mentorship to onsite/offshore developers on architecture, design, and implementation.
- Own end-to-end product features across the software development lifecycle, from development and testing to production deployment.
- Evaluate emerging Big Data technologies and develop prototypes to improve data processing architecture and performance.
- Automate data engineering workflows, testing, deployment, and operational processes.
- Strong hands-on experience in Java, Python, Scala, and Apache Spark.
- Experience with Big Data technologies and the Hadoop ecosystem.
- Strong understanding of data ingestion, transformation, and distributed data processing.
- Experience integrating multiple data sources and developing scalable data pipelines.
- Understanding of distributed data replication strategies.
- Strong object-oriented programming, design, and analysis skills.
- Experience with Agile/Scrum development methodologies.
- Knowledge of secure application development practices.
- Understanding of automated QA and testing requirements for Big Data applications.
- Ability to translate technical and business requirements into effective technical solutions.
- Excellent written and verbal communication skills.
- Apache Airflow experience is preferred.
More Info
Key Skills
distributed data replication
data pipelines
object-oriented programming
secure application development
data ingestion
distributed data processing
About Company
Skills:
Java, Hadoop, Scala, Apache Spark, Python
Skills:
data engineering , Java, Scala, Big Data, Informatica, Sql, Database Technologies, Spark, Ab Initio, Data Warehousing, Data Analytics, Python, AWS, Business Intelligence