Big Data Engineer
- Posted 5 hours ago
- Be among the first 10 applicants
Job Description
We are looking for an experienced Big Data Engineer with strong software engineering and distributed data processing skills. The ideal candidate should have hands-on experience with Java, Apache Spark, Python/Scala, and the Hadoop ecosystem to build scalable, reliable data solutions.
Roles & Responsibilities
Analyze complex systems and collaboratively design new data products and services.
Develop and integrate new data sources, tools, and data processing solutions.
Design and implement scalable and reliable distributed data replication strategies.
Work with large datasets using Hadoop ecosystem components.
Develop and deploy data ingestion and transformation pipelines.
Collaborate with onsite and offshore teams on architecture, design, and development.
Own product features throughout development, testing, and production deployment.
Evaluate big data technologies and prototype solutions to improve data processing architecture.
Automate development, testing, and deployment processes.
Required Skills
Strong hands-on experience in Java and Apache Spark.
Good programming knowledge of Python and/or Scala.
Experience with the Hadoop ecosystem and large-scale data processing.
Strong understanding of data ingestion, transformation, and distributed data processing.
Experience designing scalable and reliable data pipelines.
Strong object-oriented programming, design, and analysis skills.
Understanding of secure application development practices.
Experience working with Agile development methodologies.
Understanding of automated QA and testing requirements for Big Data applications.
Excellent written and verbal communication skills.
Good to Have
Experience with Apache Airflow.
Experience with Big Data workflow orchestration and automation.
Experience mentoring developers and providing technical direction.
Preferred Candidate: Big Data Engineers / Java Spark Developers with strong coding fundamentals and hands-on distributed data processing experience.
Skills: communication,python,big data,java,hadoop,scala
Roles & Responsibilities
Analyze complex systems and collaboratively design new data products and services.
Develop and integrate new data sources, tools, and data processing solutions.
Design and implement scalable and reliable distributed data replication strategies.
Work with large datasets using Hadoop ecosystem components.
Develop and deploy data ingestion and transformation pipelines.
Collaborate with onsite and offshore teams on architecture, design, and development.
Own product features throughout development, testing, and production deployment.
Evaluate big data technologies and prototype solutions to improve data processing architecture.
Automate development, testing, and deployment processes.
Required Skills
Strong hands-on experience in Java and Apache Spark.
Good programming knowledge of Python and/or Scala.
Experience with the Hadoop ecosystem and large-scale data processing.
Strong understanding of data ingestion, transformation, and distributed data processing.
Experience designing scalable and reliable data pipelines.
Strong object-oriented programming, design, and analysis skills.
Understanding of secure application development practices.
Experience working with Agile development methodologies.
Understanding of automated QA and testing requirements for Big Data applications.
Excellent written and verbal communication skills.
Good to Have
Experience with Apache Airflow.
Experience with Big Data workflow orchestration and automation.
Experience mentoring developers and providing technical direction.
Preferred Candidate: Big Data Engineers / Java Spark Developers with strong coding fundamentals and hands-on distributed data processing experience.
Skills: communication,python,big data,java,hadoop,scala
More Info
Key Skills
About Company
Skills:
Java, Hadoop Ecosystem, Scala, Apache Spark, Agile Scrum, Big Data Technologies, Apache Airflow, automated QA, Python, distributed data replication, data pipelines, object-oriented programming, secure application development, data ingestion, distributed data processing
Skills:
data engineering , Java, Scala, Big Data, Informatica, Sql, Database Technologies, Spark, Ab Initio, Data Warehousing, Data Analytics, Python, AWS, Business Intelligence