Search by job, company or skills

Pyspark Developer

  • Posted 3 hours ago
  • Be among the first 10 applicants

Job Description

  • Primary skills: Pyspark , Spark , Hadoop , SQL
  • At least 6+ years of experience in designing and developing large scale, distributed data processing pipelines using PySpark, Hadoop and related technologies.
  • Having expertise in Pyspark, Spark Core, Spark SQL, Batch processing and Spark Streaming
  • Experience with Hadoop, HDFS, Hive and other BigData technologies.
  • Familiarity with Data warehousing and ETL concepts and techniques
  • UNIX shell scripting will be an added advantage in scheduling/running application jobs.
  • At least 5 years of experience in Project development life cycle activities and development/maintenance projects
  • Work with business stakeholders and other SMEs to understand high level business requirements.
  • Work with the Solution Designers and contribute to the development of project plans by participating in the scoping and estimating of proposed project.
  • Apply technical background understanding, business knowledge, system knowledge in the elicitation of Systems Requirements for projects.
  • Possess good knowledge on Spark architecture and transformations using Spark and PySpark.
  • Work in an Agile environment and participation in scrum daily standups, sprint planning reviews and retrospectives.
  • Understand project requirements and translate them into technical solutions which meets the project quality standards
  • Ability to work in team in diverse/multiple stakeholder environment and collaborate with upstream/downstream functional teams to identify, troubleshoot and resolve data issues.
  • Strong problem solving and Good Analytical skills.
  • Excellent verbal and written communication skills.
  • Experience and desire to work in a Global delivery environment.
  • Stay up to date with new technologies and industry trends in Development.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 153799575

Similar Jobs

Bengaluru, India

Skills:

PysparkHiveHadoopSqoopSparkUnix Shell ScriptingSql

Bengaluru, India

Skills:

HiveHadoopPysparkSparkPythonSqlEtl

Bengaluru, India

Skills:

data wrangling Unit TestingConcurrencyCore PythonData PipelinePysparkApisDebuggingOopGitNumpyPandasException HandlingData LoadingRestful ServicesDecoratorsDataframesParallel ProcessingLambda FunctionsError HandlingApplymapVector OperationsApply Map FunctionsGen AIDB Connectivity

Bengaluru, Pune, Hyderabad

Skills:

Big DataELTPySpark Developer

Bengaluru, Chennai

Skills:

PysparkPython

Beware of Scammers

We don’t charge money for job offers