Search by job, company or skills

I

Python, Spark Scala Developer

Save
  • Posted 20 hours ago
  • Be among the first 10 applicants
Early Applicant

Job Description

Data Engineering & Development

  • Develop and maintain batch/stream data processing jobs using Spark with Scala and/or Python.
  • Build reusable components and utilities to standardize ingestion, transformation, and validation patterns.
  • Implement efficient transformations, joins, aggregations, and partitioning strategies for large-scale datasets. Performance, Quality & Reliability
  • Tune Spark jobs for performance (e.g., caching, shuffles, partition sizing) and troubleshoot failures or bottlenecks.
  • Write unit/integration tests and ensure code quality through reviews, refactoring, and documentation.
  • Monitor job runs, analyze logs/metrics, and drive continuous improvements in stability and runtime. Data & SQL
  • Use SQL to validate data, reconcile outputs, and support downstream reporting/analytics needs.
  • Collaborate with stakeholders to clarify requirements and translate them into robust data processing logic.
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • 2–5 years of hands-on experience building solutions with Python.
  • 2–5 years of experience working with Apache Spark for large-scale data processing.
  • 2–5 years of experience developing with Scala (or strong ability to work across Scala and Python codebases).
  • Working knowledge of SQL for querying, validation, and troubleshooting datasets.
  • Strong problem-solving skills and ability to collaborate effectively in a team environment.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 148894349

Similar Jobs

Hyderabad, India

Skills:

HiveScalaApache SparkPythonSql

Hyderabad, India

Skills:

JenkinsGcpDockerTerraformAnsibleCloudformationAzurePythonKubernetesAWS

Hyderabad, India

Skills:

CSSPostgreSQLHTMLGoogle CloudAngularDjangoVue.JSReactGitJavascriptMySQLFlaskMongoDBFastAPIRestful ApisAzurePythonAWSWebSocket protocols