Search by job, company or skills

INNOVA SOLUTIONS

Pyspark Developer

Save
  • Posted a month ago
  • Over 50 applicants
Quick Apply

Job Description

  • Develop and optimize scalable data pipelines using Databricks and Apache Spark.
  • Design and implement solutions for data integration, data processing, and data storage on Databricks.
  • Collaborate with data scientists and analysts to build end-to-end analytics solutions.
  • Build and maintain automated workflows, ensuring reliability and efficiency.
  • Design and manage ETL processes in Databricks.
  • Write optimized Spark SQL and PySpark code for data transformations.
  • Work with large datasets and ensure performance, scalability, and reliability of data pipelines.
  • Monitor and troubleshoot existing data pipelines and workflows.
  • Integrate Databricks with other tools and systems like AWS, Azure, or Google Cloud.

About Company

Job ID: 112012051

Similar Jobs

Chennai, India

Skills:

data wrangling Data pipelineConcurrencyCore PythonPysparkException HandlingApisData LoadingDebuggingOopGitNumpyPandasRestful ServicesUnit testsDecoratorsDataframesLambda FunctionsError handlingParallel ProcessingDB connectivityGen AIVector operationsapply map functionsapplymap

Chennai, India

Skills:

data wrangling ConcurrencyData PipelinePythonPysparkApisDebuggingOopNumpyGitPandasException HandlingData LoadingRestful ServicesDecoratorsDataframesParallel ProcessingLambda FunctionsError HandlingVector OperationsApplymapGen Agentic AIApply Map FunctionsUnit TestsDB Connectivity

Pune, India

Skills:

AWSSqlPythonAzureGcpScalaPyspark

Hyderabad, India

Skills:

PysparkPythonSql

Chennai, India

Skills:

AWS - ServicesS3PysparkApache SparkEmrPythonSqlAirflowSpark-SQL