Search by job, company or skills

INNOVA SOLUTIONS

Pyspark Developer

Save
new job description bg glownew job description bg glow
  • Posted 8 hours ago
  • Be among the first 30 applicants
Early Applicant
Quick Apply

Job Description

  • Develop and optimize scalable data pipelines using Databricks and Apache Spark.
  • Design and implement solutions for data integration, data processing, and data storage on Databricks.
  • Collaborate with data scientists and analysts to build end-to-end analytics solutions.
  • Build and maintain automated workflows, ensuring reliability and efficiency.
  • Design and manage ETL processes in Databricks.
  • Write optimized Spark SQL and PySpark code for data transformations.
  • Work with large datasets and ensure performance, scalability, and reliability of data pipelines.
  • Monitor and troubleshoot existing data pipelines and workflows.
  • Integrate Databricks with other tools and systems like AWS, Azure, or Google Cloud.

About Company

Job ID: 112012051

Similar Jobs

Chennai, India

Skills:

data engineering catalog S3Agile MethodologyPysparkBug Tracking ToolsDevopsAWSAirflowEMR ServerlessMultiple programming languagesLow code toolsetsDevelopment and testing toolsGlueAthenawikis

Pune

Skills:

PysparkHadoopSpark SQLAWSScalaDataFrame APIETL Processes

Hyderabad, India

Skills:

PysparkSqlJavaPythonMavenScalaJenkinsGitdbtAWS Ecosystem

Pune, India

Skills:

Spring BootScrumUnixAgileJavaImpalaMicroservicesTest Driven DevelopmentYarnAlgorithmsHadoopEtlCassandraBi ToolsPythonData StructuresBig DataSparkMap Reduce Framework

Hyderabad, Bengaluru, Pune

Skills:

DatasetPyspark DeveloperSpark'sDataframe