Search by job, company or skills

Impetus Consultrainers

Databricks Architect

Save
new job description bg glownew job description bg glow
  • Posted a day ago
  • Be among the first 10 applicants
Early Applicant

Job Description

  • We are seeking a highly skilled Databricks Architect to lead the design, implementation, and optimization of scalable cloud-based data platforms.
  • This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, along with proven experience delivering end-to-end enterprise data engineering solutions.

Pyspark,Spark,SQl, Bigdata+GCP+Databricks

  • Architect and implement enterprise-grade Lakehouse solutions using Databricks.Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
  • Lead implementation of:Cloud-based data lakehouse platforms integrating diverse data sources.
  • Real-time data processing pipelines for operational and analytical use cases.
  • Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
  • Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
  • Establish data quality frameworks and governance standards.Implement and manage Unity Catalog, including fine-grained security and access controls.
  • Leverage Databricks components such as:Delta Live TablesAutoloaderStructured StreamingDatabricks WorkflowsIntegration with orchestration tools (e.g., Apache Airflow)Drive CI/CD automation, deployment strategies, and DevOps best practices.
  • Optimize performance of pipelines, Spark jobs, and compute resources.
  • Provide architectural guidance and technical leadership across cross-functional teams.
  • Engage with stakeholders and clients to translate business requirements into scalable technical solutions.
  • Deep expertise in:Databricks and cloud-native storage/compute platformsApache Spark (batch & streaming)Delta Lake & Lakehouse architectureDistributed data processing systemsStrong hands-on programming skills in Python, PySpark, Scala, and SQL.

More Info

Job Type:
Industry:
Employment Type:

Job ID: 147755083

Similar Jobs

Chennai, India

Skills:

ScalaPysparkApache SparkBigdataData ModelingSqlELTApache AirflowGcpDatabricksEtlDelta Live TablesUnity CatalogDelta LakeStructured StreamingData quality frameworksAutoloaderDatabricks Workflows

Chennai, India

Skills:

Apache AirflowPysparkScalaApache SparkDatabricksPythonSqlELTEtlDelta LakeUnity Catalog

Chennai, India

Skills:

snowflake AdfBigQueryAWS GlueEmrDataprocInformaticaGcpClouderaDatabricksAzureTalendAWSGCP datafusionConfluent

Chennai, India

Skills:

ScalaPrometheusDevopsGcpMLopsCloudwatchTerraformDatabricksSplunkAzurePythonAWSMosaic AIMLflowDLTUnity CatalogDelta LakeSQL WarehousesDBRX