DATA ENGINEER - Azure Databricks
DATA ENGINEER - Azure Databricks
Happiest Minds Technologies3-5 Years
- Posted 39 minutes ago
- Be among the first 10 applicants
Job Description
As a Azure Data Engineer, you will leverage Azure technologies to drive data transformation, analytics, and machine learning. You will design scalable Databricks data pipelines using PySpark, transforming raw data into actionable insights. You will execute large-scale data processing using Spark Pools, fine-tuning configurations for efficiency. The ideal candidate holds a Bachelors or Masters in Computer Science, Data Science, or a related field, with 3+ years in data engineering and 3+ years specializing in Azure Databricks, PySpark, and Spark Pools. Proficiency in Python PySpark, Pandas, NumPy, SciPy, Spark SQL, DataFrames, RDDs, Delta Lake, Databricks Notebooks, and MLflow is required, along with hands-on experience in Azure Data Lake, Blob Storage, and Synapse Analytics.
Databricks, PySpark, Azure
Databricks, PySpark, Azure
More Info
Key Skills
RDDs
Databricks Notebooks
MLflow
DataFrames
Synapse Analytics
Delta Lake
Blob Storage
Spark Pools
About Company
Skills:
Workflows, Apis, Cloudformation, Pyspark, Sql, Devops, MLops, Terraform, Spark, Databricks, Azure, Python, AWS, Databricks Lakehouse, dbt
Skills:
Data Modelling, Scala, Azure Databricks, Encryption, Sql, Azure Data Factory, Azure Synapse Analytics, Spark, Python, GDPR Principles, IaC, CI CD, Azure Data Lake Storage, rbac, Data Security Compliance