Azure Data Engineer

JD#1

Data Engineer with Azure Databricks (R2DS - JV Project)

Mandatory Skills

Data Warehousing, Azure Data Engineer, Azure Data Lake, Data Bricks, Pyspark, Python

Job Description

• Proficiency in Databricks and experience with Spark for big data processing. - Solid programming skills in Python or Scala for developing data pipelines. - Knowledge of SQL and experience with relational databases and data warehousing concepts.
• Familiarity with cloud platforms - Azure, particularly in relation to data storage and processing.
• Understanding of data modeling, ETL processes, and data integration methodologies. - Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration abilities to work effectively in a team-oriented environment.
  
  

Skill Area Tools & Technologies

Data

Engineering

Databricks, Apache Spark, Delta Lake

Programming Python, SQL, PySpark

Cloud Platforms Azure

Education Qualificaiton

B Tech

Roles & Responsibilities

• Develop and implement scalable data processing solutions using Azure Databricks, including ETL processes that efficiently handle large data volumes.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs and objectives.
• Optimize data storage and retrieval processes to enhance performance and reduce latency. - Ensure data quality and integrity by implementing best practices for data governance and
  
  

validation.

• Stay updated with the latest trends and technologies in data engineering, and evaluate their applicability to the organization's needs.