Azure Data Architect

We are looking for a seasonedData Architectwith deep expertise inSparkto lead the design and implementation of modern data processing solutions. The ideal candidate will have extensive experience in distributed data processing, large-scale data pipelines, and cloud-native data platforms.

This is a strategic role focused on building scalable, fault-tolerant, and high-performance data systems.

Key Responsibilities:

• Architect, design, and implement large-scale data pipelines usingSpark (batch and streaming).
• Optimize Spark jobsfor performance, cost-efficiency, and scalability.
• Define and implement enterprise data architecture standards and best practices.
• Guide the transition from traditional ETL platforms to Spark-based solutions.
• Lead the integration ofSpark-based pipelinesinto cloud platforms (Azure Fabric/Spark pools).
• Establish and enforce data architecture standards, including governance, lineage, and quality.
• Mentor data engineering teams on best practices with Spark (e.g., partitioning, caching, join strategies).
• Implement and manageCI/CD pipelinesfor Spark workloads using tools likeGIT or DevOps.
• Ensure robust monitoring, alerting, and logging for Spark applications.

Required Skills & Qualifications:

• 10+ years of experience in data engineering, with7+ years of hands-on experience with Apache Spark(PySpark/Scala).
• Proficiency inSpark optimization techniques,Monitoring, Caching,advanced SQL, and distributed data design.
• Experience with Spark onDatabricksandAzure Fabric.
• Solid understanding ofDelta Lake,Spark Structured Streaming, anddata pipelines.
• Strong experience incloud platforms( Azure).
• Proven ability to handle large-scale datasets (terabytes to petabytes).
• Familiarity withdata lakehouse architectures, schema evolution, and data governance.
• Candidate to be experienced in Power BI, with at least 3+ years of experience.

Preferred Qualifications:

• Experience implementingreal-time analyticsusingSpark StreamingorStructured Streaming.
• Certifications in Databricks, Fabric or Spark would be a plus.