Job Title: Senior Data Engineer
Location: Hyderabad
Job Type: Full-time
Experience Level: Senior (5+ Years)
No of Positions: 1
Role Overview
We are seeking a highly capable Senior Multi-Cloud Data Engineer to join our enterprise data infrastructure team. In this role, you will be the driving force behind our hybrid, multi-cloud data strategy—seamlessly architecting, scaling, and managing data ecosystems spanning both Google Cloud Platform (GCP) and Microsoft Azure.
You will bridge the gap between these two massive cloud environments, ensuring high-speed data mobility, unified Lakehouse architectures, robust data modeling, and optimized cost strategies. The ideal candidate loves solving cross-cloud data replication and processing challenges at scale.
Key Responsibilities
- Multi-Cloud Architecture & Ingestion: Design, implement, and maintain scalable, secure data pipelines that move and sync data across GCP and Azure environments efficiently.
- Lakehouse Development: Lead the development of our unified Data Lakehouse environment using advanced open table formats (e.g., Delta Lake, Apache Iceberg) to seamlessly map across Azure Data Lake Storage (ADLS) and Google Cloud Storage (GCS).
- Cross-Cloud Processing & Compute: Architect batch and streaming workloads using Apache Spark via managed platforms on both clouds (Azure Databricks and GCP Cloud Dataproc/Dataflow).
- Enterprise Analytics Warehousing: Manage and optimize data warehouse footprints across both Google BigQuery and Azure Synapse Analytics / Microsoft Fabric, ensuring peak query performance and cross-platform visibility.
- Unified Data Modeling: Establish strict data modeling standards (Dimensional/Kimball, Data Vault) that scale predictably across varied multi-cloud analytical layers.
- Orchestration & Infrastructure: Build cohesive cross-cloud orchestration strategies using tools like Apache Airflow (GCP Cloud Composer) or Azure Data Factory (ADF), and manage infrastructure deterministically via Terraform.
Required Qualifications & Skills
Experience & Cross-Cloud Expertise
- 5+ years of dedicated data engineering experience, with at least 2+ years actively architecting solutions across a multi-cloud (GCP + Azure) environment.
- Certifications (Highly Valued): Google Cloud Professional Data Engineer and/or Microsoft Certified: Azure Data Engineer Associate.
Technical Deep Dive
- Google Cloud Ecosystem: Deep proficiency with BigQuery, Dataproc, Cloud Composer, Pub/Sub, and Cloud Storage.
- Microsoft Azure Ecosystem: Deep proficiency with Azure Databricks, Azure Data Factory (ADF), Synapse Analytics, Event Hubs, and ADLS Gen2.
- Big Data Frameworks: Advanced knowledge of Apache Spark (PySpark/SQL) optimized for cross-cloud clusters.
- Modern Storage Formats: Proven experience utilizing Delta Lake, Iceberg, or BigLake tables for multi-cloud Lakehouse consistency.
- Languages & Core Tech: Mastery of SQL and Python. Solid experience with dbt (data build tool) for abstracting modeling definitions away from underlying cloud-specific compute engines.
Soft Skills & Multi-Cloud Perspective
- Cost Observability Mindset: Highly focused on predicting, tracking, and minimizing cross-cloud egress fees and processing overhead.
- Architectural Flexibility: Ability to match the right workload to the right cloud environment based on performance, cost, and native feature benefits.
- Collaborative Translator: Strong communication skills to act as the primary bridge between disparate GCP-focused and Azure-focused infrastructure teams.
