Architect, design, and optimize large‑scale data platforms using Databricks, Snowflake, and modern lakehouse patterns. Provide deep hands‑on technical leadership while partnering with pre‑sales teams to shape scalable, cost‑efficient solutions. Build proofs‑of‑concept as an individual contributor to validate architectures and demonstrate value.
Responsibilities
- Define end‑to‑end architecture for enterprise data platforms across Databricks, Snowflake, and cloud ecosystems, ensuring scalability, performance, governance, and security.
- Design and implement ingestion, transformation, and compute optimization frameworks using Spark, SQL, Delta Lake, and platform‑native capabilities, balancing cost and performance.
- Develop standards for data modeling, data quality, medallion architecture, and workload orchestration, collaborating closely with data engineering, analytics, and ML teams.
- Partner with pre‑sales and solutioning teams to qualify use cases, shape solution architecture, estimate workload sizing, and build compelling POCs showcasing technical feasibility.
- Evaluate new capabilities in Databricks, Snowflake, and cloud services; drive platform modernization, automation, observability, and best‑practice adoption across teams.
Knowledge & Skills
- Expert-level knowledge of Databricks (Spark, Delta Live Tables, Unity Catalog) and/or Snowflake (Warehouses, Query Optimization, Snowpark), with strong architectural depth.
- Strong grounding in data modeling, data warehousing, data governance, lakehouse patterns, and distributed systems concepts.
- Understanding of cloud ecosystems (AWS/Azure/GCP), including storage systems, IAM, networking, orchestration, and CI/CD for data pipelines.
- Ability to articulate complex architectural concepts to technical and business stakeholders, connecting platform capabilities to business outcomes.
- Advanced SQL, Python, and Spark programming skills with ability to optimize distributed workloads, troubleshoot cluster performance, and tune compute resources.
Mandatory Experience
- 5–8 years of experience designing and delivering large‑scale data platform architectures, including at least 3 years hands‑on in Databricks and/or Snowflake.
- Demonstrated experience building POCs, reference implementations, and end‑to‑end solutions as an individual contributor with deep technical ownership.
- Strong background in performance tuning, cost optimization, cluster sizing, query optimization, and workload management on modern data platforms.
- Experience working with cross‑functional teams including data engineering, analytics, ML, and pre‑sales/sales solutioning functions.
- Hands‑on experience with cloud data services (e.g., S3/ADLS/GCS, Lambda/Functions, Glue/Synapse/Data Factory, Kubernetes, CI/CD).
