Associate Process Manager

Roles & Responsibility

• Design, develop, and maintain robust ETL/ELT pipelines using Databricks (Spark) and Python (PySpark).
• Develop and optimize complex transformations using SQL (joins, window functions, CTEs, query tuning).
• Build curated datasets and data models to support reporting, dashboards, and advanced analytics use cases.
• Implement pipeline reliability best practices: data quality checks, monitoring, alerting, and reconciliation.
• Optimize Databricks workloads for performance and cost (cluster sizing, partitioning strategies, caching, file formats).
• Work with structured and semi-structured data (JSON, CSV, Parquet/Delta) and handle schema evolution.
• Collaborate with stakeholders to understand business KPIs and deliver data solutions aligned to retail/eCommerce metrics (sales, orders, returns, inventory, customer cohorts).
• Follow engineering best practices for version control (Git), documentation, reusable code patterns, and testing.

• Good to have: Support or migrate Alteryx workflows into Python/Databricks pipelines.

  Must have skills

• 4-8 years of experience in Data Engineering / Data Warehousing / Big Data.
• Strong hands-on experience with Databricks (Jobs/Workflows, notebooks, cluster concepts, Spark tuning fundamentals).
• Strong programming skills in Python (PySpark preferred).
• Excellent SQL skills, including performance tuning and writing complex analytical queries.
• Experience building scalable pipelines and working with large datasets in distributed environments.
• Strong understanding of data engineering concepts: ETL/ELT, orchestration, data validation, and observability.
• Familiarity with modern data storage formats and practices (Delta/Parquet, partitioning, incremental loads).