V
Lead Software Engineer
Job Description
Job Summary: We are seeking a Senior Data Engineer to design, build, and optimize our enterprise data platform leveraging Snowflake and AWS. You will be responsible for architecting scalable ETL/ELT pipelines, implementing advanced data modeling, and ensuring high-performance data delivery for analytics and machine learning. The ideal candidate is an expert in Snowflake's cloud-native architecture and has a proven track record of managing complex data lifecycles in an AWS environment.
Key Responsibilities: Data Pipeline Engineering: Lead the design and implementation of reliable, metadata-driven ETL/ELT pipelines using dbt, Snowflake, and Python. AWS Integration: Architect seamless data ingestion flows from diverse sources (APIs, RDBMS, SaaS) into Snowflake using AWS services such as S3, Lambda, and Kinesis/MSK. Advanced Data Modeling: Design and optimize Star and Snowflake schemas, ensuring that data structures are high-performing, scalable, and optimized for analytical consumption. Performance Engineering: Manage Snowflake compute resources (Virtual Warehouses), optimize micro-partitioning, and tune complex SQL queries to ensure maximum cost-efficiency and speed. Automation & CI/CD: Drive engineering excellence by implementing version control (Git), automated testing via dbt, and continuous integration/deployment (CI/CD) workflows. Orchestration: Develop and maintain sophisticated job schedules and dependencies using tools such as Airflow, Dagster, or Prefect. Data Governance & Quality: Implement automated data validation, monitoring, and alerting to ensure data integrity and compliance with enterprise standards (GDPR/HIPAA). Cross-functional Collaboration: Partner with Data Scientists and Business Stakeholders to translate complex requirements into technical data products and comprehensive documentation.
Technical Skills & Qualifications: Snowflake Mastery: 3+ years of deep hands-on experience with Snowflake architecture, including clustering, zero-copy cloning, and secure data sharing. dbt (data build tool): Expert proficiency in dbt (Core or Cloud), including the use of macros, snapshots, testing, and documentation modules. AWS Cloud Stack: Strong experience with AWS S3, IAM, Lambda, and Glue. Advanced SQL & Python: Mastery of SQL for complex transformations and Python for data engineering scripts and automation.
Key Responsibilities: Data Pipeline Engineering: Lead the design and implementation of reliable, metadata-driven ETL/ELT pipelines using dbt, Snowflake, and Python. AWS Integration: Architect seamless data ingestion flows from diverse sources (APIs, RDBMS, SaaS) into Snowflake using AWS services such as S3, Lambda, and Kinesis/MSK. Advanced Data Modeling: Design and optimize Star and Snowflake schemas, ensuring that data structures are high-performing, scalable, and optimized for analytical consumption. Performance Engineering: Manage Snowflake compute resources (Virtual Warehouses), optimize micro-partitioning, and tune complex SQL queries to ensure maximum cost-efficiency and speed. Automation & CI/CD: Drive engineering excellence by implementing version control (Git), automated testing via dbt, and continuous integration/deployment (CI/CD) workflows. Orchestration: Develop and maintain sophisticated job schedules and dependencies using tools such as Airflow, Dagster, or Prefect. Data Governance & Quality: Implement automated data validation, monitoring, and alerting to ensure data integrity and compliance with enterprise standards (GDPR/HIPAA). Cross-functional Collaboration: Partner with Data Scientists and Business Stakeholders to translate complex requirements into technical data products and comprehensive documentation.
Technical Skills & Qualifications: Snowflake Mastery: 3+ years of deep hands-on experience with Snowflake architecture, including clustering, zero-copy cloning, and secure data sharing. dbt (data build tool): Expert proficiency in dbt (Core or Cloud), including the use of macros, snapshots, testing, and documentation modules. AWS Cloud Stack: Strong experience with AWS S3, IAM, Lambda, and Glue. Advanced SQL & Python: Mastery of SQL for complex transformations and Python for data engineering scripts and automation.
More Info
About Company
