Project Leader

Job Title: Snowflake Developer (Data Lake Program with Snowflake Cortex)

Role Overview

The Snowflake Developer is responsible for building and optimizing data pipelines and analytical datasets within a Snowflake-based data lakehouse. This role also includes leveraging Snowflake Cortex to enable AI-driven data transformations, natural language processing, and intelligent data applications directly within the data platform.

Key Responsibilities

• Data Ingestion & Integration
• Develop and maintain scalable data ingestion pipelines (batch and near real-time).
• Load data from databases, APIs, files, and streaming systems into Snowflake.
• Handle structured and semi-structured data (JSON, Parquet, Avro, CSV).
• Integrate Snowflake with cloud storage platforms (S3, Azure Data Lake, GCS).
• Data Transformation (ELT)
• Build ELT pipelines using Snowflake SQL and native features (Streams, Tasks, Dynamic Tables).
• Implement transformations across Bronze, Silver, and Gold layers.
• Use tools like dbt for modular, reusable, and testable transformations.
• Snowflake Cortex & AI Enablement
• Leverage Snowflake Cortex functions within SQL for AI-powered transformations.
• Implement use cases such as:
• Text summarization of large datasets (logs, documents)
• Sentiment analysis and classification
• Natural language enrichment of datasets
• Work with vector embeddings and similarity search for semantic use cases.
• Assist in building AI-ready datasets for downstream analytics and ML.
• Collaborate with data scientists to integrate AI/ML logic into Snowflake pipelines.
• Support simple prompt engineering within Cortex functions for optimized outputs.
• Data Modeling
• Design and implement analytical data models (star schema, snowflake schema).
• Build fact and dimension tables optimized for BI and AI use cases.
• Ensure datasets are structured for both analytics and AI consumption.
• Performance & Cost Optimization
• Optimize SQL queries and Snowflake workloads.
• Use clustering, caching, and efficient compute strategies.
• Monitor warehouse usage and control costs, including Cortex consumption.
• Data Quality & Validation
• Implement data validation checks and quality frameworks.
• Ensure accuracy, completeness, and consistency of data.
• Debug and resolve pipeline and data issues.
• Security & Governance
• Apply RBAC, masking policies, and row-level security.
• Ensure compliance with data governance and privacy standards.
• Maintain proper documentation and lineage for datasets.
• Collaboration & Agile Delivery
• Work with architects, analysts, and AI/ML teams.
• Translate business and AI use cases into technical implementations.
• Participate in Agile development processes.
  

Required Skills & Qualifications

Core Snowflake Skills

• Strong hands-on experience with Snowflake.
• Advanced SQL skills with performance tuning.
• Experience with Snowflake features:
• Streams, Tasks, Dynamic Tables
• Time Travel, Zero-Copy Cloning
  

Snowflake Cortex & AI Skills

• Basic to intermediate experience with Snowflake Cortex functions.
• Understanding of:
• Generative AI concepts (LLMs, embeddings)
• Text processing and NLP basics
• Familiarity with:
• Prompt engineering techniques
• Semantic search and vector similarity concepts
  

Data Engineering Skills

• Experience with ELT/ETL tools (dbt, Airflow, Informatica).
• Knowledge of cloud platforms (AWS / Azure / GCP).
• Familiarity with data lake/lakehouse architecture.
  
  

Programming & Tools

• Proficiency in SQL and working knowledge of Python.
• Experience with Git and CI/CD pipelines.
  
  

Soft Skills

• Strong analytical and problem-solving abilities.
• Good communication and collaboration skills.
• Willingness to learn and adapt to AI-driven data technologies.
  
  

Preferred Qualifications

• Snowflake certification (SnowPro Core).
• Experience working on AI-enabled data platforms.
• Exposure to vector databases or RAG architectures.
• Familiarity with BI tools (Power BI, Tableau, Looker).
• Understanding of DataOps / MLOps practices.
  
  

Key Deliverables

• Scalable and reliable data pipelines.
• AI-enriched datasets using Snowflake Cortex.
• Optimized queries and cost-efficient workloads.
• High-quality, analytics- and AI-ready data models.
  
  

Success Metrics

• Pipeline reliability and performance.
• Adoption of AI-powered data transformations.
• Query efficiency and cost optimization (including Cortex usage).
• Data quality and stakeholder satisfaction.