pipeline management , Spark SQL, Data Modelling, Etl Testing, Data Pipeline, Data Governance, Enterprise Integration, Data Intelligence, Extensive Work Experience, data integration projects, Computer Science, Data Quality Management, Enterprise Database System Administration, Agile Delivery, Data Migration Services, Content Ingest Operations

Pandas, Numpy, Python, OpenAI API, Evals, Helicone, AI system monitoring, LangSmith, DeepEval, adversarial testing, AI red-teaming, AutoGen, LangFuse, context architecture for production agents, Weights Biases, RAGAS, LangChain, LLM evaluation frameworks, prompt engineering, LangGraph

Orchestration, Cloud Security, Pyspark, Apache Spark, Kafka, Sql, Databricks, Data Governance, Python, Platform Operations, Retrieval mechanisms, Performance optimisation, Knowledge Base and RAG solutions, Batch and streaming pipelines, Access Management, Troubleshooting, Embeddings, Vectorisation, GenAI applications, Delta Lake, CI/CD, Microsoft Fabric, Monitoring

Pyspark, Apache Spark, Kafka, Sql, Git, Databricks, Python, RAG solution stacks, embeddings, CI CD workflows, vector databases, cloud-native observability, Delta Lake

Data Governance, Data Lineage, Python, Metadata Management, Pandas, Data Transformation, Distributed Systems, FastAPI, Rest Apis, Apache Iceberg, LangGraph, Pydantic, Trino, Observability, Monitoring, LangChain, Parquet, DuckDB, Microservices patterns, Data ingestion, dbt, CI/CD, tracing, SDKs, Event-driven architectures

S3, PostgreSQL, AWS Glue, Redshift, Sql, ELT, Lambda, Docker, MySQL, Restful Apis, Python, Kubernetes, Etl, embeddings, Airflow, Apache Iceberg, RAG architecture, LLMs, vector databases, Claude, QWEN, OpenAI

Apache Airflow, Hadoop, Pyspark, Apache Spark, Cloudera, Advanced Sql, Python

Pyspark, Sql, Databricks, Kafka, Microsoft Fabric, Delta Lake

Python, Sql, Spark, Hadoop, Data Modeling, Machine Learning, Cloud Computing, Apis, Data Warehousing, ETL Processes