AI Data Engineer
Job Description
Key Responsibilities
Experience
Prior Experience In One Or More
- Design and develop LLM-based applications using single-agent or simple multi-agent patterns for business use cases
- Build and maintain RAG pipelines: data ingestion → chunking → embeddings → retrieval → response generation
- Implement prompt engineering techniques (prompt templates, chaining, basic tool/function calling)
- Develop backend services/APIs for AI applications using Python frameworks (FastAPI / Flask / Streamlit)
- Integrate AI solutions with enterprise systems, databases, and APIs
- Apply basic guardrails and validation checks to improve response quality and reduce hallucination
- Work with Data Engineering teams to ensure data quality, pipeline efficiency, and proper documentation
- Collaborate with MLOps teams for deployment, monitoring, and iterative improvements
- Document solutions, reusable components, and best practices
Experience
- 4–6 years total experience, with 1+ year hands-on experience in GenAI / LLM-based applications
- Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.)
- RAG pipelines and retrieval optimisation
- GPT + Agentic AI implementation experience
- Experience with: - LangChain, LangGraph, or similar frameworks
- Agent orchestration and tool-calling architectures
- Deep understanding of: - LLM limitations, evaluation, and optimisation strategies
- Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience
- Deep data analysis experience and handling large volume of data
- Fabric/Azure Databricks/Snowflake data engineering integration skills
- Good exposure to: - Cloud platforms (Azure/AWS/GCP)
- SQL
- Containers, CI/CD, monitoring
Prior Experience In One Or More
- Data Engineering (ETL/ELT, pipelines, orchestration)
- Data Science / ML lifecycle (especially NLP)
- Analytics engineering / data products
- Exposure to model fine-tuning (LoRA/PEFT) or prompt optimisation techniques
- Experience with evaluation of LLM outputs (quality, relevance, latency)
- Understanding of enterprise data privacy and security considerations in GenAI
- Exposure to Azure AI / Azure OpenAI / AI Search ecosystems
- Experience working on real client-facing AI solutions or POCs
More Info
Key Skills
LangChain
CI CD monitoring
LLMs
Cloud platforms
Claude
retrieval optimisation
GPT
LangGraph
Analytics engineering
RAG pipelines
OpenAI
Containers
About Company
Skills:
snowflake , Containers, ELT, Data Science, Nlp, Python, AWS, Pyspark, Api Integration, Sql, Gcp, Databricks, Azure, Etl, LLMs, CI CD monitoring, Claude, retrieval optimisation, GPT, LangGraph, RAG pipelines, LangChain, Cloud platforms, Analytics engineering, Agentic AI implementation, OpenAI, Fabric
Skills:
FastAPI, Azure, Python, Azure DevOps, Application Maintenance, Agile Scrum, GenAI coding assistants, Microsoft Fabric, AI-powered development tools, CI/CD, Secure deployment practices, Performance optimization, Troubleshooting, Data pipeline management
Skills:
Prometheus, Kafka, Grafana, Cloud Storage, Terraform, Postgres, Python, BigQuery, Graphql, Google Cloud Platform, Pulumi, Sql, REST, DataFlow, Kubernetes, Cloud Spanner, Pub Sub, GitHub Actions, Vertex AI, Kubeflow, Cloud Functions, GKE, Cloud Monitoring, Argo CD
Skills:
snowflake , Apis, Sql, Gcp, Data Governance, Data Lake, Databricks, Azure, Python, AWS, GenAI, data pipelines, Llm, rbac, Lakehouse
Skills:
S3, BigQuery, Redshift, Sql, Lambda, DataFlow, Python, Airflow, Galileo, Azure Blob Storage, AWS OpenSearch, R, Glue, Azure AI Search