Knowledge Engineer

Job Information

Description

We are building the next generation of AI-powered analytics platforms that makes large language models dramatically more accurate when working with enterprise data warehouses such as Databricks and Snowflake.

Key Responsibilities

• Design and build end-to-end context enrichment pipelines that supply LLMs with structured, relationship-aware context at query time

• Develop automated metadata extraction, annotation, and enrichment workflows for tabular data (Snowflake, Databricks)

• Implement semantic chunking, embedding generation, and vector store indexing pipelines (Pinecone, Weaviate, pgvector)

• Build and maintain ontology-driven retrieval systems using graph databases (Neo4j, Amazon Neptune, or Stardog)

• Model domain ontologies in OWL / SKOS and load them into a knowledge graph store

• Define and maintain entity relationships, business rules, and cross-domain concept mappings

• Build Cypher / SPARQL query layers that retrieve relevant subgraphs at inference time and inject them into LLM prompts

• Implement and optimise multi-stage RAG pipelines using LangChain, LlamaIndex, or DSPy

• Evaluate and select the right LLM provider (Anthropic, OpenAI, Cohere, Mistral) per use-case accuracy and cost profile

Collaboration with AI product team

• Previous experience creating products for AI, Cloud/Data Platforms like - Gen AI, Databricks, Snowflake, Cloud (Azure, AWS, GCP)

• Product-oriented thinking with attention to user experience and business outcomes.

• Work with AI product managers to translate business requirements into enrichment specifications (column descriptions, verified queries, ontology nodes)

• Collaborate with data engineers and analytics engineers to align enrichment pipelines with dbt model conventions, Unity Catalog tagging standards, and Snowflake semantic model YAML schemas

•Support AI product managers in communicating enrichment-driven quality improvements to stakeholders through eval dashboards and before/after demos