Senior AI Data Engineer

Global Growth – Global BD Agent Suite:

Build the permission-aware knowledge layer under the Global BD agents. Ingest and keep fresh the Global BD Hub, SharePoint, Salesforce and licensed sources; extract, chunk and PII-scrub content; carry access-control metadata to the search index; and make sure agents only ever retrieve what the requesting user is entitled to see.

Mandatory focus:

Microsoft Graph delta queries and change webhooks; Salesforce CDC / SOQL / Bulk API; Azure Data Factory / Logic Apps and Azure Functions; ADLS Gen2; Azure AI Search indexing with ACL metadata; document extraction and PII scrubbing.

Key Responsibilities:

• Build ingestion from SharePoint Online / Global BD Hub using Graph delta queries and change webhooks, and from Salesforce using CDC and SOQL watermark pulls.
• Land data in an immutable, source-first, date-partitioned ADLS Gen2 raw zone and maintain lineage.
• Extract and chunk PDF, Word, Excel and PowerPoint content, preserving tables, headings and metadata for retrieval.
• Propagate permissions by carrying SharePoint / Salesforce ACLs into chunk metadata so Azure AI Search can security-trim results per user.
• Scrub PII in pitch records, client bios and contact data in line with GDPR and Purview classification.
• Onboard licensed sources such as COMvergence and other third-party data, validating API availability, licensing and usage constraints with source owners.
• Monitor freshness quality and failures, and flag stale, duplicate or contradictory content back to the Hub team.

Required Skills & Expected Capability:

• Core Data Engineering: Strong Python and SQL; pipeline orchestration with Azure Data Factory/Logic Apps; Azure Functions
• M365 / SharePoint Data: Microsoft Graph API, delta queries, change notifications / webhooks, SharePoint lists, libraries and permissions model
• Salesforce Data: Salesforce CDC, SOQL, Bulk API, Platform Events
• Search Indexing: Azure AI Search indexes, indexers and skillsets; embeddings pipelines; hybrid (vector + metadata) retrieval support
• Governance: ACL propagation, PII detection and scrubbing, Purview, data lineage, GDPR awareness

You will fit right in if you have:

• 5–8 years in data engineering, with 1+ year preparing unstructured content for RAG / LLM use
• Has built ingestion from SharePoint / M365 and / or Salesforce in an enterprise tenant
• Understands that a wrong permission on a chunk is a data-exposure incident, not a bug