Search by job, company or skills

Emagine

Data Engineer

Save
  • Posted 2 days ago
  • Be among the first 20 applicants
Early Applicant

Job Description

Summary

We are seeking a talented Data Engineer (Senior Consultant) to enhance and manage Azure data pipelines that support our CMO's single source of truth (SSOT). The ideal candidate will ensure reliable data ingestion, consolidation, and continuity across all cost domains, while facilitating effective data delivery through Palantir Foundry and discovery via Sinequa where applicable.

Main Responsibilities:

As a Data Engineer, you will play a vital role in the following:

  • Deliver robust ELT/ETL pipelines that integrate multi-source cost data and maintain the integrity of SSOT datasets.
  • Automate refresh, incremental loads, and (when relevant) streaming ingestion for improved timeliness and efficiency.
  • Produce curated, consumption-ready datasets in Palantir Foundry to support analytics and reporting at scale.
  • Embed data quality checks and reconciliation controls to minimize incidents and enhance resolution times.
  • Monitor pipeline health and troubleshoot failures to swiftly restore service and protect SSOT availability.
  • Collaborate with business teams to verify cost outputs and drive improvements in data accuracy.
  • Maintain comprehensive documentation, catalog metadata, and lineage to enhance governance adoption.
  • Enable Sinequa discovery by providing indexed datasets and enforcing security protocols.

Key Requirements:

  • Platform Engineering (Azure): Proficiency in ADLS Gen2, networking fundamentals, Key Vault/secrets management, managed identities, and Azure Monitor/Log.
  • Palantir Foundry: Prior experience in using this platform.
  • Sinequa: Familiarity with the Sinequa platform.
  • Experience with Azure data services, particularly ADLS Gen2 and Azure Monitor.
  • Familiarity with ETL/ELT pipeline development.
  • Ability to handle multi-source data integration.
  • Strong troubleshooting skills for addressing data pipeline issues.

Nice to Have:

  • Experience utilizing Palantir Foundry.
  • Knowledge of the Sinequa platform.
  • Past experience in cost data domains.

Other Details

This position offers an opportunity to work in a dynamic and supportive environment that prioritizes data excellence. Remote work options are available to enhance work-life balance.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 149333507

Similar Jobs

Pune, India

Skills:

PysparkData ModelingSqlNosqlTypescriptJavascriptData IntegrationReact NativePythonpipeline buildingSite-reliability engineeringData engineering best practicesSoftware engineering best practicesPalantir Foundry

Pune, India

Skills:

GitGcpMySQLPostgreSQLAzurePythonSqlELTEtlAWS

Pune, India

Skills:

snowflake S3PysparkPl SqlJsonApache AirflowLambdaAws CloudPythonCsvSparksqlSqlDataframesRDDParquetAWS data pipeline developmentIcebergOrchestration toolsData ingestionLake FormationGlueDBT Data Build ToolSnowflake TasksAthenaAbinitio ETL toolAWS Data Analytics Technology Stack

Pune, India

Skills:

semantic modeling PysparkAzure DatabricksSqlAzure Data FactoryPandasFastAPIPythonMetrics Layer designDelta LakeGenAI-based BI workflowsDatabricks GeniePrompt EngineeringETL pipelines

Pune, India

Skills:

data preparation snowflake Bi ToolsData ModellingPower BiSQL ServerTableauSqlcloudOraclePythonEtlanalytics conceptsAlteryx Designerdata quality principlesRRelational DatabasesAlteryx Server