Search by job, company or skills

M

Data Engineer

1-4 Years

This job is no longer accepting applications

new job description bg glownew job description bg glow
  • Posted 2 months ago
  • Over 200 applicants

Job Description

MSCI Services is looking for a Data Engineer to join our team. This role focuses on the design, development, and maintenance of robust and efficient data pipelines. You will be instrumental in optimizing data processing for large datasets and collaborating with cross-functional teams to deliver high-quality data solutions.

Responsibilities:

  • Design, develop, and maintain data pipelines and ETL/ELT processes using PySpark/Databricks/BigQuery/Airflow/Composer.
  • Optimize performance for large datasets through techniques such as partitioning, indexing, and Spark optimization.
  • Collaborate with cross-functional teams to resolve technical issues and gather requirements.
  • Ensure data quality and integrity through data validation and cleansing processes.
  • Analyze existing SQL queries, functions, and stored procedures for performance improvements.
  • Develop database routines like procedures, functions, and views/MV.
  • Participate in data migration projects and understand technologies like Delta Lake/Warehouse/BigQuery.
  • Debug and solve complex problems in data pipelines and processes.

Qualifications:

  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Strong understanding of distributed data processing platforms like Databricks and BigQuery.
  • Proficiency in Python, PySpark, and SQL programming languages.
  • Experience with performance optimization for large datasets.
  • Strong debugging and problem-solving skills.
  • Fundamental knowledge of cloud services, preferably Azure or GCP.
  • Excellent communication and teamwork skills.

Nice to Have:

  • Experience in data migration projects.
  • Understanding of technologies like Delta Lake/Warehouse.

More Info

Job Type:
Function:
Employment Type:

About Company

MSCI is a leading provider of critical decision support tools and services for the global investment community. With over 50 years of expertise in research, data and technology, we power better investment decisions by enabling clients to understand and analyze key drivers of risk and return and confidently build more effective portfolios. We create industry-leading research-enhanced solutions that clients use to gain insight into and improve transparency across the investment process.

Job ID: 120339513

Similar Jobs

Navi Mumbai, Mumbai, India

Skills:

Spark SQLPerformance TuningKafkaAzure DatabricksData Warehousing ConceptsQuery OptimizationData ModellingGovernance and Access managementSpark Structured StreamingWindow functionscluster configurationMedallion ArchitectureData management strategiesDataFramescaching strategiesUnity CatalogForeign catalogDatabricks AutoloaderDelta Lake

Mumbai, India

Skills:

unstructured data Machine LearningPythondata science fundamentalsstructured datadata pipelinesdata-intensive problemsData Processingcode reviews

Navi Mumbai, Mumbai, India

Skills:

Api DevelopmentPythonBig Data TechnologiesCloud-based big data technologiesRTraditional Application DevelopmentAnalytics librariesOpen-Source Natural Language ProcessingStatistical and big data computing libraries

Mumbai, India

Skills:

GitlabSqlSQL ServerDataFlowPysparkRest ApisAWSMySQLCloudformationOracleSpark SQLAzureGcpTerraformJenkinsPostgreSQLWeb ServicesAzure DevOpsInformatica Cloud Services IDMC IICSDelta LakeCI CD pipelines

Mumbai, India

Skills:

S3LambdaGitCloudwatchECSPythonSqlAWSGlue