MRSI Snowflake Data Migration : Data Brick Engineer : B2\/C1 Copy

• 7+ years of hands-on experience in Python with a strong focus on data engineering and big data technologies.

Role Overview

The Python Big Data Engineer will design, develop, and optimize scalable data processing solutions using modern cloud and big data platforms. The role involves working with large datasets, implementing lakehouse architectures, and ensuring high-performance data pipelines.

Key Responsibilities

• Develop and maintain data processing pipelines using Python with Pandas and NumPy.
• Build and optimize big data workflows using Spark, PySpark, and Spark SQL.
• Design and implement lakehouse solutions using Delta Lake and Medallion Architecture.
• Work extensively on Databricks, including Unity Catalog for data governance and security.
• Develop and manage data ingestion and transformation pipelines using Azure Data Factory.
• Handle large-scale data stored in ADLS and process files in Parquet and JSON formats.
• Integrate and manage cloud data solutions with Snowflake.
• Ensure data quality, performance tuning, and optimization of ETL processes.
• Collaborate with analytics and business teams to translate requirements into scalable data solutions.
• Implement best practices for data modeling, storage, and lifecycle management.