Senior Principal Consultant - Data Engineer

Inviting applications for the role of Senior Principal Consultant - Data Engineer

In this role you are responsible for Design, implement, and deploy data loaders to load data into the Engineering Sandbox.

Responsibilities:

• Pull/ingest data from different sources, transform and stitch, and wrangle it for advanced analytics activities.

• Design, implement, and deploy data loaders to load data into the Engineering Sandbox.

• Leverage data best practices and tools and assist ML engineer in pulling, filtering, tagging, joining, parsing, and normalizing data sets for use.

• Provide input to ML engineer/cloud engineer for the design and implementation of data management and/or architecture solutions.

• Implement data integration and transformation solutions using Azure Databricks.

• Collaborate with cross-functional teams to gather requirements and understand data needs.

• Develop and deploy data models and solutions using Azure services.

• Monitor and optimize data pipelines for performance and reliability.

• Troubleshoot and resolve data-related issues in a timely manner.

• Implement data security and privacy measures to protect sensitive information.

• Collaborate with data scientists and analysts to support their data requirements.

• Participate in code reviews and provide feedback to improve data engineering processes.

• Stay updated with the latest trends and technologies in data engineering and Azure ecosystem.

Key Skill Sets:

• Strong experience with Databricks , Azure ADF, Python ,Pyspark

• Proficient in SQL and data modeling concepts.

• Familiarity with data warehousing and data lakes concepts.

• Strong problem-solving and troubleshooting skills.

• Excellent communication and collaboration skills.

Minimum Qualifications:

• Bachelor%27s degree in Computer Science, Information Systems, or a related field.

• Proven experience as a Data Engineer or similar role.

• Hands-on experience with Databricks , Azure ADF, Python ,Pyspark

• Proficient in SQL and data modeling concepts.

Preferred Qualifications:

• Master%27s degree in a related field.

• Experience with big data technologies like Hadoop, Spark, or Hive.

• Knowledge of programming languages like Python or Java.

• Familiarity with machine learning concepts and algorithms.

• Certifications in Azure data engineering or related fields.