Data Engineer

Primary Responsibilities:

• Use knowledge to architect data warehousing and BI solutions that solve problems that cut across multiple business functions
• Interact with business customers, gathering requirements and delivering complete Data Engineering, Data Warehousing, and analytics solutions.
• Interface with other technology teams to extract, transform, and load data from a wide variety of data sources

Job Requirements:

• Minimum 5-12 years of end-to-end Data Engineering Development experience – Total professional experience does not matter much. This is mainly for 5-10+ years experienced profiles.
• Worked extensively across different industry domain specially Retail, FMCG, Manufacturing, Finance, Oil & Gas, etc.
• Worked extensively across functional domains like Sales, Procurement, Cost Control, Business Development, Management Reporting, Finance, etc.
• 3 to 10 years of experience in working on data engineering projects using Azure / AWS services.
• Extensive hands-on experience implementing data transformation, processing and data migration using Azure or AWS services: ADLS, Azure Data Factory, Databricks Azure Functions, Synapse/DW, Azure SQL DB, Event Hub, Azure Stream Analytics, Databricks, AWS Glue, Redshift, Athena
• Exposure to MS Fabric and its components is added advantage.
• Experience in working with different source/target systems like Oracle Database, SQL Server Database, Blob Storage, Azure Data Lake Storage, ERP, CRM, SCM systems.
• Ability to read data from sources using APIs/Web Services and having worked with API to write data to target systems.
• Experience with Data Cleanup, Data Cleansing, and optimization related exercises.
• Experience working with non-structured data sets in Azure will be an added advantage.
• Experience with other analytics tools like Power BI, Azure Analysis service would be an advantage.
• Experience with private and public cloud architectures.
• Excellent written and verbal communication skills

Key Requirements -

• MS Azure Data Factory, Python, PySpark Coding, Synapse Analytics, Azure Function Apps, Azure Databricks or AWS Services – Glue, Athena, Redshift, Databricks Pysark
• MS Fabric exposure
• Integration to various applications/systems – ERP, CRM, SCM, WebApp using APIs, Cloud, On-premise systems, DBs, file systems.
• Min 3 Full Cycle Data Engineering Implementations (5-10 Yrs experience) with exposure to building data warehouses and implementing data models on projects.
• Exposure to consulting industry is a must.
• Excellent verbal and written communication skill.