Primary Responsibilities:
- Use knowledge to architect data warehousing and BI solutions that solve problems that cut across multiple business functions
- Interact with business customers, gathering requirements and delivering complete Data Engineering, Data Warehousing, and analytics solutions.
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources
Job Requirements:
- Minimum 5-12 years of end-to-end Data Engineering Development experience – Total professional experience does not matter much. This is mainly for 5-10+ years experienced profiles.
- Worked extensively across different industry domain specially Retail, FMCG, Manufacturing, Finance, Oil & Gas, etc.
- Worked extensively across functional domains like Sales, Procurement, Cost Control, Business Development, Management Reporting, Finance, etc.
- 3 to 10 years of experience in working on data engineering projects using Azure / AWS services.
- Extensive hands-on experience implementing data transformation, processing and data migration using Azure or AWS services: ADLS, Azure Data Factory, Databricks Azure Functions, Synapse/DW, Azure SQL DB, Event Hub, Azure Stream Analytics, Databricks, AWS Glue, Redshift, Athena
- Exposure to MS Fabric and its components is added advantage.
- Experience in working with different source/target systems like Oracle Database, SQL Server Database, Blob Storage, Azure Data Lake Storage, ERP, CRM, SCM systems.
- Ability to read data from sources using APIs/Web Services and having worked with API to write data to target systems.
- Experience with Data Cleanup, Data Cleansing, and optimization related exercises.
- Experience working with non-structured data sets in Azure will be an added advantage.
- Experience with other analytics tools like Power BI, Azure Analysis service would be an advantage.
- Experience with private and public cloud architectures.
- Excellent written and verbal communication skills
Key Requirements -
- MS Azure Data Factory, Python, PySpark Coding, Synapse Analytics, Azure Function Apps, Azure Databricks or AWS Services – Glue, Athena, Redshift, Databricks Pysark
- MS Fabric exposure
- Integration to various applications/systems – ERP, CRM, SCM, WebApp using APIs, Cloud, On-premise systems, DBs, file systems.
- Min 3 Full Cycle Data Engineering Implementations (5-10 Yrs experience) with exposure to building data warehouses and implementing data models on projects.
- Exposure to consulting industry is a must.
- Excellent verbal and written communication skill.
