Responsibilities:Qualifications
- Lead the migration and modernization of data platforms, moving applications and pipelines to Azure-based solutions.
- Actively contribute to code development in projects and services.
- Manage and scale data pipelines from internal and external data sources to support new product launches and ensure high data quality.
- Develop automation and monitoring frameworks to capture key metrics and operational KPIs for pipeline performance.
- Implement best practices around systems integration, security, performance, and data management.
- Collaborate with internal teams, including data science and product teams, to drive solutioning and proof-of-concept (PoC) discussions.
- Develop and optimize procedures to transition data into production.
- Define and manage SLAs for data products and operational processes.
- Prototype and build scalable solutions for data engineering and analytics.
- Research and apply state-of-the-art methodologies in data and Platform engineering.
- Create and maintain technical documentation for knowledge sharing.
- Develop reusable packages and libraries to enhance development efficiency.
Qualifications:
- Bachelor's degree in Computer Science, MIS, Business Management, or related field
- 10 + years experience in Information Technology
- 4 + years of Azure, AWS and Cloud technologies
- Experience in data platform engineering, with a focus on cloud transformation and modernization.
- Strong knowledge of Azure services, including Databricks, Azure Data Factory, Synapse Analytics, and Azure DevOps (ADO).
- Proficiency in SQL, Python, and Spark for data engineering tasks.
- Hands-on experience building and scaling data pipelines in cloud environments.
- Experience with CI/CD pipeline management in Azure DevOps (ADO).
- Understanding of data governance, security, and compliance best practices.
- Experience working in an Agile development environment.
- Prior experience in migrating applications from legacy platforms to the cloud.
- Knowledge of Terraform or Infrastructure-as-Code (IaC) for cloud resource management.
- Familiarity with Kafka, Event Hubs, or other real-time data streaming solutions.
- Experience with lagacy RDBMS (Oracl, DB2, Teradata)
- Background in supporting data science models in production.
