Lead and deliver the projects within APJ Data Engineering Team
Perform source to target data analysis and mappings.
Collaborate with admins, developers, data engineers and analysts for successful functionality delivery.
Perform requirement analysis and co-ordinate with project managers and development team to drive the delivery cycle.
Collaborate with the scrum master on product backlogs and helps with sprint planning.
Improve/Support solutions using Pyspark/EMR, SQL and databases, AWS Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other Data Engineering technologies.
Write Complex Queries and edit them as required for implementing ETL/Data solutions.
Handle database migration projects from Redshift to Data Lake S3, ensuring seamless transition and minimal disruption to operations.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Your main tasks will involve using Databricks, advanced SQL, and Power BI to achieve this goal.
Keys to Success:
Education
Bachelor's degree in Computer Science with at least 6 years of Data Engineering experience using AWS services, Pyspark/EMR.
Having Certifications like AWS Certified Data Analytics, CCA Spark and Hadoop Developer or CCP Data Engineer is recommended.
Experience
6+ Years of Experience in Databricks, Data Analytics & Business Intelligence Solutions and Technical Lead.
Full life cycle project implementation experience in AWS using Databricks Pyspark/EMR, Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other handled services
Good experience on Power BI.
Experience with agile development methodologies by following DevOps, Data Ops and Dev Sec Ops practices.
Experience using Jira for task prioritization and Confluence and other tools for documentation.
Strong analytical experience with database in writing complex queries, query optimization, debugging, user defined functions, views, indexes etc.
Knowledge, Skills, Abilities
Strong communication skills in both written and verbal forms, with a focus on effective collaboration with others, including customers, partners, and team members.
Excellent prioritization and problem-solving skills.
Learn and train other team members
Aim to meet and exceed BI Operational SLAs for Service Now incidents, Major Incidents, xMatters alerts, Employee Experience Metrics and BI application /process availability metrics.
