Interested suitable candidates please be in touch with
Priti Kumari<[Confidential Information]>
Job Location: Anywhere in India Preferably Mumbai/Hyderabad/Chennai/Bangalore/Bhopal/Kochi
Exp Level: 7+ Yrs
As a Data Engineer You'll Be
- Developing end to end ETL/ELT Pipeline on Azure / Databricks / DBT
- More than 3 years of extensive experience with DBT
- Designing, developing, and implementing scalable, automated processes for data extraction, processing, and analysis in a Data Mesh architecture
- Be a go-to expert for data technologies and solutions
- Ability to provide on the ground troubleshooting and diagnosis to architecture and design challenges
- Looking for ways of improving both what and how data pipelines are delivered
- Translating business requirements into technical requirements, such as entities that need to be modelled, DBT models that need to be build, timings, tests and reports
- Owning the delivery of data models and reports end to end
- Perform exploratory data analysis in order to identify data quality issues early in the process and implement tests to ensure prevent them in the future
- Working with Data Analysts to ensure that all data feeds are optimized and available at the required times. This can include Change Capture, Change Data Control and other delta loading approaches
- Discovering, transforming, testing, deploying and documenting data sources
- Applying, help defining, and championing data warehouse governance: data quality, testing, coding best practices, and peer review
- Building Looker Dashboard for use cases if required
What We Are Looking For
- You have 7+ years of extensive development experience using delta lake, snowflake or similar data warehouse technology
- You have experience in agile processes, such as SCRUM
- You have extensive experience in writing advanced SQL statements and performance tuning them
- Experience in working with databricks
- You have experience in data mining, data warehouse solutions, and ETL, and using databases in a business environment with large-scale, complex datasets
- You have experience architecting analytical databases (in Data Mesh architecture) is added advantage
- You have experience working in agile cross-functional delivery team
- You have high development standards, especially for code quality, code reviews, unit testing, continuous integration and deployment
- You have strong technical documentation skills and the ability to be clear and precise with business users
- You have business-level of English and good communication skills
- You have basic experience in Power BI, Tableau
- You have basic understanding of various systems across the Azure
- Experience in Insurance domain is a plus
- Experience with python, governance tool (e.g. Atlan, Alation, Collibra) or data quality tool (e.g. Great Expectations, Monte Carlo, Soda) will be added advantage
Our Tech Stack
- Azure
- DBT
- Databricks for delta lake and orchestration
- Informatica for data cataloging
- SQL
- Power BI, Tableau
