Azure Data Engineering Advanced: Senior Data Engineer
Job requirements
Primary Duties and Responsibilities:
10+ years of SQL Server ETL Development experience using SSIS, Azure Data Factory and Azure Databricks. Strong ETL, data warehouse, T-SQL skills.
Understanding of data warehouse and data mart architecture, and Kimball methodology.
8+ years of strong hands-on experience building large-scale databases
Design, build and launch efficient & reliable data pipelines to move and transform data
Broad understanding and experience of data modeling, data management, ETL and BI
Experience in building enterprise data warehousing with table partitioning, encryption and data compression
Experience with cloud migration
Perform ongoing monitoring, automation and refinement of reports and BI solutions.
Work with BI Management to identify and resolve data related issues.
Work with business departmental teams to help them define and document business requirements for new metrics and reports.
Participate in process reviews and enhancements.
Provide analysis and issue resolution on business reported concerns.
Participate in design and delivery of cubes, dashboards and various self-service business intelligence solutions. Other:
Enforces established documentation guidelines and best practices.
Maintains a high degree of business functional knowledge and current automated technologies and tools.
Adheres to all policies and procedures concerning all confidential information including but not limited to internal use and restricted information Qualifications: Required
7+ years of experience in working as Data Engineer for Enterprise DWH and BI project for both on prem and cloud (Azure) with minimum of 3+ years of hands-on experience in working on Microsoft Technologies (SQL Server, Synapse, SSIS, Data Factory, Data Bricks)
Hands on experience on scripting language python, C# or equivalent
Advanced level experience on MS SQL Server, MySql , or Oracle
Experience with Big Data Technologies (NoSQL databases, Hive, Hbase , Pig or Spark)
Exposure and hands on exposure to ETL using SSIS, Data Factory
Experience and detailed knowledge with Master Data Management, ETL, Data Quality, metadata management, data profiling, micro-batches, streaming data loads.
Experience of using one/more scalable (MPP) relational databases such as Synapse or Snowflake for advanced analytics
Working knowledge of application and system availability, scalability and distributed data platforms
Strong experience in data warehousing technologies, ETL/ELT implementations
Knowledge and experience with full SDLC lifecycle
Experience with Lean / Agile development methodologies
Experience in providing technical leadership and other engineers for the best practices on the data engineering space. Desired
Experience in defining new architectures and ability to drive an independent project from an architectural stand point
Familiarity with data modeling and ETL tools such as Erwin, Talend etc.
Support users with troubleshooting, tuning queries (partition, bucket recommendations etc.),
Experience with design and implementation of ETL/ELT framework for complex warehouses/marts. Knowledge of large data sets and experience with performance tuning and troubleshooting
Hands-on development mentality, with a willingness to troubleshoot and solve complex problems
