Role Purpose
The Data Engineer is tasked with overseeing the full life-cycle of back-end development
of the business's data requirements/products. The Data Engineer is responsible for the
development of ETL processes and various administration and data structural designs.
The Data Engineer works closely with various internal stakeholders, including, but not
limited to data analysts, data scientists and business members in order to power insight
and avail meaningful data products for the business and enable consistently informed
management decisions
Requirements
- 6+ years of experience in a similar role
- Experience in PowerBI and Databricks is mandatory
- Experience with Databases (Postgres, SQL Server, Oracle, MySQL)
- Experience with cloud software (AWS - primary, Azure or Google - secondary).
- Data Engineering/Warehousing experience (current)
- SQL advanced coding
Duties and Responsibilities
KEY ACCOUNTABILITIES / KRAs / KPIs
ETL: Develop and maintain the ETL processes:
- Identify, analyse, design and maintain the data sources and pipelines.
- Do necessary checks to ensure that the tables loaded balances back to the
operational system in terms of control totals and record counts as per Audit
requirements.
- Develop and maintain procedures and standards for processes e.g. (Extraction and
loading of data, Coding and naming standards, Scheduling of data related scripts
and jobs, Develop Change control standards)
Database Maintenance
- Identifying potential improvements to the design of the database and warehouses
and ETL processes.
- Maintain security and access to the database.
- Monitor the database and tables space usage.
- Archive historical data and delete unused tables.
- DB performance monitoring and performance.
- Data Statistics and Reporting.
Effective Project Delivery
- Migration
- Upgrades
- New system developments and projects.
- Internal and external auditing if required.
- Effective project delivery - Delivery of projects on time within the agreed budgets.
- Ensure Continuous Service Improvement (CSI).
- Achieving Operational Excellence.
Innovation
- Promote new techniques and Data tools that shape the future of Data in the
business, i.e. Unstructured data, Data Lakes, Apache Hadoop/ Spark/Hive/Kafka.
Objectives
- Team Relationship management - Engaging and partnering with other IT and
Business teams.
- Ensuring effective service delivery to our Business Partners.
- Manage own performance and development.
- Learn, grow, and enjoy contributing to a platform and team which aims at growing
the business and solving complex problems.
As an applicant, please verify the legitimacy of this job advert on our company career page.
