Data Engineer

Job Summary

• Liaising with coworkers and clients to elucidate the requirements for each task.
• Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed.
• Reformulating existing frameworks to optimize their functioning.
• Testing such structures to ensure that they are fit for use.
• Preparing raw data for manipulation by data scientists.
• Detecting and correcting errors in your work.
• Ensuring that your work remains backed up and readily accessible to relevant coworkers.
• Remaining up-to-date with industry standards and technological advancements that will improve the quality of your outputs.
• Experience with Azure ADLS, Apache Parquet, Iceberg, Kubeflow and Airflow