You should have a bachelors or masters degree in computer science, Information Technology or other quantitative fields
You should have at least 5 years working as a data engineer in supporting large data transformation initiatives related to machine learning, with experience in building and optimizing pipelines and data sets
Strong analytic skills related to working with unstructured datasets.
Experience with Azure cloud services: ADF, Azure Synapse, Blob Storage, ADLS, App Insights, and familiarity with various log formats from Azure.
Experience with object-oriented/object function scripting languages: Python, Py-spark, Java, etc.
Experience with big data tools: Hadoop, Spark, Kafka, etc.
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
You should be a good team player and committed for the success of team and overall project.
