As a Data Engineer, you will be responsible for designing, implementing, and maintaining our data infrastructure to support our rapidly growing business needs.
The ideal candidate will have expertise in Apache Iceberg, Apache Hive, Apache Hadoop, SparkSQL, YARN, HDFS, MySQL, Data Modeling, Data Warehousing, Spark Architecture, and SQL Query Optimization.
Experience with Apache Flink, PySpark, Automated Data Quality testing & Data Migration is considered a plus.
Also, it's mandatory to know any one cloud stack (AWS or Azure) for Data Engineering to Create Data Jobs and Workflows and Scheduler it later for Automation.
Job Responsibilities & Requirements
Bachelor's degree in computer science, Information Technology, or a related field.
Master's degree preferred.
4-5 years of experience working as a Data Engineer.
Mandatory experience in PySpark Development for Big data processing.
Strong proficiency in Apache Iceberg, Apache Hive, Apache Hadoop, SparkSQL, YARN, HDFS, Data Modeling, and Data Warehousing.
Core PySpark Development and Optimizing SQL queries and performance tuning to ensure optimal data retrieval and processing.
Experience with Apache Flink, and Automated Data Quality testing is a plus.
It's mandatory to know any one cloud stack (AWS or Azure) for Data Engineering to Create Data Jobs and Workflows and Scheduler later for Automation.
