SENIOR DATA ENGINEER - Kafka

Years of Experience: 5-8 Years

Job Summary: At Allianz, we are focused on data-driven decision-making. As a Data Engineer, you will be part of the Data Development and Engineering (DDE) team, responsible for integrating data from various sources and building ETL pipelines on our Global Data Platform using MS Azure. You will collaborate closely with the Information Data Governance team and support data scientists and Business Intelligence teams in a fast-paced, agile environment.

Responsibilities:

• Develop scalable solutions using big data technologies.
• Analyze and organize raw data from diverse sources to build data systems and pipelines.
• Evaluate business needs to create and maintain optimal data pipeline architecture.
• Assemble large, complex data sets that meet functional and non-functional business requirements.
• Build analytics tools that provide actionable insights into key business performance metrics.
• Hands-on development experience with Hadoop, Spark, Hive, HBase, and Kafka.
• Experience with NoSQL databases like Cassandra and MongoDB.
• Collaborate with teams to resolve issues and undertake production support activities.
• Report and update progress, status, issues, and risks as per project processes.

Mandatory Skills:

• 4+ years of experience in data engineering and developing scalable solutions.
• Expert understanding of Azure Data lake and experience with SQL Server (T-SQL).
• Proficiency in at least two programming languages among Java, Scala, or Python.
• Strong understanding of data warehousing concepts, ETL/ELT processes, and reporting tools.
• Experience with Kafka Stream for meta-data driven data ingestion.
• High level of accuracy and attention to detail.
• Experience in coaching and mentoring less experienced engineers.
• Strong customer and value focus, driven to deliver business value.

Preferred Skills:

• Experience in the financial services or insurance sector.
• Understanding of Postgres databases.
• Experience with data visualization tools (e.g., Chartio).
• Understanding of Linux OS core principles, performance, and tuning.

Qualifications:

• Degree, MSc, or PhD in a numerate or scientific discipline (e.g., Computer Science, Physics, Mathematics).