We are looking for Data Engineer ( AWS, Confluent & Snaplogic )
- Data IntegrationIntegrate data from various Siemens organizations into our data factory, ensuring seamless data flow and real-time data fetching.
- Data ProcessingImplement and manage large-scale data processing solutions using AWS Glue, ensuring efficient and reliable data transformation and loading.
- Data StorageStore and manage data in a large-scale data lake, utilizing Iceberg tables in Snowflake for optimized data storage and retrieval.
- Data TransformationApply various data transformations to prepare data for analysis and reporting, ensuring data quality and consistency.
- Data ProductsCreate and maintain data products that meet the needs of various stakeholders, providing actionable insights and supporting data-driven decision-making.
- Workflow ManagementUse Apache Airflow to orchestrate and automate data workflows, ensuring timely and accurate data processing.
- Real-time Data StreamingUtilizeConfluent Kafkafor real-time data streaming, ensuring low-latency data integration and processing.
- ETL ProcessesDesign and implement ETL processes usingSnapLogic, ensuring efficient data extraction, transformation, and loading.
- Monitoring and LoggingUse Splunk for monitoring and logging data processes, ensuring system reliability and performance.
- Youd describe yourself as:
- Experience3+ relevant years of experience in data engineering, with a focus on AWS Glue, Iceberg tables, Confluent Kafka, SnapLogic, and Airflow.
Technical Skills:
- Proficiency in AWS services, particularly AWS Glue.
- Experience with Iceberg tables and Snowflake.
- Knowledge of Confluent Kafka for real-time data streaming.
- Familiarity with SnapLogic for ETL processes.
- Experience with Apache Airflow for workflow management.
- Understanding of Splunk for monitoring and logging.
- Programming SkillsProficiency in Python, SQL, and other relevant programming languages.
- Data ModelingExperience with data modeling and database design.
- Problem-SolvingStrong analytical and problem-solving skills, with the ability to troubleshoot and resolve data-related issues.
