Data Engineer
Data Engineer
SaturamThis job is no longer accepting applications
Job Description
Requirements
- Strong programming skills in Python and SQL.
- Experience with data processing leveraging programming skills in Python and Spark.
- In-depth understanding of the Kafka platform for (real-time) data ingestion and processing of high-volume data.
- Design and architect data flows and data management in a Cloud environment that are scalable, repeatable, and eliminate time-consuming steps.
- Clear skills in using version control systems like GIT and developing (technical) documentation (e. g., in WIKI).
- Experiences with infrastructure components as code for robust deployment, replication, and uniform management.
- Effective collaboration and communication skills to engage with cross-functional teams.
- A proactive mindset with the ability to drive tasks to completion.
- Proficiency in Agile/Scrum/Kanban methodologies for efficient product delivery and management.
- Familiarity with AWS cloud services like Glue, Athena etc.
- Bachelor's or Master's degree in Computer Science/Information Systems.
- At least 4+ years of experience in building data flow and data management on a modern big data tech stack.
- Strong experience in using ETL frameworks (e. g., Airflow, Jenkins) to build and deploy production-quality ETL pipelines.
- Knowledge of data structures.
- Open to learning and implementing new technologies.
- Ability to think and perform data engineering workstreams with a product mindset.
- Fluency in English.
More Info
About Company
Skills:
Pyspark, AWS Glue, Data Modeling, Emr, Avro, Autosys, Sql, Cloudwatch, Terraform, Etl Tools, Databricks, Python, AWS, Airflow, Parquet, AWS Lake Formation, Iceberg
Skills:
Power Bi, Pyspark, Agile Scrum, Tableau, Databases, Git, Data Warehousing, Informatica Powercenter, Python, Airflow, Data Lake concepts, Advanced SQL and performance tuning, IICS, dbt, CI/CD
Skills:
Kms, Pyspark, Iam, Sql, Python, AWS Lake Formation
Skills:
data engineering , AWS Glue, Data Modeling, Sql, Etl, Step Functions, Airflow, IAM roles and permissions
Skills:
Scala, Python, Spark