Data Engineering - Developer

Job description

Responsibilities

• Design, build, test and deploy scalable and reusable systems that handle large amounts of data.
• Collaborate with product owners and data scientists to build new data products.
• Ensure data quality and reliability

Qualifications

• Experience designing and managing data flows.
• Experience designing systems and APIs to integrate data into applications.
• 4+ years of Linux, Bash, Python, and SQL experience
• 2+ years using Spark and other frameworks to process large volumes of data.
• 2+ years using Parquet, ORC, or other columnar file formats.
• 2+ years using AWS cloud services, esp. services that are used for data processing e.g. Glue, Dataflow, Data Factory, EMR, Dataproc, HDInsights , Athena, Redshift, BigQuery etc.
• Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges