We're seeking a highly skilled Data Engineer with a strong background in processing large-scale data.
Key Qualifications & Responsibilities
- Data Processing Expertise: You'll be primarily focused on processing data pipelines using Databricks Spark SQL on Hadoop distributions such as AWS EMR, Databricks, or Cloudera.
- Large-Scale Data Operations: You should be highly proficient in conducting large-scale data operations using Databricks and be very comfortable using Python.
- Cloud & AWS Familiarity: You'll need familiarity with AWS compute, storage, and IAM concepts. This includes hands-on experience with S3 Data Lake as the storage tier.
- Problem Solving & Optimization: You'll carefully evaluate alternative risks and solutions before taking action, optimize the use of all available resources, and develop solutions that align with business needs and objectives.
- Collaboration & Communication: We're looking for excellent collaboration and cross-functional leadership skills, along with strong written and verbal communication.
- Analytical Prowess: You should possess exceptionally strong analytical and problem-solving skills.
- ETL & Data Warehousing: Relevant experience with ETL methods and retrieving data from dimensional data models and data warehouses is essential. You'll also need strong experience with relational databases and data access methods, especially SQL.
- Migration Knowledge: You should have working knowledge of migrating relational and dimensional databases on the AWS Cloud platform.
- Adaptability: We value your ability to manage multiple initiatives and priorities in a fast-paced, collaborative environment, and leverage data assets to respond to complex questions requiring timely answers.
Good to Have
- Any ETL background (e.g., Talend, AWS Glue) is a plus, but not required.
- Experience with Cloud Warehouse solutions (e.g., Snowflake) is a huge plus.
