Role Overview
We are seeking a skilled and passionate Data Engineer with deep expertise in Google Cloud Platform (GCP) and Google BigQuery. In this role, you will architect, build, and maintain the scalable data pipelines that are the foundation of our analytics and data science initiatives.
You will be responsible for the entire data lifecycle, from ingestion and processing to warehousing and serving. You will work on creating a reliable, efficient, and high-quality data ecosystem that empowers our data analysts, data scientists, and business leaders to make critical data-informed decisions.
Responsibilities
- Develop and maintain data pipelines and ETL/ELT processes using Python
- Design and implement scalable, high-performance applications
- Work collaboratively with cross-functional teams to define requirements and deliver solutions
- Develop and manage near real-time data streaming solutions using PubSub or Beam
- Contribute to code reviews, architecture discussions, and continuous improvement initiatives
- Monitor and troubleshoot production systems to ensure reliability and performance
Requirements
- Python
- Google Cloud Platform (GCP)
- Google BigQuery
- ETL/ELT processes
- SQL
- Data pipelines
- Database concepts
- CI/CD pipelines
- Cloud Logging/Monitoring
- Data processing frameworks (Pandas)
- Software design patterns
- Unit testing
- Test automation
- Infrastructure as code
- Docker
- Kubernetes
- Version control
Preferred Skills
- GCP Cloud Run
- GCP Dataflow
- Pub/Sub
- Apache Airflow
- Data visualization tools
- GitLab CI/CD
- Terraform
- Snowflake
- Databricks
- Apache Beam
Qualifications
- 3+ years of professional software development experience with Python
- Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)
- Proven track record of delivering complex software projects
- Excellent problem-solving and analytical thinking skills
- Strong communication skills and ability to work in a collaborative environment
- Experience working with Google Cloud Platform
- Strong understanding of software engineering best practices (testing, version control, CI/CD)
- Experience building and optimizing ETL/ELT processes and data pipelines
- Proficiency with SQL and database concepts
- Experience building CI/CD pipelines for data, monitoring systems (Cloud Logging/Monitoring), and ensuring security/compliance
- Experience with data processing frameworks (e.g., Pandas)
- Understanding of software design patterns and architectural principles
- Ability to write clean, well-documented, and maintainable code
- Experience with unit testing and test automation
- Experience with CI/CD pipelines and Infrastructure as code
- Experience with Containerization technologies like Docker or Kubernetes
- GCP Data Engineer certification (preferred)
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
