Senior Data Engineer

Collaborate with stakeholders and cross-functional teams to gather and define data requirements and ensure alignment with business objectives.

Design and build production data engineering solutions on Google Cloud Platform (GCP) using services such as BigQuery, Dataflow, DataForm, Astronomer, Data Fusion, DataProc, Cloud Composer/Air Flow, Cloud SQL, Compute Engine, Cloud Functions, Cloud Run, Artifact Registry, GCP APIs, Cloud Build, App Engine, and real-time data streaming platforms like Apache Kafka and GCP Pub/Sub.

Partner with application, product, business and governance teams to develop and deliver based on business needs.

Partner with analytics teams to understand how value is created using data.

Design and implement batch, real-time streaming, scalable, and fault-tolerant solutions for data ingestion, processing, and storage.

Create insights into existing data to fuel the creation of new data products.

Perform necessary data mapping, impact analysis for changes, root cause analysis, and data lineage activities, documenting information flows.

Implement and champion an enterprise data governance model.

Actively promote data protection, sharing, reuse, quality, and standards to ensure data integrity and confidentiality.

Develop and maintain documentation for data engineering processes, standards, and best practices.

Ensure knowledge transfer and ease of system maintenance.

Utilize GCP monitoring and logging tools to proactively identify and address performance bottlenecks and system failures.

Provide production support by addressing production issues as per SLAs.

Optimize data workflows for performance, reliability, and cost-effectiveness on the GCP infrastructure.

Work within an agile engineering team.

Deliver code frequently using Test-Driven Development (TDD), continuous integration, and continuous deployment (CI/CD).

Continuously enhance your domain knowledge.

Stay current on the latest data engineering practices.

Contribute to the company's technical direction while maintaining a customer-centric approach.

GCP certified Professional Data Engineer

Successfully designed and implemented data warehouses and ETL processes for over five years, delivering high-quality data solutions.

5+ experience with programming languages such as Python, SQL, Apache ecocystem and GCP expertise, specializing in managing cloud infrastructure and applications to production-scale solutions.

In-depth understanding of GCP's underlying architecture and hands-on experience of crucial GCP services, especially those related to data processing (Batch/Real Time) leveraging Terraform, Big Query, Dataflow, Pub/Sub, Data form, astronomer, Data Fusion, DataProc, Pyspark, Cloud Composer/Air Flow, Cloud SQL, Compute Engine, Cloud Functions, Cloud Run, Cloud build and App Engine, alongside and storage including Cloud Storage

DevOps tools such as Tekton, GitHub, Terraform, Docker.

Expert in designing, optimizing, and troubleshooting complex data pipelines.

Experience developing and deploying microservices architectures leveraging container orchestration frameworks

Experience in designing pipelines and architectures for data processing.

Passion and self-motivation to develop/experiment/implement state-of-the-art data engineering methods/techniques.

Self-directed, work independently with minimal supervision, and adapts to ambiguous environments.

Evidence of a proactive problem-solving mindset and willingness to take the initiative.

Strong prioritization, collaboration & coordination skills, and ability to simplify and communicate complex ideas with cross-functional teams and all levels of management.

Proven ability to juggle multiple responsibilities and competing demands while maintaining a high level of productivity.

Master's degree in computer science, software engineering, information systems, Data Engineering, or a related field.

Experience in coaching and mentoring Data Engineers

Project management tools like Atlassian JIRA

Experience working in an implementation team from concept to operations, providing deep technical subject matter expertise for successful deployment.

Experience with data security, governance, and compliance best practices in the cloud.