Lead GCP Data Engineer

Educational Qualification

• Bachelor's degree in Computer Science, or a related field, or equivalent experience

Key Responsibilities

• Design, develop, test, and maintain scalable ETL data pipelines using Python
• Architect enterprise solutions with technologies including Kafka, GCP services, GKE, Load Balancers, APIGEE, DBT, LLMs, data redaction, and DLP solutions
• Build and manage data workflows on Google Cloud Platform using Dataflow, Cloud Functions, BigQuery, Cloud Composer, Cloud Storage, IAM, and Cloud Run
• Implement data ingestion, transformation, and cleansing to ensure high-quality data delivery
• Develop and enforce data quality checks, validation rules, and monitoring processes
• Collaborate with data scientists, analysts, and engineering teams to deliver efficient data solutions
• Manage version control using GitHub and participate in CI/CD pipeline deployments
• Write complex SQL queries for data extraction and validation from relational databases (SQL Server, Oracle, PostgreSQL)
• Document pipeline designs, data flow diagrams, and operational support procedures

Required Skills & Competencies

• Strong hands-on experience in Python for backend or data engineering projects
• Expertise in GCP services: Dataflow, BigQuery, Cloud Functions, Cloud Composer, Cloud Storage, Cloud Run, IAM
• Experience in data pipeline architecture, data integration, and transformations
• Proficiency in Apache Spark, Kafka, Redis/Bigtable, FastAPI, and Airflow
• Strong SQL skills with at least one enterprise database (SQL Server, Oracle, PostgreSQL)
• Experience in CI/CD practices and version control (GitHub)

Good to Have / Optional Skills

• Experience with Snowflake cloud data platform
• Hands-on knowledge of Databricks for big data processing and analytics
• Familiarity with Azure Data Factory (ADF) and Azure data engineering tools