GCP Data EngineerIntermediate
GCP Data EngineerIntermediate
infovision inc.- Posted 12 hours ago
- Be among the first 10 applicants
Job Description
- We are seeking a skilled GCP Data Engineer with 3 to 7 years of experience to join our growing data engineering team in Chennai/Hyderabad/Bangalore/Pune, focused on building and maintaining scalable cloud-based data pipelines and infrastructure on Google Cloud Platform.
- The ideal candidate will have hands-on expertise in GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Dataproc
- Proficiency in leveraging Claude AI and other LLM-based tools for intelligent data transformation, automation of data workflows, and natural language query generation is highly desirable.
- Strong programming skills in Python are required for developing ETL/ELT pipelines, data ingestion scripts, and custom data transformation logic across batch and streaming workloads.
- Solid command of SQL is essential for writing complex queries, optimizing data models in BigQuery, and supporting analytics and reporting teams with structured data access layers.
- Candidates must have experience designing and implementing data lakehouse architectures, dimensional data models, and governance frameworks that ensure data quality, lineage, and security compliance.
- Experience working in agile development environments with CI/CD pipelines, version control using Git, and infrastructure-as-code tools such as Terraform for GCP resource provisioning is expected.
- Familiarity with data orchestration platforms such as Apache Airflow or Cloud Composer and monitoring tools like Cloud Logging and Cloud Monitoring will be considered a strong advantage.
Roles & Responsibilities
- Design, develop, and optimize end-to-end data pipelines on GCP using BigQuery, Dataflow, Cloud Composer, and Pub/Sub to support real-time and batch data processing requirements.
- Integrate multi-cloud data sources across GCP and AWS(nice to have) environments, ensuring seamless data movement, transformation, and availability for downstream analytics and machine learning teams.
- Utilize Claude AI and generative AI tools to automate data catalog creation, generate SQL queries from natural language inputs, and enhance the efficiency of data operations workflows.
- Write and maintain high-quality Python scripts for data ingestion, transformation, and pipeline orchestration, ensuring code is modular, well-documented, and production-ready.
- Develop and maintain complex SQL queries and stored procedures in BigQuery to support business intelligence dashboards, ad hoc analysis, and scheduled reporting pipelines.
- Collaborate with data architects, analysts, and ML engineers to translate business requirements into robust data models and ensure data accuracy, freshness, and consistency across the platform.
- Monitor and troubleshoot data pipeline failures, performance bottlenecks, and data quality issues using GCP-native observability tools and proactively implement preventive measures.
- Participate in code reviews, contribute to data engineering best practices, and mentor junior team members to uphold high engineering standards across the data platform.
More Info
Key Skills
About Company