Key Skills: GCP, GCP Data stack, Cloud (AWS / Azure / GCP),Data Engineer
Roles and Responsibilities:
- Design, develop, and maintain batch and streaming data pipelines on Google Cloud Platform.
- Build, optimize, and manage BigQuery datasets, tables, views, and complex SQL queries.
- Develop data pipelines using Dataflow (Apache Beam), Pub/Sub, and Cloud Storage.
- Perform data ingestion from diverse data sources including APIs, databases, and files.
- Ensure data accuracy, consistency, validation, monitoring, and robust error handling.
- Optimize data pipelines for performance, scalability, and cost efficiency.
- Implement IAM, security controls, and compliance standards across data platforms.
- Automate and schedule workflows using Cloud Composer (Airflow).
- Collaborate with business and technical stakeholders to translate requirements into scalable data solutions.
Skills Required:
- Strong experience working with Google Cloud Platform and its data engineering services.
- Hands-on experience with the GCP data stack including BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
- Strong understanding of batch and streaming data processing concepts and architectures.
- Experience building and optimizing SQL queries, datasets, tables, and views in BigQuery.
- Hands-on experience with data ingestion from multiple sources such as APIs, databases, and flat files.
- Experience ensuring data quality through validation, monitoring, and error-handling mechanisms.
- Strong understanding of cloud performance optimization and cost management practices.
- Experience implementing IAM, security, and compliance best practices in cloud environments.
- Ability to work with workflow orchestration tools such as Cloud Composer (Airflow).
- Good communication skills with the ability to collaborate effectively with cross-functional teams.
Education:Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, along with strong practical experience in Google Cloud Platform and data engineering practices.
