We are seeking a skilled Data Engineer to design, build, and maintain data pipelines and data models that support analytical and business intelligence needs. The ideal candidate will have hands-on experience with Python or SQL, Google Cloud Platform (GCP), and a strong understanding of data management, quality, and security best practices.
Key Responsibilities
- Build and maintain moderately complex data pipelines, ensuring data flow, transformation, and usability for analytical projects.
- Design and implement data models, optimizing for performance and scalability.
- Apply knowledge of data characteristics and supply patterns to develop rules and tracking processes that support data quality models.
- Prepare data for analytical use by gathering, integrating, cleansing, and structuring data from multiple sources and systems.
- Perform design, creation, and interpretation of large and highly complex datasets.
- Troubleshoot pipeline and data issues to ensure accuracy and reliability.
- Stay up-to-date with GCP advancements and recommend innovative solutions.
- Implement security best practices within data pipelines and cloud infrastructure.
- Collaborate with global teams to share and adopt best practices in data management, maintenance, reporting, and security.
- Develop and execute data quality checks to ensure consistency and integrity.
- Work with credit data products and perform analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark.
Mandatory Skills
- Python or SQL Proficiency: Experience with Python or SQL and intermediate scripting for data manipulation and processing.
- GCP & Cloud Fundamentals: Intermediate understanding and experience with Google Cloud Platform (GCP) and overall cloud computing concepts.
- Data Pipeline Construction: Proven ability to build, maintain, and troubleshoot moderately complex pipelines.
- Data Modeling & Optimization: Experience designing and optimizing data models for performance.
- Data Quality Governance: Ability to develop rules, tracking processes, and checks to support a data quality model.
- Data Preparation & Structuring: Skilled in integrating, consolidating, cleansing, and structuring data for analytical use.
- Security Implementation: Knowledge of security best practices in pipelines and cloud infrastructure.
- Big Data Analysis Tools: Hands-on experience with Google BigQuery, BigTable, DataFlow, Scala + Spark or PySpark.
- Advanced Data Formats: Experience working with JSON, AVRO, and PARQUET formats.
- Communication & Best Practices: Strong communication skills to promote global best practices and guide adoption.
Preferred Qualifications
- Cloud certification (e.g., GCP Data Engineer, AWS, Azure).
- Experience with credit data products.
- Familiarity with data governance frameworks and metadata management tools.
Skills: security,cloud,data,data models,pipelines,data quality,gcp
