Data - Senior Engineer

Job Description

Job Duties & Responsibilities

Design, develop, and optimize complex data pipelines and transformation processes using Snowflake, dbt, and AWS services.

Databricks is Mandatory

Implement and manage data integration workflows using Fivetran to ensure timely and accurate data ingestion from various sources.

Develop and maintain scalable data models and schemas in Snowflake, ensuring they meet performance and business requirements.

Monitor and fine-tune the performance of data pipelines, queries, and data models to ensure optimal efficiency and cost-effectiveness.

Utilize Snowflake's features, such as Time Travel, Zero-Copy Cloning, and Data Sharing, to enhance data management and performance.

Leverage AWS services, such as AWS Lambda, S3, and Glue, to build and manage serverless data processing workflows and data storage solutions.

Implement data security measures and ensure compliance with data privacy regulations and organizational policies.

Troubleshoot and resolve complex data issues, including data sync errors, performance bottlenecks, and integration challenges.

Provide support for data-related incidents and ensure effective resolution of production issues.

Collaborate with data analysts, and other stakeholders to understand data needs and deliver effective solutions.

Document data processes, models, and workflows, ensuring clear communication and knowledge sharing across teams.

Independently assess situations, apply sound judgment and discretion, and make decisions on matters of significant impact without direct supervision

Basic Qualifications

• Bachelor's Degree in Computer Science, Data Science, Information Technology, Engineering, or a related field. Snowflake certification a plus.
• 5-7 years of experience in data engineering, with a strong focus on data transformation, integration, and cloud-based data solutions
• Experience in writing complex SQL queries
• 5 years of experience on Snowflake platform.
• Experience in data modeling and writing data transformation with dbt platform, Tableau data prep, etc.
• Experience in building connector with Fivetran platform.
• Experience in writing test automation scripts.
• Experience using/understanding Snowflake Warehouses, reader accounts, SSO Setup, and data masking policies.
• Implemented Change Data Capture (CDC) and Change Data Tracking mechanisms in data pipelines.
• Experience designing and optimizing data ingestion pipelines to handle large-volume datasets.
• Experience writing complex data transformation logic using Advanced SQL query skills such as complex joins, sub-queries, grouping, aggregation, and stored procedures.
• Experience with programming languages, especially Python and Java.
• Designed and/or managed data warehousing concepts, methodologies, and best practices.
• Excellent oral and written communication skills including the ability to speak in front of large groups.

Preferred Qualifications

• Data Architecture & Data modeling experience in Clinical trials domain / Life Sciences.
• Knowledge of data management tools and process Data governance tools is a plus.
• Working experience with version control platforms, e.g. Github, and agile methodologies and supporting tools JIRA.
• Ability to handle stress and interact with others in a professional manner.
• Comfort working in a geographically distributed team-based environment.

Mandatory Competencies