JD - Data Engineer Tech Lead
As a Data Engineer Tech Lead, you will play a crucial role in designing, developing, and maintaining our data infrastructure and pipelines. You will lead a team of data engineers, ensuring the quality and efficiency of data processing, and supporting data-driven decision-making across the organization.
Years of Experience: 8-10 years
Job Summary: We are seeking an experienced Data Engineer Tech Lead to oversee our data engineering team. You will be responsible for building and optimizing data pipelines, ensuring data quality, and collaborating with various stakeholders to deliver data solutions that drive business insights.
Key Responsibilities
- Team Leadership: Lead and mentor a team of data engineers, providing technical guidance and support. Set clear objectives and priorities for the team and ensure the successful execution of projects. Foster a collaborative and innovative work environment while engaging with stakeholders, business leaders, and subject matter experts.
- Data Pipeline Development: Build and optimize data pipelines for collecting, processing, and storing data from various sources. Implement data transformation and enrichment processes to support analytics and reporting.
- Data Quality and Governance: Implement data quality checks and validation procedures to ensure data accuracy and consistency. Define and enforce data governance standards and best practices.
- Integration and Collaboration: Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and provide data solutions. Integrate data from multiple sources, including databases, APIs, and external data providers.
- Technology Stack: Stay up-to-date with the latest data engineering technologies and best practices. Select and manage the appropriate tools and technologies for data processing and storage, such as databases, data lakes, and ETL frameworks. Hands-on experience managing and working with Informatica, Snowflake, Java, Python, and cloud technologies is required.
- Performance Optimization: Monitor and optimize data pipelines for performance, reliability, and cost-effectiveness. Troubleshoot and resolve data-related issues as they arise.
- Documentation and Knowledge Sharing: Maintain thorough documentation of data processes, architectures, and workflows. Promote knowledge sharing within the team and across the organization.
Mandatory Skills
- Proven experience in data engineering with a focus on ETL, data modeling, and data pipeline development.
- Strong programming skills in Python (deal breaker skill).
- Expertise in working with data storage solutions like SQL and NoSQL databases, data lakes, and cloud-based storage.
- Familiarity with big data technologies and cloud platforms (e.g., AWS, Azure, GCP) with a minimum of 3 years of hands-on experience.
- Experience with cloud-based ETL solutions (e.g., AWS Glue, Azure Data Factory).
- Leadership and team management experience.
- Strong problem-solving and Communication Skills.
- Understanding of Data Governance and data quality best practices.
Preferred Skills
- Relevant certifications in data engineering or cloud platforms.
- Experience with real-time data processing and streaming technologies.
Note: The consultant will work with the global team and may need to align work timings as per business needs.
Python
