Here Are The Skill Sets Client Is Looking For
Note: The selected candidate must be available to work with at least 45 hours of overlap with CST to support team collaboration.
Job Summary
Responsible for designing, building, and optimizing complex data systems and pipelines that support the organization's data analytics and business intelligence needs. This role involves leading data engineering projects, providing technical expertise, and ensuring the reliability and scalability of data infrastructure. Will work closely with cross-functional teams to deliver high-quality data solutions that enhance decision-making and operational efficiency.
Job Duties
- Design, develop, and maintain complex ETL (Extract, Transform, Load) pipelines to manage large-scale data processing and integration. Optimize data workflows for performance, scalability, and reliability, ensuring that they meet the organization's evolving needs.
- Contribute to the design and implementation of advanced data architectures, including data lakes, data warehouses, and real-time data processing systems. Develop sophisticated data models and data integration frameworks to support various business functions.
- Oversee the management and optimization of relational and NoSQL databases. Implement advanced indexing, partitioning, and performance tuning techniques to enhance database efficiency and handle large volumes of data.
- Establish and enforce data quality standards and governance practices. Implement robust data validation and cleansing processes to ensure data accuracy, consistency, and integrity across systems.
- Provide technical leadership and mentorship to junior data engineers and other team members. Review code, designs, and architectures to ensure best practices and high-quality deliverables.
- Lead and manage complex data engineering projects from inception to completion. Coordinate with project managers, business stakeholders, and other teams to ensure projects are delivered on time and within scope.
- Continuously monitor the performance of data pipelines, systems, and databases. Identify and resolve performance bottlenecks, optimize data processing and storage solutions, and implement best practices for system efficiency.
- Leverage advanced data engineering tools and technologies, such as Apache Kafka, Apache Airflow, and cloud-based platforms (e.g., AWS Redshift, Google BigQuery). Stay updated on emerging tools and technologies to drive innovation and improvements.
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and provide solutions that meet their needs. Communicate complex technical concepts clearly to non-technical stakeholders.
- Ensure that data engineering practices adhere to security policies and compliance requirements. Implement measures to protect sensitive data and ensure data privacy and regulatory compliance.
Experience
- Bachelor's degree or educational experience in related field preferred, or relevant work experience.
- 4-6 years experience with database management systems, including both relational and NoSQL databases. Strong Microsoft SQL Server experience required.
- Expertise in programming languages such as SQL, Python, Java, or SQL, and experience with data engineering tools and platforms like Apache Kafka, Apache Airflow, or cloud-based data services (e.g., AWS, Azure, Google Cloud).
- Experience in leading data engineering projects and providing technical leadership within a large or complex organization.
- Familiarity with data warehousing solutions, data lake architectures, and real-time data processing technologies.
- Experience with data visualization tools and business intelligence platforms.
- Advanced proficiency in SQL and extensive experience with various database technologies, including relational and NoSQL databases.
- Expertise in data engineering tools and platforms for ETL, data processing, and integration, including big data technologies and cloud-based services.
- Strong knowledge of data governance frameworks, data quality management, and compliance requirements.
- Excellent analytical and problem-solving skills, with the ability to address complex data challenges and optimize data systems.
- Strong leadership and mentorship abilities, with experience guiding and developing technical teams.
- Effective communication skills, with the ability to convey complex technical information to diverse audiences and collaborate across teams.
