Data Engineering Manager

Design, develop, and maintain data solutions for data generation, collection, and processing.

• Lead and be hands-on for the technical design, development, testing, implementation, and support of data pipelines that load the data domains in the Enterprise Data Fabric and associated data services.
• Create data pipelines and ensure data quality by implementing ETL processes to migrate and deploy data across systems.
• Take ownership of data pipeline projects from inception to deployment, manage scope, timelines, and risks.
• Collaborate with cross-functional teams to understand data requirements and design solutions that meet business needs.
• Develop and maintain data models, data dictionaries, and other documentation to ensure data accuracy and consistency.
• Be able to translate data models (ontology, relational) into physical designs that are performant, maintainable, and easy to use.
• Implement data security and privacy measures to protect sensitive data.
• Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions.
• Collaborate and communicate effectively with product teams.
• Identify and resolve complex data-related challenges.
• Adhere to best practices for coding, testing, and designing reusable code/components.
• Explore new tools and technologies that will help improve ETL platform performance.
• Participate in sprint planning meetings and provide estimations on technical tasks.
• Collaborate with RunOps engineers to continuously increase our ability to push changes into production with as little manual overhead and as much speed as possible.

What we expect of you

Basic Qualifications:

• Master's degree and 4 to 6 years of Computer Science, IT, or related field experience OR
• Bachelor's degree and 6 to 8 years of Computer Science, IT, or related field experience OR
• Diploma and 10 to 12 years of Computer Science, IT, or related field experience.

Preferred Qualifications:

Functional Skills:

Must-Have Skills:

• Hands-on experience with big data technologies and platforms, such as Databricks, Apache Spark (PySpark, SparkSQL), workflow orchestration, and performance tuning on big data processing.
• Proficient in SQL for extracting, transforming, and analyzing complex datasets from both relational and graph data stores (e.g., Marklogic, Allegrograph, Stardog, RDF Triplestore).
• Experience with ETL tools such as Apache Spark, Prophecy, and various Python packages related to data processing and machine learning model development.
• Strong understanding of data modeling, data warehousing, and data integration concepts.
• Able to take user requirements and develop data models for data analytics use cases.

Good-to-Have Skills:

• Knowledge of Python/R, Databricks, SageMaker, and cloud data platforms.
• Experience using graph databases such as Stardog, Marklogic, Neo4J, Allegrograph, etc., and writing SPARQL queries.
• Experience working with agile development methodologies such as Scaled Agile.

Professional Certifications:

• AWS Certified Data Engineer preferred.
• Certified Data Engineer / Data Analyst (preferred on Databricks or cloud environments).

Soft Skills:

• Excellent critical-thinking and problem-solving skills.
• Strong communication and collaboration skills.
• Demonstrated awareness of how to function in a team setting.
• Demonstrated presentation skills.