Data Engineer

Responsibilities:

• Data Solution Design: Translate detailed technical drawings and specifications into actionable plans for data infrastructure and pipeline development.
• Estimation & Budgeting: Prepare accurate estimates and budgets for data engineering projects, including resource allocation and technology costs.
• Project Specification: Create accurate and comprehensive project specifications for data ingestion, processing, storage, and transformation solutions.
• Experiment Design: Design and conduct engineering experiments related to data processing, performance optimization, and new technology evaluation.
• Technical Reporting: Create detailed technical reports and documentation for internal teams and external customers, explaining complex data solutions and their implications.
• Regulatory Compliance: Complete necessary regulatory documents concerning data safety, privacy, and compliance issues, ensuring all solutions meet legal and ethical standards.
• Data Pipeline Development: Design, build, and maintain scalable and robust data pipelines for data ingestion, transformation, and delivery.
• Database Management: Work with various database technologies (SQL, NoSQL, data warehouses, data lakes) to store, manage, and retrieve data efficiently.
• Data Quality & Governance: Implement processes and tools to ensure data quality, integrity, and governance throughout the data lifecycle.
• Collaboration: Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver appropriate solutions.

Required Skills:

• Proficiency in making plans using detailed drawings related to data architecture.
• Experience in preparing estimates and budgets for data projects.
• Ability to create accurate project specifications for data solutions.
• Skills in designing engineering experiments related to data.
• Capability to create technical reports for customers.
• Familiarity with completing regulatory documents concerning safety issues in data.
• Strong understanding of data modeling, ETL/ELT processes, and data warehousing concepts.
• Proficiency in programming languages relevant to data engineering (e.g., Python, Scala, Java).
• Experience with cloud data platforms (e.g., AWS, Azure, GCP) is often preferred.