Data Engineer

As a Data Engineer, you will play a crucial role in designing, building, and maintaining data infrastructure that supports our data-driven initiatives. This position is vital in ensuring that our data pipelines are efficient, scalable, and reliable, enabling our data scientists and analysts to derive actionable insights. The ideal candidate will have a strong foundation in data engineering principles, be comfortable working with big data technologies, and possess excellent problem-solving skills to tackle complex data challenges.

Key Responsibilities:

• Design and implement scalable data architectures to support large-scale data processing.
• Develop and maintain ETL processes to ensure high-quality data for analysis.
• Collaborate with data scientists to understand data needs and optimize data workflows.
• Ensure data integrity and security across all data systems.
• Optimize data pipelines for performance and cost-efficiency.
• Troubleshoot and resolve data pipeline issues to minimize downtime.
• Participate in code reviews and contribute to documentation of data processes.
• Integrate new data sources and technologies into the existing data infrastructure.
• Monitor and maintain data system performance, identifying areas for improvement.
• Work closely with other engineering teams to align data initiatives with business goals.

Qualification Criteria:

• Proven experience in designing and implementing data architectures.
• Strong background in ETL processes and data pipeline management.
• Familiarity with big data technologies such as Hadoop, Spark, or similar.
• Excellent problem-solving skills with a focus on data-related challenges.
• Experience with SQL and NoSQL databases.
• Proficiency in scripting languages such as Python or Scala.
• Strong understanding of data governance and compliance requirements.
• Ability to work in a fast-paced, collaborative environment.
• Excellent communication skills to articulate technical concepts to non-technical stakeholders.
• Bachelor's degree in Computer Science, Data Engineering, or related field.

Primary Skills:

• ETL
• SQL
• big data

Key Skills:

• Python
• Scala
• Hadoop
• Spark
• NoSQL
• data governance
• data security
• cloud computing
• performance optimization
• data compliance