Job Description
About The Role
We are looking for a Software Engineer – Data Engineering to design, build, and maintain robust, scalable, and high-performance data infrastructure. In this role, you will work on end-to-end data pipelines, ensuring reliable data availability for analytics, machine learning, and business applications.
Key Responsibilities
Data Pipeline Development:
Design, implement, and optimize ETL/ELT pipelines for structured and unstructured data.
Integrate data from multiple internal and external sources into centralized systems.
Build scalable batch and real-time data processing workflows.
Data Infrastructure & Architecture:
Develop and maintain data lake, data warehouse, or data mesh architectures.
Ensure high availability, performance, and scalability of data systems.
Implement data modeling best practices to support analytics and ML use cases.
Software Engineering:
Write clean, efficient, and maintainable code in Python, Java, or Scala.
Implement unit tests, integration tests, and CI/CD pipelines for data workflows.
Apply software engineering principles to build reusable data services and APIs.
Data Quality & Governance:
Implement validation, monitoring, and observability to ensure data accuracy and reliability.
Enforce data security, compliance, and privacy standards (e.g., GDPR).
Collaborate with stakeholders to establish data definitions, lineage, and documentation.
Requirements
Bachelor's or Master's degree in Computer Science, Engineering, or related field.
2–4 years of experience in data engineering, software engineering, or backend systems.
Strong programming skills in Python, Java, or Scala.
Experience with distributed data processing frameworks (Apache Spark, Flink, Beam, or similar).
Proficiency with SQL and relational databases; familiarity with NoSQL systems (Cassandra, MongoDB, DynamoDB).
Hands-on experience with cloud platforms (AWS, GCP, Azure) and data services (e.g., Redshift, BigQuery, Snowflake, Databricks).
Familiarity with workflow orchestration tools (Airflow, Luigi, Dagster) and CI/CD pipelines.
Solid understanding of software engineering best practices (version control, testing, code reviews).
