Job Description
We are looking for a skilled Software & Data Engineer with deep expertise in Python, AWS, Snowflake, and modern data engineering practices to design, develop, and maintain robust data pipelines and backend systems. The ideal candidate will operate at the intersection of software development and data engineering, owning the entire data workflow from infrastructure as code (IaC) and pipeline orchestration to testing and deployment.
You will be part of a high-performing, cross-functional Agile/Data Engineering team responsible for delivering reliable, secure, and scalable data solutions to support R&D and business analytics. You'll collaborate with data scientists, engineers, and stakeholders to ensure high-quality, production-grade software and Data products.
Key Responsibilities
Design and develop end-to-end data pipelines using Python, DBT, and Snowflake (First class SQL writing skills).
Code Quality: Ensure code quality through best practices, including writing clean, efficient, and maintainable code.
Code Reviews: Conduct and participate in code reviews to maintain high code standards and provide constructive feedback to peers.
Use Infrastructure as Code tools (Terraform/CloudFormation) to automate infrastructure deployment, as required.
Integrate data from multiple sources using AWS Glue, Lambda, Athena, S3, and Step Functions.
Ensure data quality using unit testing, automated testing frameworks like DBT Tests, Great Expectations, or Pytest.
Implement CI/CD pipelines using GitHub Actions, Jenkins, or similar tools.
Optimize pipeline performance and cost and monitor for issues in production environments.
Support version-controlled, testable, and maintainable code following clean coding principles.
Documentation: Maintain up-to-date documentation for software designs, processes, and codebases to ensure knowledge sharing and project continuity.
Project Planning: Contribute to project planning and estimation efforts.
Continuous Learning: Stay updated with new technologies, tools, and industry trends to bring innovative solutions to the team.
Collaborate across teams and mentor junior team members in data and software best practices.
