Key Skills: Big Data, Azure Databricks, Databricks, Python, Data Engineering, Terraform
Roles and Responsibilities:
- Deliver business requirements end-to-end, from discovery and solution design through development, production rollout, and operational readiness.
- Build and enhance data engineering toolchains, platform capabilities, and reusable data engineering patterns in a Databricks environment.
- Apply software engineering best practices with an automation-first approach and a continuous learning and improvement mindset.
- Ensure solutions adhere to enterprise architecture direction and architectural standards.
- Collaborate in a high-performing team environment and demonstrate the ability to influence outcomes.
- Contribute to continuous learning and improvement by iterating on designs and implementation based on feedback and results.
Skills Required:
- Strong experience in Big Data technologies and distributed data processing frameworks.
- Deep hands-on expertise in Azure Databricks for building scalable data solutions.
- Proficiency in Python for developing robust data pipelines and data engineering workflows.
- Solid experience in designing and implementing end-to-end data engineering solutions.
- Good understanding of Databricks architecture, performance tuning, and optimization techniques.
- Experience in building and managing data pipelines, ETL processes, and data integration workflows.
- Familiarity with Terraform for infrastructure as code and cloud resource provisioning.
- Strong knowledge of cloud-based data platforms, particularly within Azure environments.
- Experience with API development, middleware integration, and automation practices.
- Good understanding of software engineering best practices, including version control, testing, and CI/CD.
- Strong problem-solving skills with the ability to design scalable and high-performance systems.
- Effective collaboration and communication skills to work within cross-functional teams.
Education:
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- A master's degree is an added advantage.
