The candidate will be responsible to deliver business needs end to end from requirements to development into production.
Through hands-on engineering approach in Databricks environment, this individual will deliver data engineering toolchains, platform capabilities, and reusable patterns.
The applicant will be responsible to follow software engineering best practices with an automation first approach and continuous learning and improvement mindset.
The applicant will ensure adherence to enterprise architecture direction and architectural standards.
The applicant should be able to collaborate in a high-performing team environment, and an ability to influence and be influenced by others.
Required Skills:
Cloud-based security principles and protocols like OAuth2, JWT, data encryption, hashing data, secret management, etc.
Expertise in Big data technologies such as Spark, Hadoop,
Databricks, Snowflake, EMR, Azure Data Factory
Good understanding of Kafka, Kafka Streams, Spark
Structured streaming, configuration-driven data transformation and curation
Expertise in building cloud-native microservices, containers, Kubernetes and platform-as-a-service technologies such as OpenShift, CloudFoundry
Experience in multi-cloud software-as-a-service products such as Databricks, Snowflake
Experience in Infrastructure-as-Code (IaC) tools such as terraform
Experience in messaging systems such as Apache ActiveMQ, WebSphere MQ, Apache Artemis, Kafka, Azure Service Bus
Experience in API and microservices stack such as Spring Boot, Quarkus,
Expertise in Cloud technologies such as Azure Data Factory, Lambda, S3, Elastic Search, API Gateway, CloudFront
Experience with one or more of the following programming and scripting languages - Python, Scala, JVM-based languages, or JavaScript, and ability to pick up new languages
Experience in building CI/CD pipelines using Jenkins, Github Actions, Azure DevOps
Strong expertise with source code management and its best practices
Proficient in self-testing of applications, unit testing and use of mock frameworks, test-driven development (TDD)
Knowledge on Behavioural Driven Development (BDD) approach
Ability to perform detailed analysis of business problems and technical environments
Strong oral and written communication skills
Ability to think strategically, implement iteratively and estimate financial impact of design/architecture alternatives
Continuous focus on an on-going learning and development
Expertise in Agile software development principles and patterns
Expertise in building streaming, batch and event-driven architectures and data pipelines
