DevOps Architect II

About the role....

This SRE professional will have the opportunity to work for an AI-based Unicorn which is recognized as one of the fastest-growing companies on the Inc. 5000 list. This role will provide you opportunity to deploy, maintain and support the o9 Digital Brain Platform across the world on AWS, AZURE, GCP &; Samsung Cloud utilizing state of the art CI/CD tools. This role will empower you to continuously challenge the status quo and implement the great ideas you may have to create value for o9 clients.

Role Overview:

This role involves the management, installation, and configuration of Hadoop/Spark components (Hive, HDFS, Ambari, NiFi), as well as designing the architecture to integrate with the o9 platform. You will play a key role in the design, maintenance, and productionization of Hadoop/Spark/Delta Lake clusters, addressing daily production issues, and implementing containerization and automation to facilitate easier maintenance for the DevOps team.

Key Responsibilities:

• Collaborate with customers and technical consultants to design and recommend big data architecture solutions tailored to customer requirements.
• Analyze complex distributed production environments and provide actionable recommendations for performance optimization.
• Develop and present detailed architectural documentation for customers technical teams.
• Partner with o9 s Development, DevOps, and project teams to ensure the successful execution of projects.
• Design and implement distributed systems and configurations for Spark, Delta Lake, Hadoop, and related technologies, particularly in cloud-based deployments.
• Produce and maintain comprehensive technical documentation and user manuals for customers.
• Stay current with the latest advancements in the distributed big data ecosystem and cloud technologies.

Qualifications and Skills:

• Experience:
• 8+ years of experience in DevOps, with a focus on architecting large-scale storage, data processing, and globally distributed solutions.
• Proven expertise in designing and deploying large-scale, data-intensive solutions using Spark, Delta Lake, and Airflow schedulers.
• Strong experience designing and optimizing data queries within Spark/Delta Lake environments, using tools such as Apache Hive, HDFS, Apache Spark, and Hadoop.
• Experience with the installation, administration, and tuning of cloud-agnostic and distributed data systems.
• Extensive experience implementing solutions across cloud platforms (Azure, AWS, GCP).
• Hands-on experience implementing big data use cases and a solid understanding of standard design patterns for Spark/distributed systems, including Airflow, NiFi, Kafka, and others.
• Significant experience working with network-based APIs, ideally REST/JSON or XML/SOAP.
• Skills:
• Strong understanding of enterprise security solutions such as LDAP, Kerberos, and SSL implementation.
• Expertise in networking, troubleshooting, and optimizing data transfer across systems.
• Proficient in Python and Java development, including debugging and profiling.
• Solid background in database administration and design; experience with data modeling is a plus.
• Excellent verbal and written communication skills.
• Familiarity with R and machine learning algorithms is an advantage.

What we ll do for you

• Competitive Salary
• Flexibility: Work in Hybrid mode and flexible working hours
• Get social: When we work from home, we play from home with fun after-work activities like Friday Socials. If you re in the office, feel free to join these events in person.
• Flat organization: With a very strong entrepreneurial culture (and no corporate politics).
• Support network: Work with a team you can learn from and every day.
• Diversity: We pride ourselves on our international working environment.