Lead Data Engineer

The candidate must have:

• Lead and participate in the design and implementation of our Analytics infrastructure to align with the vision of Visa Data Platform Infrastructure.
• Design and implement technical solutions to support the ecosystem of Analytics Infrastructure.
• Responsible for the implementation of ongoing Hadoop infrastructure including monitoring, tuning and troubleshooting.
• Design, implement and maintain security, data capacity, node forecasting and planning.
• Provide hardware architectural guidance, plan and estimate cluster capacity, and create roadmaps for the Hadoop cluster deployment.
• Actively participate in all phases of software development lifecycle: analysis, technical design, planning, development, testing/CICD, release, post production/escalation support
• Work on a highly scalable, available and resilient multi-tenant Platforms that can host large scale and highly critical applications.
• Innovate and Re-pivot Problems with Models and Higher Dimension Concepts to Support Current and Potential Use Case(s).
• Develop large scale multi-tenant software components on the Platform in an Agile based methodology to provide self-service capabilities.
• Collaborating with partners from business and technology organizations, develop key deliverables for Data Platform Strategy - Scalability, optimization, operations, availability, roadmap.
• Build partnerships with other technology teams to drive efficiency in their use of platforms and services that PaaS provide.
• Expertise in applying the appropriate software engineering patterns to build robust and scalable systems.
• Ability to act as a Tech lead, guide and mentor junior members in the scrum team. Stays current with new and evolving technologies.
• Take ownership of Platform/PaaS architecture and drive it to the next level of effectiveness to support current and long term requirements.
• Builds strong commitment within the team to support the appropriate team priorities.

This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.

Qualifications

Basic Qualifications

• Bachelor's degree in Computer Science, or related technical discipline.
• With 12+ years of software development experience in building PaaS or Application frameworks.
• Good to be as contributor/committer in one of the Bigdata technologies – Spark, Kafka, Kubernetes, Flink, Druid, Yugabyte, Pinot, Starrocks, temporal.
• Proficiency in engineering practices and writing high quality code, with expertise in either one of Java or Python or Go.
• Experience in building REST services and API's following best practices of service abstractions, Micro-services and Orchestration frameworks.
• Experience building generic services and instrumentation of libraries for wide use across an organization / company.
• Experience in Agile methodology and CICD - tool Integration, automation, configuration management in GIT.
• Excellent verbal and written communication and presentation skills, analytical and problem-solving skills
• Appreciation of diverse LOBs(Line of business) and their platform needs combined with deep technical expertise are required for the role.