We are looking for Versatile, curious, and energetic Software Engineers who embrace solving complex challenges on a global scale.As a Visa Software Engineer, you will be an integral part of a multi-functional development team inventing, designing, building, and testing software products that reach a truly global customer base. While building components of powerful payment technology, you will get to see your efforts shaping the digital future of monetary transactions.
The Work Itself:
- Engineer data systems and pipelines that handle vast datasets influencing Visa's internal standards for data scalability, security, and reusability.
- Collaborate across multiple teams to create design artifacts and develop best-in-class data solutions for Visa's diverse technical offerings.
- Actively contribute to data quality improvements, valuable service technology, and new business flows in diverse agile squads.
- Develop robust and scalable data products intended for various stakeholders, including end-user merchants, B2B, and business-to-government solutions.
- Leverage innovative technologies to build the next generation of payment services, transaction platforms, real-time analytics, and data-driven insights.
- Opportunities to make a difference on a global or local scale through mentorship and continued learning opportunities.
Essential Functions:
- Demonstrates relevant technical working knowledge to understand data requirements and architectural decisions.
- Identifies and contributes to solution strategies that improve the design and functionality of data pipelines and processing frameworks, under minimal guidance.
- Applies standard processes on the use of programming languages and tools (e.g., java, Scala, Python, SQL, Spark) to develop and optimize data workflows.
- Collaborates with others to support the piloting of new data capabilities and features that enhance analytics and reporting across e-commerce products.
- Analyzes data anomalies for simple issues and applies debugging tools to verify assumptions and improve data integrity.
The Skills You Bring:
- Energy and Experience: A growth mindset that is curious and passionate about data technologies and enjoys challenging projects on a global scale.
- Challenge the Status Quo: Comfort in pushing the boundaries, hacking beyond traditional data solutions.
- Language and Tool Expertise: Proficiency in one or more data processing languages and tools (e.g., Python, SQL, Spark).
- Builder: Experience building and deploying modern data services and pipelines with quality and scalability.
- Learner: Constant drive to learn new technologies such as Hadoop, Kafka, Kubernetes, Docker, etc.
- Partnership: Experience collaborating with Product, Test, DevOps, and Agile/Scrum teams.
This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.
Qualifications
Basic Qualifications
- 8+ years of relevant work experience with a Bachelor's Degree or at least 5 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 2 years of work experience with a PhD, OR 11+ years of relevant work experience.
Preferred Qualifications
- 9 or more years of relevant work experience with a Bachelor Degree or 7 or more relevant years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 3 or more years of experience with a PhD
- Educational Background: Bachelor's or Master's degree in Computer Science, Information Technology, Data Science, or a related field.
- Experience: 6+ years of hands-on experience in data engineering, data automation, or a related role.
- Programming Skills: Proficiency in programming languages such as Python, Java, or Scala for data manipulation and automation.
- Database Management: Strong experience with SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra).
- Data Processing Frameworks: Experience with data processing frameworks such as Apache Spark, Apache Hadoop, or Apache Flink.
- ETL Tools: Proficiency in ETL tools like Apache NiFi, Talend, Informatica, or Microsoft SSIS.
- Data Generators and Copying: Ability to build data generators and automate data copying processes.
Batch Processing Expertise:
- Extensive experience in designing, implementing, and maintaining batch processing systems.
- Proficiency in scheduling and orchestrating batch jobs using tools like Apache Airflow, Control-M, or Cron jobs.
- Strong understanding of batch processing principles, including job dependencies, scheduling, and error handling.
Data Automation Framework Development:
- Ability to develop custom automation frameworks tailored for batch processing applications.
- Experience in automating the end-to-end data pipeline, including data extraction, transformation, and loading (ETL) processes.
- Creation of reusable and scalable automation scripts using Python, Bash, or other scripting languages.
Data Generators and Copy Automation:
- Expertise in building and maintaining data generators for synthetic data creation to support testing and development environments.
- Experience in automating data copying and replication processes across different environments and databases.
Performance Optimization:
- Experience in optimizing batch jobs for performance and efficiency, including resource allocation, parallel processing, and job prioritization.
- Ability to monitor and troubleshoot batch job performance issues and implement necessary improvements.
Error Handling and Recovery:
- Development of robust error handling and recovery mechanisms to ensure the reliability and resilience of batch processing workflows.
- Implementation of alerting and notification systems to promptly address job failures and exceptions.
Data Quality and Validation:
- Implementation of automated data quality checks and validation steps within batch processes to ensure data accuracy and integrity.
- Experience with tools and frameworks for automated data validation, such as Great Expectations or custom validation scripts.
Documentation and Best Practices:
- Strong documentation skills to create comprehensive guides, best practices, and standard operating procedures (SOPs) for batch automation processes.
- Knowledge of industry best practices for batch processing and data automation.
Tools and Technologies for Automation in Batch Applications:
- Scheduling and Orchestration: Apache Airflow, Control-M, Cron
- Scripting Languages: Python, Bash, Shell scripting
- Data Processing Frameworks: Apache Spark, Apache Hadoop
- ETL Tools: Apache NiFi, Talend
- Data Quality and Validation: Great Expectations, custom validation scripts
- Monitoring and Alerting: Prometheus, Grafana, custom alerting systems
