Job Title: AIML QE Architect
Location: Bangalore, Hybrid
Why should you choose us
Rakuten Symphony is a Rakuten Group company, that provides global B2B services for the mobile telco industry and enables next-generation, cloud-based, international mobile services. Building on the technology Rakuten used to launch Japan's newest mobile network, we are taking our mobile offering global. To support our ambitions to provide an innovative cloud-native telco platform for our customers, Rakuten Symphony is looking to recruit and develop top talent from around the globe. We are looking for individuals to join our team across all functional areas of our business from sales to engineering, support functions to product development. Let's build the future of mobile telecommunications together!
What Do We Expect From You
This role owns the master test strategy for validating all intelligent features within the OSS. The architect will define the how and what of AI/ML validation by designing the test situations, data validation techniques, and evaluation metrics required to verify the accuracy, performance, and robustness of models used in use cases like Anomaly Detection, predictive Root Cause Analysis (RCA), and Network Optimization. The strategic purpose is to guarantee that the AI-driven decisions made by the OSS are trustworthy, accurate, and reliable by engineering the comprehensive validation strategy to certify them.
The company may expect you to undertake other tasks outside of this job description. This job description is not exhaustive and may be updated from time to time.
Roles & Responsibilities:
Model Performance & Accuracy Validation
- Architect the framework for validating the predictive performance of all AI/ML models, establishing the definitive metrics (e.g., accuracy, precision, recall, F1-score) for certification.
- Design the test strategies to validate model outputs against defined business objectives and Service Level Objectives (SLOs), ensuring models deliver tangible value.
MLOps & Pipeline Integrity Architecture
- Architect the end-to-end validation strategy for the entire MLOps pipeline, ensuring quality and integrity from data ingestion and feature engineering to model training, deployment, and monitoring.
- Design the automated feedback loops for continuous model validation in production, including A/B testing frameworks and canary deployment validation.
Data Quality & Model Robustness Strategy
- Design the automated test strategies to proactively detect and mitigate issues like data/concept drift and data quality degradation before they impact model performance.
- Architect the validation approach for model robustness, designing adversarial tests and corner-case scenarios to uncover and mitigate potential vulnerabilities.
Explainability & Bias Validation
- Define and implement the methodologies for validating model explainability (XAI), using techniques like SHAP or LIME to ensure model decisions are transparent and understandable.
- Architect the framework for fairness and bias detection, designing tests to identify and mitigate unintended bias in training data and model behavior.
Technical Governance & Tooling Strategy
- Define and evangelize the technical requirements for specialized AI/ML validation tools, data labelling platforms, and automation frameworks, providing clear specifications to the engineering teams.
- Act as the primary technical authority on AI/ML quality, providing strategic guidance and quality benchmarks to Data Science and MLOps teams to influence model design and data strategy.
Qualification:
Experience and Expertise
- 5+ years in a data science, ML engineering, or ML QA role, with at least 2 years focused on designing and implementing validation strategies for production AI systems.
- Proven experience working with large-scale, real-world datasets, particularly time-series data relevant to the telecom or a similar domain.
- Demonstrable experience in setting up and executing formal validation plans for machine learning applications.
Analytical and Problem-Solving Skills
- Strong statistical analysis skills and the ability to critically evaluate the performance and fairness of a model beyond simple accuracy metrics.
- Ability to design robust experiments (e.g., A/B tests) to isolate variables and quantify the impact of model changes.
- Aptitude for translating ambiguous business problems into quantifiable validation metrics and acceptance criteria.
Technical Skills
- Strong understanding of the ML development lifecycle, including data preprocessing, model training, and evaluation.
- Proficiency in Python and common ML libraries such as scikit-learn, TensorFlow, or PyTorch.
- Knowledge of MLOps principles and hands-on experience with tools like MLflow, Kubeflow, or Seldon Core.
Collaboration & Communication
- Ability to communicate complex ML concepts and validation results to data scientists, engineers, and non-technical product managers.
- Experience creating clear, data-driven reports and visualizations that summarize model performance and risks for executive stakeholders.
- Proven ability to influence technical decisions in data science and product teams based on validation findings and risk analysis.
Educational Background
- Bachelor's or Master's degree in Computer Science, Statistics, Data Science, or a related quantitative field.
Additional Skills
- Deep knowledge of telecom network data (e.g., performance counters, logs, traces) and common AI/ML use cases in the industry.
- Experience with big data technologies (e.g., Spark, Kafka, Flink) for data processing and analysis.
- Publications or active participation in the AI/ML, MLOps, or data science communities. Prior exposure to working in cross-continental distributed organizations.
- Working knowledge of Atlassian suite, Confluence, and modern OKR tracking tools.
- Good understanding of Agile principles and processes.
- Experience working in a fluid, start-up-like environment
Rakuten Shugi Principles:
- Our worldwide practices describe specific behaviours that make Rakuten unique and united across the world. We expect Rakuten employees to model these 5 Shugi Principles of Success.
- Always improve, always advance. Only be satisfied with complete success - Kaizen.
- Be passionately professional. Take an uncompromising approach to your work and be determined to be the best.
- Hypothesize - Practice - Validate - Shikumika. Use the Rakuten Cycle to success in unknown territory.
- Maximize Customer Satisfaction. The greatest satisfaction for workers in a service industry is to see their customers smile.
- Speed!! Speed!! Speed!! Always be conscious of time. Take charge, set clear goals, and engage your team.
