Role: Senior ML Engineer(Databricks MLOps)
Location: Hyderabad
Experince: 8+ Yrs
Job Description Summary
ML Engineer within the Data Science and Machine Learning team leverages and third-party software to create solutions to business problems defined by specific business requirements. In this position, you will draw upon technical, AI/ML engineering, Data and MLOps experience to solve complex marketing analytics problems on very large volumes of data.
Requirements
As a Machine Learning (ML) Engineer You will build and operationalize machine learning pipelines involving terabytes of data. You will have the responsibility to help define requirements, create software designs, implement code to these specifications, As an ML Engineer, you will be expected to:
Develop and maintain a comprehensive enterprise architecture for AI/ML/GenAI initiatives, ensuring alignment with overall business strategy and technology roadmap.
Architect hyperscale MLOps solutions and pipelines
Work with Applied Scientists, Data Scientists, Product owners, ML Engineers, and Software Engineers to design and deliver ML solutions in production at scale.
Develop automated AI and ML workflows and end-to-end pipelines for data preparation, training, deployment, monitoring, etc., and ensure the quality of architecture and design of our ML systems and data infrastructure.
Leverage AI to develop GenAI powered solutions to complement our data science and product build capabilities
Assess current state AI/ML/GenAI capabilities across various business domains, identify gaps, and design target state architectures to drive innovation, revenue growth and operational excellence.
Lead transformational initiatives to bridge the gap between current and desired AI/ML capabilities, collaborating with cross-functional teams to ensure successful implementation.
Establish governance frameworks and decision criteria for AI/ML and GenAI projects, ensuring adherence to industry standards, regulatory requirements, Responsible AI and architectural principles.
Create and maintain reference architectures, patterns, and best practices for AI/ML lifecycle and integration within enterprise ecosystem.
Lead the technology evaluation and process improvements to drive Experimentation, Model Development and ML Ops at scale
Lead and drive standardization of LLM onboarding process, RAG pipelines and application development
Conduct architecture reviews and risk assessments for proposed AI/ML solutions, ensuring they meet security, scalability, and interoperability requirements.
Utilize advanced data science techniques (e.g., Natural Language Processing, clustering, predictive analytics, regression analyses, survival analysis, segmentation, and experimentation) to propose enhancements and innovations to business processes.
Conduct sophisticated statistical analyses and maintain high reliability of machine learning pipelines in production environments, ensuring minimal downtime and optimal performance.
Collaborate with business leaders and product to identify opportunities for AI/ML-driven innovation and guide the development of use cases that deliver tangible business value.
Foster a culture of continuous learning and innovation in AI/ML practices across the enterprise architecture team and broader organization.
Basic Qualifications
- 8+ of years of experience in enterprise architecture, with a focus on AI/ML integration and transformation projects.
- 6+ years professional experience in software development
Bachelor's Degree in Computer Science or Associate Degree & 3+ years of development experience or equivalent experience
- Computer Science fundamentals in object-oriented design
- Computer Science fundamentals in data structures
- Computer Science fundamentals in algorithm design, problem solving, and complexity analysis
- Knowledge of, at least, one modern programming language such as Python, Java, C++, C, Java, Python or Perl
Preferred Qualifications
8+ years of experience architecting scalable ML infrastructure and big data systems.
Databricks Architect Certification is required
6+ years of architecting solutions using Databricks. Strong experience using Mosaic AI, Unity Catalogue, mlflow, workflow orchestration and other databricks native MLOps capabilities.
At least 1+ year experience in GenAI (Technical familiarity with 2 or more) - OpenAI API, Bedrock API, Vertex API, LangGraph, other agentic frameworks
High attention to detail and proven ability to manage multiple, competing priorities simultaneously.
Experience MLOps and orchestration tools such as Airflow, Kubeflow, DAGster, Optuna, Mlflow or other similar MLOps tools.
Experience with operationalizing and migrating ML models into production at scale.
Developing Large scale model inference solutions using parallel execution framework using spark, EMR, databricks
Experience developing complex orchestration and MLOps pipelines stitching together large volumes of data for training and scoring
Experience with Large Language Models, fine tuning and deployment frameworks using HuggingFace capabilities or cloud provider solutions such as Amazon Bedrock, Vertex AI model garden etc.
Familiarity with Vector databases such as Pinecone, ChromaDB or similar tools.
Experience in CI/CD/DevOps, Deployment and Automation Tool CI/CD, Jenkins, Terraform, Cloud Formation Template or similar
Proficiency with Apache Spark, EMR/DataProc and Cloud based tools (Snowflake, Redshift, EMR, Glue, Step Functions, Lambda, Step functions, AWS Batch, or similar etc.).
Experience with ML libraries like H20, scikit learn and deep learning frameworks (PyTorch, TensorFlow, etc.).
Experience with end-to-end software development and life cycle of ML solutions.
Excellence in technical communication with scientists and engineers.
At least 2years Database (SQL) experience, Linux
At least 6+ years of AWS infrastructure experience - Cloud run, App server, RDS, S3, EC2, EMR or equivalent GCP experience
What Will Set You Apart
Databricks Certification
Langgraph, Databricks MLFlow experience, Docker experience, Kubernetes experience
Knowledge of LLM observability platforms
Good communication skills: communicate ideas clearly and effectively to other members of the analytics team and to the client at multiple levels (both technical and business)
Analytic problem-solving skills with the ability to think outside-the-box
Analytical thinker that excels at analyzing and understanding data to answer questions
Excellent understanding of data concepts, data architecture, data manipulation/engineering, and data engineering design
Passion for considering how projects fit into the wider business picture
An understanding in multiple types of programming languages in order to be adaptable (statically typed vs. dynamically typed and object-oriented vs. procedural)
Self-Starter Able to work independently with little guidance
Adaptable - Able to adapt to diverse technical challenges and systems
Ability to formulate and present insights with gathered data to both technical and non-technical peers, leaders, and clients
