Job Summary
We are seeking an experienced
Agentic AI Lead with deep expertise in Databricks to design, develop, and deploy advanced generative AI and agentic systems. The ideal candidate will architect scalable, intelligent solutions that combine agentic AI frameworks with generative AI models, leveraging Databricks unified analytics platform to drive business value. You will collaborate with cross-functional teams to deliver innovative AI solutions that enable autonomous decision-making and generative capabilities.
Key Responsibilities
- Architect Agentic AI Systems: Design and implement agentic AI frameworks that enable autonomous, goal-oriented systems capable of reasoning, planning, and decision-making.
- Develop Generative AI Solutions: Build and deploy generative AI models (e.g., LLMs, GANs, VAEs) for applications such as content generation, data augmentation, and creative automation.
- Leverage Databricks: Utilize Databricks for end-to-end AI workflows, including data ingestion, preprocessing, model training, deployment, and monitoring.
- Optimize AI Pipelines: Create scalable, efficient, and robust AI pipelines on Databricks, integrating with tools like Delta Lake, MLflow, and Spark for data processing and model management.
- Collaborate with Teams: Work closely with data scientists, engineers, and business stakeholders to translate requirements into technical solutions.
- Drive Innovation: Stay updated on advancements in agentic AI, generative AI, and Databricks capabilities, proposing innovative solutions to enhance system performance and capabilities.
- Ensure Scalability and Reliability: Architect solutions that are scalable, secure, and optimized for performance in production environments.
- Mentor and Guide: Provide technical leadership and mentorship to junior team members on AI best practices and Databricks workflows.
Required Qualifications
- Education: Bachelor's or Master's degree in Computer Science, Data Science, Machine Learning, or a related field.
- Experience:
- 5+ years of experience in AI/ML architecture, with at least 2 years focused on generative AI and/or agentic systems.
- 3+ years of hands-on experience with Databricks, including Delta Lake and Spark.
- Proven track record of designing and deploying production-grade AI solutions.
- Technical Skills:
- Expertise in generative AI models (e.g., GPT, BERT, Stable Diffusion) and agentic AI frameworks (e.g., LangChain, AutoGen, llamaindex, langraph).
- Proficiency in Databricks for data engineering, model training, and deployment.
- Strong programming skills in Python, PySpark, and SQL.
- Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their integration with Databricks.
- Soft Skills:
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities to work with cross-functional teams.
- Ability to lead projects and mentor team members.
Preferred Qualifications
- Experience with reinforcement learning, multi-agent systems, or autonomous AI agents.
- Knowledge of Databricks-specific tools like Databricks SQL, Feature Store, or AutoML.
- Familiarity with additional AI frameworks (e.g., TensorFlow, PyTorch, Hugging Face).
- Certifications in Databricks, AWS, Azure, or Google Cloud.
