Lead Gen AI Data Scientist - GenAI

Responsibilities:

• Design and implement advanced solutions utilizing Large Language Models (LLMs).
• Demonstrate self-driven initiative by taking ownership and creating end-to-end solutions.
• Conduct research and stay informed about the latest developments in generative AI and LLMs.
• Develop and maintain code libraries, tools, and frameworks to support generative AI development.
• Participate in code reviews and contribute to maintaining high code quality standards.
• Engage in the entire software development lifecycle, from design and testing to deployment and maintenance.
• Collaborate closely with cross-functional teams to align messaging, contribute to roadmaps, and integrate software into different repositories for core system compatibility.
• Possess strong analytical and problem-solving skills.
• Demonstrate excellent communication skills and the ability to work effectively in a team environment.

Primary Skills:

• Natural Language Processing (NLP): Hands-on experience in use case classification, topic modeling, QA and chatbots, search, Document AI, summarization, and content generation.
• Computer Vision and Audio: Hands-on experience in image classification, object detection, segmentation, image generation, audio, and video analysis.
• Generative AI: Proficiency with SaaS LLMs, including Lang chain, llama index, vector databases, Prompt engineering (COT, TOT, ReAct, agents). Experience with Azure OpenAI, Google Vertex AI, AWS Bedrock for text/audio/image/video modalities.
• Familiarity with Open-source LLMs, including tools like TensorFlow/Pytorch and huggingface. Techniques such as quantization, LLM finetuning using PEFT, RLHF, data annotation workflow, and GPU utilization.
• Cloud: Hands-on experience with cloud platforms such as Azure, AWS, and GCP. Cloud certification is preferred.
• Application Development: Proficiency in Python, Docker, FastAPI/Django/Flask, and Git.

Tech Skills (10+ Years Experience):

1.Machine Learning (ML) Deep Learning:

- Solid understanding of supervised and unsupervised learning.

- Proficiency with deep learning architectures like Transformers, LSTMs, RNNs, etc.

2. Generative AI:

- Hands-on experience with models such as OpenAI GPT4, Anthropic Claude, LLama etc.

- Knowledge of fine-tuning and optimizing large language models (LLMs) for specific tasks.

3. Natural Language Processing (NLP):

- Expertise in NLP techniques, including text preprocessing, tokenization, embeddings, and sentiment analysis.

- Familiarity with NLP tasks such as text classification, summarization, translation, and question-answering.

4. Retrieval-Augmented Generation (RAG):

- In-depth understanding of RAG pipelines, including knowledge retrieval techniques like dense/sparse retrieval.

- Experience integrating generative models with external knowledge bases or databases to augment responses.

5. Data Engineering:

- Ability to build, manage, and optimize data pipelines for feeding large-scale data into AI models.

6. Search and Retrieval Systems:

- Experience with building or integrating search and retrieval systems, leveraging knowledge of Elasticsearch, AI Search, ChromaDB, PGVector etc.

7. Prompt Engineering:

- Expertise in crafting, fine-tuning, and optimizing prompts to improve model output quality and ensure desired results.

- Understanding how to guide large language models (LLMs) to achieve specific outcomes by using different prompt formats, strategies, and constraints.

- Knowledge of techniques like few-shot, zero-shot, and one-shot prompting, as well as using system and user prompts for enhanced model performance.

8. Programming Libraries:

- Proficiency in Python and libraries such as PyTorch, Hugging Face, etc.

- Knowledge of version control (Git), cloud platforms (AWS, GCP, Azure), and MLOps tools.

9. Database Management:

- Experience working with SQL and NoSQL databases, as well as vector databases

10. APIs Integration:

- Ability to work with RESTful APIs and integrate generative models into applications.

11. Evaluation Benchmarking:

- Strong understanding of metrics and evaluation techniques for generative models.