Data Engineer_Sr_Python Specialist

Job Description

..Key Responsibilities:..

• Design and build a centralized data backbone/platform integrating data from diverse sources (APIs, databases, files, streaming systems)
• Develop and maintain scalable data pipelines using Python for ingestion, transformation, and processing
• Create and manage data models and database schemas optimized for performance and scalability
• Design and maintain data dependency graphs and lineage systems to track data flow and relationships
• Ensure data consistency, integrity, and quality across all systems
• Collaborate with domain experts to translate business logic into robust data models and AI-ready datasets
• Build reusable frameworks for data orchestration and workflow management
• Integrate AI/ML pipelines into the data backbone for training, inference, and feature engineering
• Design and manage feature stores for machine learning models
• Enable LLM-based applications by structuring and curating high-quality datasets for retrieval (RAG pipelines, embeddings, vector databases)
• Implement pipelines for data preprocessing, labeling, and augmentation for ML use cases
• Optimize data storage, querying, and processing performance
• Implement monitoring, logging, and alerting for both data and ML pipelines
• Support model lifecycle workflows including training, evaluation, deployment, and versioning
• Incorporate feedback loops for continuous model and data improvement
• Document data architecture, flows, dependencies, and AI pipeline integrations clearly

Qualifications

B.E / B.Tech

Additional Information

6 - 8 Years