Location: Mumbai, Maharashtra (Onsite)
Experience: 5+ years
Employment Type: Full-time
About the Role:
We're seeking a Senior Data Engineer with deep expertise in large-scale data scraping and end-to-end data architecture design. This role focuses on building the backbone of our AI-driven systemsfrom developing responsive, semantic scraping frameworks to architecting robust, scalable data pipelines from the ground up. You'll play a key role in ensuring that our data infrastructure is intelligent, efficient, and future-ready.
Key Responsibilities:
- Design and implement scalable web scraping systems capable of handling high-volume, high-velocity data sources.
- Develop responsive and semantic scraping solutions using advanced AI and NLP methods to extract meaningful, context-rich information.
- Architect and build data pipelines from scratch, integrating structured and unstructured data across multiple platforms.
- Establish reliable, fault-tolerant data architectures for long-term scalability and maintainability.
- Integrate scraped data into RAG (Retrieval-Augmented Generation) and other intelligent workflows.
- Continuously enhance scraping performance, pipeline efficiency, and data quality through automation and innovation.
Required Skills and Qualifications:
- 5+ years of experience in data engineering, data scraping, and pipeline development.
- Proven track record of building end-to-end data systems from the ground up.
- Deep knowledge of web scraping tools (Scrapy, Selenium, BeautifulSoup) and anti-blocking, concurrent scraping strategies.
- Strong understanding of AI-driven data extraction, NLP, and semantic enrichment.
- Experience implementing RAG pipelines and integrating LLMs with retrieval systems.
- Familiarity with cloud infrastructure (AWS/GCP/Azure) and CI/CD best practices.
Why Join Us:
- Build cutting-edge, AI-powered data scraping ecosystems at production scale.
- Shape the data architecture roadmap for next-generation intelligent systems.
- Collaborate with a highly skilled, forward-thinking engineering team.
- Competitive salary, challenging projects, and high growth potential.
If you're passionate about turning large-scale web data into meaningful, AI-ready insights, we want you on our team.
