Senior Python Developer

Responsibilities:

• Develop and maintain efficient web crawlers with a focus on speed, performance, and scalability.
• Write clean, reusable, and optimized Python code for large-scale data extraction.
• Analyze and understand website structures, implement logic to handle dynamic and JS-rendered pages.
• Troubleshoot and debug crawl issues, optimize request handling, and improve crawler reliability.
• Collaborate with backend and data teams to ensure smooth data flow and integration.
• Monitor crawler performance, logs, and resolve runtime errors.
• Participate in code reviews, documentation, and continuous process improvements.

Skills Required:

• Strong coding skills in Python.
• Good understanding of how web pages, HTTP, and crawlers work.
• Hands-on experience with Docker, Postman, and PowerShell.
• Basic understanding of HTML, AngularJS, RabbitMQ, and Celery.
• Experience with Git and Bitbucket for version control and team collaboration.
• Familiarity with XPath, CSS selectors, and regular expressions for content extraction.

Preferred:

• Exposure to Scrapy, Puppeteer/Playwright, or other crawling frameworks.
• Understanding of Elasticsearch, HAProxy, or Curator for large-scale data handling.