Python Developer

Job Summary

GrabOn is looking for a highly skilled Python Developer with deep expertise in large-scale web scraping, browser automation, and distributed systems. The ideal candidate will design and maintain high-resilience, fault-tolerant scraping infrastructure capable of operating reliably against modern anti-bot defenses.

You will work on distributed, cloud-native scraping pipelines, collaborate on agentic and

autonomous systems, and continuously optimize for scale, cost, and reliability.

Key Responsibilities

Scraping & Automation

• Design, build, and maintain high-resilience web scraping systems at scale
• Implement advanced Selenium / Playwright automation (headless, stealth, browser fingerprint control)
• Handle anti-bot mechanisms, including:
• IP rotation & proxy orchestration
• CAPTCHA detection & mitigation
• Browser fingerprinting and evasion strategies

Distributed Systems & Cloud

• Architect and maintain distributed scraping pipelines using:
• AWS Lambda
• SQS
• EC2 (auto-scaling worker fleets)
• Build retry-safe, idempotent, and fault-tolerant pipelines
• Ensure graceful handling of failures, throttling, and partial outages

Performance, Monitoring & Optimization

• Monitor scraping throughput, failure rates, and infrastructure health
• Debug production issues across distributed workers
• Optimize AWS cost, latency, and system throughput

Agentic & Autonomous Systems

• Collaborate on agent-based architectures for:
• Autonomous task execution
• Decision-making workflows
• Self-healing data pipelines

Engineering Practices

• Write production-grade Python code (clean, testable, maintainable)
• Work comfortably in Linux server environments
• Follow best practices for concurrency, rate-limiting, and queue-based systems

Mandatory Requirements (Non-Negotiable)

• Strong Python expertise (production systems, not notebooks or scripts)
• Deep hands-on experience in web scraping at scale
• Expert-level Selenium and/or Playwright knowledge
• Proven experience with AWS Lambda, SQS, and EC2
• Strong understanding of:
• Concurrency & parallelism
• Queues and distributed workflows
• Retries, backoff strategies, and rate limits
• Comfortable managing and debugging Linux servers & cloud infrastructure