Remote: Distributed Web Crawler Engineer
- Location
- United Kingdom
source world-class data from the open web and thrive on building scalable, reliable tooling. You’ll tackle challenges like content extraction from messy HTML, deduplication at web scale, freshness and recrawl strategies, and politeness controls, while designing pipelines to find high-value data sources #J-18808-Ljbffr ...