HeadlinesBriefing favicon HeadlinesBriefing.com

Anubis deploys proof‑of‑work to curb AI web scraping

Hacker News •
×

Hacker News users now encounter a gatekeeper called Anubis on sites fearing large‑scale AI scraping. The system forces visitors to solve a lightweight Proof‑of‑Work challenge modeled after Hashcash, an early spam‑deterrent. For a single human the cost is tiny, but for bots sending thousands of requests it adds measurable expense. When bots overwhelm a server, legitimate visitors experience outages, prompting operators to adopt the filter.

Site operators view the pause as a stopgap while they develop more precise fingerprinting techniques, such as detecting headless browsers via font‑rendering anomalies. Until those methods mature, the PoW page remains the front line, shielding resources from downtime caused by aggressive crawlers. The approach deliberately sacrifices a tiny fraction of user convenience for broader availability.

Because Anubis relies on modern JavaScript, extensions like JShelter block it, prompting users to disable such plugins for the affected domain. Administrators hope the added friction will discourage mass scraping long enough to roll out smarter detection. As of now, the proof‑of‑work barrier is the only active defense deployed. The measure has already slowed several high‑traffic crawlers, confirming its practicality.