The rise of AI companies aggressively scraping websites has caused significant downtime, rendering resources inaccessible for legitimate users. To mitigate this threat, website administrators are turning to Anubis, a novel protection mechanism that employs a proof-of-work scheme to distinguish between human visitors and automated bots.
Anubis is inspired by Hashcash, a proof-of-work concept originally designed to combat email spam. The system requires each visitor to perform a minimal computational task, which is trivial for individuals but cumulatively burdensome for scrapers operating at scale. This approach serves as a temporary solution, allowing time for the advancement of browser fingerprinting technologies that can identify headless browsers—automated tools that mimic human behavior—through methods such as analyzing font rendering. The goal is to eventually phase out the proof-of-work challenge for users who are likely genuine. Anubis represents a compromise between security and accessibility.
It is important to note that Anubis depends on contemporary JavaScript functionality. Plugins like JShelter, which enhance privacy by disabling such features, will interfere with the system. Therefore, users must disable JShelter or similar extensions to access the website when encountering this protection.
Source: Hacker News · Summarized by HeadlinesBriefing