A year ago, we evaluated five leading bot detection systems, including Proof of Human, Google reCAPTCHA v3, hCaptcha, Fingerprint Pro, and Cloudflare Turnstile, against automated browsers completing predefined tasks. Detection rates ranged from 33% to 87%. Systems combining behavioral signals with device-level signals generally outperformed those relying on device information alone.
Since then, browser automation has advanced considerably. Modern AI agents can interpret webpages, plan actions, and interact with interfaces through increasingly capable browser environments. We revisited the benchmark with an expanded set of adversaries, including conventional browser automation, stealth browsers, and AI-powered browsing agents. We tested Proof of Human, Google reCAPTCHA v3, hCaptcha, Cloudflare Turnstile, and Cloudflare Bot Management.
The results reveal similarly wide variation in detection performance. Each of the five adversaries attempted a five-question survey combining multiple-choice and open-ended responses 20 times, yielding up to 100 sessions per detection system. Some systems failed or crashed before session completion, leaving between 89 and 100 evaluable sessions for each.
The adversaries span increasingly sophisticated forms of automation. Chromium, a standard browser driven through Playwright, serves as the baseline "obvious bot" and retains common automation indicators. Cloak Browser, a stealth-focused Chromium fork, alters fingerprints at the source-code level and was run through a residential proxy with humanization features enabled. Browser Use, an open-source AI-agent framework powered by an LLM, plans actions from visual page information and operates an unmodified browser.
Source: Hacker News · Summarized by HeadlinesBriefing