HeadlinesBriefing favicon HeadlinesBriefing.com

CAPTCHAs Face New AI Challenge with Process Turing Test

Hacker News •
×

CAPTCHAs, long seen as a gatekeeper against bots, now face a new challenge. Researchers from a recent machine‑learning conference show that vision‑language models can spot patterns humans use to solve them, but they do so differently. While AI matches human accuracy, its click patterns diverge sharply.

The study spotlights statistical gaps in sequential clicks, direction shifts, and over‑selection, traits that distinguish human from agent behavior. By mapping these differences, the team created CogCAPTCHA30, a 30‑task battery blending classic CAPTCHA with cognitive challenges. The test measures both output and underlying process, revealing that similar answers mask divergent thinking in real world applications.

Comparing top models, the authors found that OpenAI’s GPT, Anthropic’s Claude, and Google DeepMind’s Gemini fell short of human‑like process patterns, whereas smaller open‑source models such as Qwen and Centaur matched humans more closely. The researchers argue that sheer scale does not guarantee human‑like cognition, challenging assumptions about progress in AI capability today for industry.

The team proposes a Process Turing Test, which evaluates whether an agent’s internal logic mirrors human reasoning rather than just matching outputs. Their experiments show that fine‑tuning models on observed behavioral features can close the gap, but the advantage disappears when feature sets are restricted or cross‑task generalization is required. This highlights limits of methods.