More than half of web traffic now comes from software agents, not humans. These agents skip ads and subscriptions, leaving websites with rising costs but no revenue from those requests. Cloudflare, sitting as a reverse proxy for a large share of the internet, is uniquely positioned to change this dynamic.

Cloudflare classifies automated traffic by behavior: search indexing, real-time agent actions, and AI training. Its first solution was blocking AI crawlers outright. Then it introduced Pay Per Crawl, letting site owners charge a flat per-request fee for crawler access, with Cloudflare handling billing.

Now Cloudflare is developing the x402 protocol to settle identity, permission, and payment within a single HTTP request. This enables anonymous micropayments, moving value collection from after the request (ads) to during the request itself. The approach turns the old block-or-allow binary into a three-option model that lets content earn from agent traffic at network scale.

Key open questions remain around adoption and pricing fairness, but Cloudflare has demonstrated a practical path for websites to monetize AI crawlers effectively.