Cloudflare's Crawler Rules Went Live in September. The Bigger Move Is the Agentic Internet (Part 1 of 2)
Cloudflare's three-tier crawler defaults went live September 15. Behind them sits a bigger plan: infrastructure for AI agents to discover, authenticate, and read your site. Part 1 of 2 covers the readiness layer and the protocols behind it.