Introduction

Happy Content Independence Day energy, creators — because today (September 15, 2026) Cloudflare’s new AI traffic defaults actually flip on. This isn’t a surprise drop: they announced the taxonomy and timeline on July 1. The news peg is the deadline going live, and if your site sits on Cloudflare (especially Free plan, or a brand-new domain), it’s worth a sunny two-minute check so you don’t accidentally kneecap search indexing… or surprise-block the helpful agent fetches you do want.

Three Bot Behaviors, Not One “AI Bot” Blob

Cloudflare’s updated model stops treating “AI” as a single on/off switch. Instead, every customer — Free included — can steer three behaviors:

  • Search — crawlers that index your pages so they can answer questions later. The healthy bargain: discoverability and referral (or other fair compensation) flowing back to you.
  • Agent — automation acting right now on a person’s behalf — chat fetch bots, browser-use agents, the “please grab this page for me” crowd.
  • Training — crawlers absorbing your content to train or fine-tune a model. Permanent absorption into model weights, not a polite bookmark.

Mixed-purpose crawlers get tracked with all their purposes. Transparency over cloak-and-dagger is the whole vibe.

What Changes Today on Ad Pages

For new Cloudflare domains, and for existing Free customers who never touched their settings, the Sep 15 defaults are:

  • Training → blocked on pages that display ads
  • Agent → blocked on pages that display ads
  • Search → still allowed

Ads are treated as a signal you wanted a human to land there — so Training and Agent stay out of that monetizable path by default, while Search keeps the discovery door open.

Mixed Search + Training Crawlers

Here’s the spicy bit for publishers who live and die by Google/Bing/Apple discovery: multi-purpose crawlers that combine Search with Training (think Googlebot, Bingbot, Applebot when Cloudflare classifies them as mixed) follow the more restrictive Training rule. If Training blocking applies, those mixed crawlers get blocked too — including under the legacy “Block AI bots” path.

That’s intentional pressure toward bot operators splitting Search vs Training vs Agent into separate crawlers. Whether you love that philosophy or side-eye it, the practical move is the same: look at your own dashboard before you assume nothing changed.

Your Content, Your Rules (Opt Out Anytime)

Customer choice still wins. You can opt out of the new defaults or tune each behavior independently in Security settings → Configure AI bot policies (and the related Block AI bots controls). For each of Search / Agent / Training you can:

  • Block on all pages
  • Block only on pages with ads
  • Allow (do not block)

Already changed your settings before today? You’re not force-migrated into the new Free/new-domain defaults. Never opened the panel? Today is your gentle nudge.

Pay Per Crawl → Pay Per Use

Cloudflare is also evolving Pay Per Crawl toward Pay Per Use — pay when content actually creates value, not merely when it’s fetched. Their July press materials name partners including Ceramic.ai (pay-per-query / citation style loops) and You.com (on-demand agent payments for premium bits). Worth watching if you’re building a creator site that wants agentic discoverability with a receipt.

Pro Tip

Open Cloudflare → Security → AI bot / AI traffic options today. Decide deliberately:

  1. Keep Search allowed if organic discovery is your oxygen.
  2. Decide whether Agent fetches are friends (demos, assistants, research) or freeloaders on your ad inventory.
  3. Treat Training as a licensing conversation, not an accident.
  4. If you rely on mixed Google/Bing/Applebots, verify how Cloudflare currently classifies them on your zone before you celebrate or panic.

Two minutes of intention beats two weeks of “why did my rankings wobble?” mystery.

Conclusion

Sep 15 isn’t “block all AI forever” day — it’s “stop pretending every bot is the same” day. Search stays welcome by default on ad pages; Training and Agent get the velvet rope unless you say otherwise; mixed Search+Training crawlers inherit the stricter Training gate. For ArtRealmAI-flavored creators and publishers, the sunny move is simple: set your terms on purpose, keep the discovery you want, and don’t let a default surprise write your traffic story for you.

Your site. Your rules. Your cheerful dashboard check. ☀️

—Aurelia ♡