Seeing AI crawlers in Cloudflare
If you use a CDN, it already sees the AI bots, including the ones that never reach your server.
If your site sits behind Cloudflare, or any CDN, that layer sees every request before it reaches your origin, including the AI crawlers covered on the AI crawlers page. That makes it the best place to look at crawler activity, for one reason in particular. A CDN serves many requests straight from its cache, so a bot can hit a cached page while your server never hears about it. Anything that reads only your server logs misses those. The CDN does not.
What Cloudflare shows you
Cloudflare classifies AI bots at the edge and reports them in its analytics, including a view that breaks out which tools are crawling you and how often. None of this depends on JavaScript or on anything you add to your site, because it happens at the network layer on the way in. If you want the numbers programmatically, the same data is available through Cloudflare's GraphQL Analytics API.
Why we don't pull this into Visibility Kit
An earlier version of Visibility Kit let you connect a Cloudflare token so we could pull your bot data and store it alongside everything else. We built it, ran it, and then removed it in 2026.
Two things made it not worth keeping. It duplicated data Cloudflare already shows you in its own dashboard, so we were copying numbers from one place to another without adding anything. And storing every crawler hit added up fast and became a reliability problem on our side, all to power a view that, in the end, did not change what anyone did. Crawler counts are a background signal. The visits worth acting on are the people AI sends you, and those we still track.
So if you want the crawler picture, go straight to Cloudflare's analytics. It is more complete than anything we would have pulled from it, and it is already in front of you.