A driver’s license tells you who someone is and that they passed a driving test, not whether they’ll drive responsibly in the future. The same is true for bots: verifying who operates it creates accountability, but trust depends on what it does next. A known bot can still ignore website preferences, overwhelm origins, or evade controls. It’s not always malicious — often it’s just a misconfiguration or a bug. As we described in a previous blog post on bot behavior, we strive to verify two questions: “Are you who you say you are?” and “Can we trust you?” To be a Cloudflare Verified Bot, you broadly need to meet three criteria:
Identify yourself accurately and securely
Do only what you say you will do
Don’t engage in abusive behavior
Bots use Web Bot Auth to sign requests that establish their identity, which BotBase then catalogues in an open format. With identification addressed, our focus shifts to the second cardinal rule of Verified bots: thou shalt do no harm . Today we introduce a system to evaluate and report when Verified bot behavior starts to deviate from this rule, ensuring that the answer to “Can we trust you?” remains “Yes.” This is where the distinction between risk and trust becomes important:
Risk describes the likelihood that a particular request or point-in-time action may be harmful, and it can change from one moment to the next. Trust is built over time through repeated and consistent behavior. For example, we de-listed Perplexity’s bot from our Verified bot list in 2025 when it sought to access resources on websites that opted out of scraping and subsequently attempted to circumvent controls when those requests were blocked. Do not engage in abusive behavior
This simple requirement sounds like a freebie, but there is a lot of nuance in that statement. When crawlers don’t respect site preferences , that undermines trust. When sites request a crawler to not crawl or to slow down, and the crawler ignores these requests, that undermines trust. We developed a system to measure how often a crawler respects site preferences. These observations will translate to an explainable trust band: a range showing how consistently a bot respects site preferences and which observed behaviors informed that assessment, rather than a permanent judgment about a bot or its operator. The initial signals we evaluate are:
Does the bot respect instructions? Did it respond to direct feedback? We consider these to be “common sense” rules, a courtesy to the operators who provide the information bots seek. In the future, we will be sharing these scores publicly on Radar , as well as expanding the ways we measure. This score gives responsible bot operators a way to demonstrate that trust is being upheld over time. Verify, but continuously re-verify. When in Rome, do as the Romans do
A robots. txt file is the primary way to communicate site owner preferences. Our system considers whether a bot’s observed traffic is consistent with those published preferences over time. Robots. txt tells all (or specific) bots to only crawl specific parts of a website. Isolated violations and other types of one-offs may not be significant…