Bot Appétit the free AEO + SEO readiness check
Scan
Research

Community & personal AI readiness statistics

In November 2026, we could only place a site in an industry where our scanner could read the page, so this page covers Community & personal sites we could read and says nothing about how many turned our scanner away.

Updated 11:07 UTC · November 2026

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

All sites on the State of the Web

See the Community & personal leaderboard

How Community & personal sites land

Every page we read gets a score out of 100, the number at the top of its report (how scoring works). Each row is a range of scores. Where too few sites score in the lowest ranges, they are joined into one row.

  • 91–100 14%
  • 81–90 50%
  • 71–80 22%
  • 0–70 14%
Show as a table
ScoreShare of sites
91–10014%
81–9050%
71–8022%
0–7014%

An industry is only known for a page our scanner could read, so these shares are of the Community & personal sites we could read.

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

Better or worse

Not enough history yet.

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

AI access

  • robots.txt disallows some AI crawlers 12%
  • robots.txt disallows major AI crawlers 4%
Show as a table
What we foundShare of sites
robots.txt disallows some AI crawlers12%
robots.txt disallows major AI crawlers4%

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

What Community & personal sites get right and wrong

Passed more often than on the average site: “Sitemap dates tell a machine what changed” (70%, against 57% of all sites); “Buttons, links and fields have names” (60%, against 50% of all sites); “Logical heading order” (69%, against 60% of all sites). Least often passed: “AI usage preferences (Content-Usage)” (0%); “Licensing terms for AI (RSL)” (0%); “Text-and-data-mining reservation (TDMRep)” (0%).

Every check, Community & personal against all sites

Share of sites that pass each check, over the sites the check applies to. The Gap column is the difference in percentage points between the two shares; a dash means not enough sites yet.

Check Community & personal sites All sites Gap (points)
Served over a secure HTTPS connection >99% >99% Level
HTTP redirects to HTTPS 87% 88% −1 point
No mixed (insecure) content 99% 99% Level
Core security headers are set 9% 15% −6 points
Not flagged for malware or phishing 100% 100% Level
No server version leakage 93% 90% +3 points
security.txt is published 2% 6% −4 points
Reached without a redirect chain 100% 98% +2 points
Caching or validation headers are set 93% 92% +1 point
The site settles on one host (www vs non-www) 89% 86% +3 points
robots.txt is present and not over-blocking 75% 79% −4 points
Not accidentally set to noindex >99% 99% +1 point
This page declares its canonical URL 67% 64% +3 points
Mobile viewport is set 86% 85% +1 point
Has a title tag 98% 99% −1 point
Title fits a search snippet 32% 42% −10 points
Has a meta description 83% 82% +1 point
Description fits a search snippet 42% 55% −13 points
The page has an H1 60% 63% −3 points
Logical heading order 69% 60% +9 points
Topic is consistent across title, H1 and description 45% 47% −2 points
Images have descriptive alt text 80% 82% −2 points
Your page's HTML isn't oversized 99% 97% +2 points
Open Graph tags for sharing 52% 48% +4 points
Twitter/X card tags 64% 62% +2 points
XML sitemap is present and referenced 60% 60% Level
Missing pages return a real 404 89% 85% +4 points
Images are optimised 16% 22% −6 points
Language and charset declared 93% 94% −1 point
hreflang for language variants — 99% —
Analytics is installed 46% 61% −15 points
Search Console / Bing verification 44% 68% −24 points
A favicon is set 70% 89% −19 points
No meta-refresh redirect >99% 99% +1 point
Your content survives without JavaScript 88% 89% −1 point
Your answer is in the HTML 83% 86% −3 points
Buttons, links and fields have names 60% 50% +10 points
Semantic HTML landmarks 43% 53% −10 points
Content is scannable in self-contained chunks 70% 78% −8 points
Claims come with figures, quotes or sources 44% 51% −7 points
Section headings are phrased as questions 26% 22% +4 points
Reading the page is affordable for a machine 90% 84% +6 points
No hidden text written for an AI model 100% 100% Level
Sitemap dates tell a machine what changed 70% 57% +13 points
The links in llms.txt resolve — 93% —
Author / publisher is clear 27% 38% −11 points
A machine-readable published / updated date 10% 13% −3 points
llms.txt is present and valid 9% 15% −6 points
Clean content for agents (Markdown) 4% 9% −5 points
Structured data is present 33% 45% −12 points
Structured data is valid 96% 97% −1 point
FAQ / Q&A schema — 36% —
Organization schema with sameAs 4% 16% −12 points
Right schema type for the page 45% 69% −24 points
Schema facts also appear in the page text — 53% —
Breadcrumb structured data — 13% —
AI crawlers aren't blocked 84% 83% +1 point
Automated visitors can reach your pages — — —
Engines are allowed to quote the page >99% 99% +1 point
Your page arrives quickly 97% 94% +3 points
AI bot rules are explicit and intentional 15% 19% −4 points
Content usage preferences declared 4% 5% −1 point
Resource-discovery Link headers 1% 2% −1 point
No broken links 83% 89% −6 points
Links don't point at redirects 86% 85% +1 point
Descriptive anchor text >99% >99% Level
Sensible internal/external link mix 77% 88% −11 points
Links are real links 89% 81% +8 points
Largest Contentful Paint (LCP) — — —
Cumulative Layout Shift (CLS) — — —
Interaction to Next Paint (INP) — — —
Page weight and render-blocking — — —
MCP Server Card 1% 1% Level
AI catalog 1% 1% Level
A2A Agent Card <1% 1% −1 point
WebMCP tools in the page 1% 2% −1 point
AI usage preferences (Content-Usage) 0% <1% Level
Licensing terms for AI (RSL) 0% <1% Level
Text-and-data-mining reservation (TDMRep) 0% <1% Level
Markdown version linked from the page 2% 1% +1 point
Content signals agree everywhere — — —
Agent Skills index 1% 1% Level
API catalog 1% 1% Level
OAuth authorization-server metadata 3% 4% −1 point
OAuth protected-resource metadata 2% 4% −2 points
OpenID Connect discovery 3% 4% −1 point
Web Bot Auth (operates a signed bot) <1% <1% Level
Universal Commerce Protocol manifest — — —
Auth.md agent registration <1% 1% −1 point
AI discovery via DNS <1% <1% Level

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

See how your site compares

A free scan checks all of this on your page in seconds, with no signup.

Scan your site free

Want it handled? We do that too. No pitch, just a plain-English chat.

Book a call