Research · October 2026 · Original data
State of AI Search for Small Business 2026
Everyone worries that small businesses are blocking ChatGPT. We checked 104 real sites. Almost none are. What actually keeps them out of AI answers is quieter: missing or broken llms.txt files, missing business schema, and firewalls that turn automated visitors away.
- Sites scanned
- 104
- Analysed
- 101 (3 excluded)
- Segments
- 36 dentists · 34 law · 34 home services
- Scanned
- 2026-10-19
- Data
- public files only
Key findings
- 01
3 of 101
sites block any AI crawler in robots.txt. Two of them block only Bytespider, ByteDance's training crawler. Just 1 blocks a crawler that powers AI answers (PerplexityBot), alongside a list of training bots. For small businesses, robots.txt is rarely the problem.
- 02
69 of 101
sites have no llms.txt. 32 serve something at /llms.txt, but 10 of those 32 serve something broken: 4 return an HTML web page, 3 are files with zero links and 3 fail the spec's one hard rule (start with an H1 title). 8 of the 28 real files were generated by Wix, Yoast or Rank Math.
- 03
30 of 93
readable homepages have no Organization or LocalBusiness-type JSON-LD, so nothing on the page tells a machine the business's name, address or phone in structured form. Only 35 of 93 use an industry-specific type such as Dentist, LegalService or Plumber; 8 have FAQPage schema and 18 carry review or rating markup.
- 04
8 of 101
sites returned HTTP 403 to our automated request, even when we retried with a normal browser user agent. Their robots.txt files (read from a second network) allow AI crawlers. We can't see whether the firewalls also refuse OpenAI's or Perplexity's own crawlers, which come from published IP ranges, but it's the first thing these owners should check.
- 05
23 of 93
readable homepages have neither a valid llms.txt nor business schema: no machine-readable summary of the business at all. Separately, 3 sites have no robots.txt file, which is harmless for access but means no sitemap pointer.
Also found and excluded from the numbers: one dentist's domain now 301-redirects every visitor to an unrelated online-gambling site, one law-firm site returns a raw server error, and one site never returned a page.
- Valid file
- Zero links
- Fails spec
- HTML page
- None
No llms.txt
No business schema*
Firewall 403 to our scanner
Blocks any AI crawler
*Of homepages we could read (93 of 101).
All the numbers
Download as CSV →| Metric | All sites | Dentists | Law firms | Home services |
|---|---|---|---|---|
| robots.txt blocks at least one AI crawler | 3/101 (3%) | 2/34 (6%) | 0/33 (0%) | 1/34 (3%) |
| robots.txt blocks an AI search/answer crawler | 1/101 (1%) | 0/34 (0%) | 0/33 (0%) | 1/34 (3%) |
| robots.txt blocks only Bytespider | 2/101 (2%) | 2/34 (6%) | 0/33 (0%) | 0/34 (0%) |
| No robots.txt file (HTTP 404) | 3/101 (3%) | 1/34 (3%) | 1/33 (3%) | 1/34 (3%) |
| Firewall returned HTTP 403 to our automated request | 8/101 (8%) | 5/34 (15%) | 1/33 (3%) | 2/34 (6%) |
| Serves something at /llms.txt | 32/101 (32%) | 9/34 (26%) | 10/33 (30%) | 13/34 (38%) |
| Valid llms.txt with at least one link | 22/101 (22%) | 4/34 (12%) | 7/33 (21%) | 11/34 (32%) |
| llms.txt present but has zero links | 3/101 (3%) | 1/34 (3%) | 1/33 (3%) | 1/34 (3%) |
| llms.txt present but fails the spec (no H1 title) | 3/101 (3%) | 2/34 (6%) | 1/33 (3%) | 0/34 (0%) |
| /llms.txt returns an HTML page | 4/101 (4%) | 2/34 (6%) | 1/33 (3%) | 1/34 (3%) |
| No llms.txt | 69/101 (68%) | 25/34 (74%) | 23/33 (70%) | 21/34 (62%) |
| Of sites serving /llms.txt: broken, empty or HTML | 10/32 (31%) | 5/9 (56%) | 3/10 (30%) | 2/13 (15%) |
| Of real llms.txt files: made by Wix, Yoast or Rank Math | 8/28 (29%) | 3/7 (43%) | 4/9 (44%) | 1/12 (8%) |
| Homepage has no Organization/LocalBusiness JSON-LD | 30/93 (32%) | 8/29 (28%) | 11/32 (34%) | 11/32 (34%) |
| Homepage uses an industry-specific schema type | 35/93 (38%) | 9/29 (31%) | 13/32 (41%) | 13/32 (41%) |
| Homepage has FAQPage schema | 8/93 (9%) | 1/29 (3%) | 3/32 (9%) | 4/32 (13%) |
| Homepage has review/rating schema | 18/93 (19%) | 3/29 (10%) | 10/32 (31%) | 5/32 (16%) |
| No valid llms.txt and no business schema | 23/93 (25%) | 7/29 (24%) | 9/32 (28%) | 7/32 (22%) |
Share or cite this
Free to quote with a link back. The CSV is licensed CC BY 4.0.
GeoLumen scanned 104 US small-business websites (36 dentists, 34 law firms, 34 home-service companies) on 2026-10-19; 101 were analysed. Only 3 of 101 block any AI crawler in robots.txt, and just 1 blocks an AI search crawler. The bigger gaps: 69 of 101 have no llms.txt, 10 of the 32 that serve one serve a broken, empty or HTML file, and 30 of 93 readable homepages have no Organization or LocalBusiness schema. Source: https://geolumen-six.vercel.app/research/state-of-ai-search-small-business-2026
Method and limits
Sample. 104 independent US small-business websites across 23 mid-sized cities: 36 dental practices, 34 law firms and 34 home-service companies (plumbing, heating and cooling, roofing, electrical and landscaping). It is a convenience sample, not a random one, so read the percentages as a snapshot rather than a national estimate.
What we fetched. On 2026-10-19, for each site: /robots.txt, /llms.txt and the homepage, with the same parsers that power our free AI Crawler Checker and llms.txt Validator. robots.txt was evaluated per RFC 9309 against every named AI crawler the checker tracks (OpenAI, Anthropic, Perplexity, Google, Apple, Common Crawl, ByteDance, Meta and others). A site “blocks” a crawler when its rules disallow the root path for that bot.
Definitions. “Business schema” means homepage JSON-LD with an Organization or LocalBusiness-family type. “Valid llms.txt” means a non-HTML file that starts with an H1 and has at least one link. “Firewall 403” means the homepage returned HTTP 403 to our data-centre request with both our bot and a browser user agent.
Exclusions. 3 sites were excluded: one unreachable, one returning a server error and one redirecting to an unrelated site. Schema figures use only the 93 homepages we could read.
Privacy. We read only public files. This page and the CSV contain aggregates only, with no business names, addresses or URLs.
Check your own site
Rather not do this yourself? We'll do the whole thing and send you the receipts.