We use AI tools constantly here — they're not the enemy in this post, they're more like well-meaning coworkers we lean on every day. So when we spent an afternoon asking Perplexity, ChatGPT, Gemini, and Google AI Mode to grade our own site for AEO (Answer Engine Optimization), we expected useful, if imperfect, feedback. We got that. We also got something more interesting: every single one of them made the exact same specific, checkable mistake.
First, the scores didn't agree with each other
We ran the same salon-beauty-blueprint.html page through three different tools on the same day. No edits in between.
| Tool | Grade |
|---|---|
| Perplexity | 72/100 |
| ChatGPT | 84–87/100 (B+) |
| Gemini | 91/100 (A-) |
That's a 19-point spread on content that didn't change. Our domain-wide grade told the same story — Perplexity landed at 79, ChatGPT at 91, Gemini at an A-, Google AI Mode at a B+. Four tools, one live site, no consensus. Lesson one, gently: a single AI tool's 0-100 score is a vibe check, not a measurement. Fun to see, not something to chase.
Then we noticed the pattern
What was actually interesting wasn't the score spread — it's that we could check some of their specific claims against our own source code, because it's our site. And every time a grade mentioned our structured data (JSON-LD schema), it was wrong.
"Add JSON-LD Schema Markup... FAQPage and Product or Service schema"
The page had 12 validated FAQPage questions live in schema at the time of grading, matching the visible FAQ accordion exactly.
"Incorporating explicit FAQ schema (FAQPage)... on service or pillar pages"
FAQPage schema was already deployed across all 16 industry blueprints, all 18 audit pages, the 5-page AI-search content cluster, the main FAQ page, the homepage, and the incident log by the time this was written.
"The site lacks formal JSON-LD schema for optimal answer-engine citation"
Same gap, same tool family, same answer: not accurate at the time it was said.
"Would likely benefit from... Organization, Service, FAQPage"
The homepage has a ProfessionalService schema block (the Organization-family type, with three Service-typed offerings inside its catalog) and an 11-question FAQPage. All three "missing" things were live and validated.
Four for four. We weren't trying to catch anyone — we only noticed because we happen to know exactly what's in our own head tag.
The tool told us its own method
The most useful moment of the whole afternoon was Perplexity's own follow-up answer, when asked directly about the homepage's schema a second time:
That's not a gotcha, that's an honest admission of method. "From the text alone" means: reading rendered, visible content, not parsing the actual <script type="application/ld+json"> blocks in the page source. If that's genuinely how these tools evaluate a page when a URL is dropped into a conversation, then any claim they make about a page's structured data needs a real source-code check before anyone acts on it — including us, and including anyone reading this who's about to trust a similar grade for their own site.
We still like these tools. A lot.
This isn't an argument that AI graders are useless — plenty of the non-schema feedback across all four tools was genuinely good and we're implementing parts of it: visible "last reviewed" dates, a "who this is for" section, tighter lead sentences. The schema miss is a specific, narrow blind spot, not a referendum on the whole exercise. We just think it's more useful to name the blind spot precisely than to pretend every AI-generated audit is either gospel or garbage.
The standing rule this produced
Same discipline we apply to every claim on this site, extended to AI-generated feedback about the site itself: no "your schema is missing X" claim gets acted on without checking the actual live file first. Five for five wrong this round is enough of a pattern to write down, not enough to make us stop asking these tools for feedback — just enough to always verify before we trust.
Try the AI Citation Checker →