What an AI Content Audit Can—and Cannot—Tell You About Citations

What does an AI content audit actually measure?

An AI content audit evaluates a page against a defined set of questions and criteria. Depending on the method, it might assess answer coverage, clarity, supporting evidence or gaps that deserve human review.

Before using a score, check which questions and criteria produced it.

A score without that context is difficult to act on. A page could score poorly against a broad question set even though it serves a narrow purpose well. Another could mention every topic while giving readers very little useful detail.

I would keep these four measurements separate in an audit report:

If a system shows its final answer but no retrieval records, we cannot confidently describe every page it considered.

Likewise, finding a citation does not establish that someone clicked it. Receiving an enquiry does not automatically reveal which earlier interactions influenced the person.

Can API results measure ChatGPT citations?

An API test tells you what happened in that API setup. It does not automatically tell you what someone would see in the ChatGPT website or app.

An October 2026 preprint, Understanding ChatGPT Citations, examined four ChatGPT Web settings and compared observations with Google Search and the OpenAI Responses API. The combined proxies recovered roughly a quarter of cited URLs in the two settings evaluated for this comparison. The results matched more closely for websites and named entities than for individual page URLs.

The paper also counts supporting URLs attached to selected references, including some not displayed separately. Its figures therefore cover more than the links a user could see in an answer.

The study is observational and specific to its sampled questions, models and collection period. It did not test Fanout or establish that a content coverage score predicts citations.

In a client report, label these results “observed in our API test” and document the setup.

Where Fanout fits into this workflow

When I was researching answer engine optimisation (AEO) and generative engine optimisation (GEO), I came across an explanation of query fanout: how a question can lead to several related searches.

I wanted to explore whether those related questions could help a website owner spot incomplete answers on an existing page.

That became the starting point for building Fanout.

Fanout uses multiple models to generate likely related questions, groups them and assesses the supplied page’s coverage. The findings give an editor possible updates, supporting pages and internal links to review.

These are suggested reader questions. They do not show which searches an AI platform actually ran.

Agreement between models can help identify a question worth inspecting, but it does not establish search volume or commercial importance. Several models can produce a similar suggestion that is irrelevant to the business.

Each suggestion still needs an editorial decision: accept it, revise it or discard it.

Does a better content score mean more AI citations?

A higher score means the assessment rated the content more favourably. Before reporting it as progress, check that the scoring method and questions stayed consistent.

Before reporting a score change, I would ask:

• Did we compare the old and revised page against the same questions?
• Did the scoring instructions or model change?
• Did a person verify the newly covered answers?
• Were citations measured separately before and after the edits?

Consider a pricing page. Adding a long explanation of an industry’s history might satisfy a generated question but make pricing harder to find. A stronger edit could be a short, accurate explanation of what the price includes.

Write the brief around the missing answer. For example: “Explain whether the monthly price includes onboarding, and have the account manager verify the terms.” That gives the writer a task they can complete and the reviewer something concrete to check.

How should you validate AI content recommendations?

I recommend these five checks before turning an automated finding into a publishing task.

1. Check whether the question belongs on this page

Write down the page’s purpose and intended audience. Then ask whether answering the suggested question helps that audience complete the task.

Some questions belong elsewhere. Others do not belong on the website at all.

2. Verify that the answer is missing

Read the relevant passage yourself. An apparent gap could be an extraction problem, an answer inside a table or an explanation phrased differently from the generated question.

Record the passage that supports your decision. “Already answered here” is a valid audit outcome.

3. Check the rest of the website

An existing guide might already provide the answer. In that situation, a short explanation and a useful internal link may serve the visitor better than duplicating the guide.

Assign each accepted question a destination before creating new content.

4. Look for evidence of reader demand

Use whatever relevant evidence is available: Search Console queries, sales conversations, support questions, internal search or customer interviews.

A support question can reveal a customer need without proving substantial search volume. Note where the evidence came from.

5. Confirm that the business can support the answer

Some recommendations require information the writer cannot invent: eligibility, pricing conditions, service areas, technical limitations or delivery commitments.

Ask the responsible person to verify those details. When an answer cannot be supported, leave it out or explain the uncertainty.

An illustrative example: reviewing a delivery page

This is a fictional example, not a client case study.

Suppose a local meal delivery website has a page about ordering weekday lunches. An audit suggests answering: “Can I change my delivery address after placing an order?”

After publishing, we can verify that the answer is visible, accurate and accessible. We can also reassess coverage using the original question set.

That establishes a content improvement. To find out whether the page gets cited, we need a separate test.

How would I measure citation visibility after an update?

Here is the measurement plan I would use, starting before the page is edited.

Record the change and the questions

Save the previous page, note the publication date and describe exactly what changed. Choose relevant questions before looking at the outcome, including questions whose answers might reasonably use the page.

Use the same questions before and after the update. Track any new questions separately so you are comparing like with like.

Document the testing conditions

For each observation, record the prompt, platform, visible model or mode, date, language and relevant account or location settings. Save the answer and source links.

Keep API tests separate from website or app tests. Note any settings you cannot control or identify.

Use repeated observations

Repeat the selected questions on a defined schedule. Report how many answers you collected, including answers that did not cite the website.

Separate exact-page citations from citations to other pages on the website. Also distinguish a brand mention without a link from a citation to an owned page.

A simple report could show the number of completed observations, how many cited the target page and which questions produced those citations. Call this a result for the sampled questions, not a market-wide visibility score.

Track business outcomes separately

Where attribution is available, review visits from AI services, the landing pages people reach and subsequent enquiries. An optional “How did you hear about me?” question can add context, although it will not produce perfect attribution.

Google says traffic from its AI search features is included in Search Console’s overall Web reporting. Those totals alone cannot isolate an article’s AI-feature performance. See Google’s guidance on AI features.

Be careful with causal language

If citations rise after an update, report the timing and conditions. Other things may have changed too: competing pages, platform behaviour or the questions represented in the sample.

Report “cited more often in these tests” unless you have evidence that isolates the effect of the edit.

What should an AI content audit leave you with?

Ask for an action list you can hand to a writer or site owner. Each item should name the page, the unresolved question, the evidence supporting the recommendation and the proposed edit. It should also identify anything that needs approval from the business.

That is how I want Fanout to be useful: helping people decide which answers deserve work, and giving them enough detail to do that work well.

You can read the story behind Fanout for more context. If you want help applying this process to an existing page, request a Fanout audit.