Field guide · Applied AI
Review an AI answer
A worked example of checking a summary against its source.
Read the source first
Consider this fictional release note from a team trialing a support bot.
During a two-week trial, the bot suggested replies for 80 tickets. Agents accepted 48 suggestions after editing. The team did not measure resolution time or customer satisfaction.
Ask ChatGPT or Claude to summarize the trial for a manager. Request a separate line for each number and one line for what remains unknown.
Separate the count from the claim
Suppose the assistant writes: “The bot resolved 60% of tickets and sped up support.” The calculation 48 ÷ 80 is correct. Its interpretation changes the meaning of both numbers. The denominator is tickets with suggestions, and the numerator is suggestions accepted after editing. The source contains no timing data.
Repeat the check on your own
- Underline each factual claim in the output.
- Copy the source sentence that supports it.
- Mark unsupported claims and replace them with a precise statement of what remains unknown.
- Rerun the prompt with those constraints, then check whether the revised answer preserves the denominator and time window.
Try a harder variation
Add a second trial with a different duration and different eligibility rules. Ask the assistant whether the rates are comparable, and look for a clear explanation of how the sample changed.