How to fact-check anything ChatGPT tells you (a 5-minute everyday skill)

A friend of mine asked ChatGPT for a quote to open a wedding speech. It gave him a lovely one, beautifully phrased, attributed to a famous author. He almost used it. On a whim he searched the exact words — and the author had never said them. The model had invented a plausible-sounding quote and stapled a famous name to it, with total confidence and zero hesitation.

That's the thing about AI tools: they're not lying, exactly. They don't know they're wrong. And that's precisely why you need a quick way to check them — not a paranoid, distrust-everything way, just a five-minute habit that catches the embarrassing stuff before it reaches anyone else.

Why AI is confidently wrong (in plain language)

Here's the one idea that makes everything else make sense. A chatbot is not a database you're looking things up in. It's a prediction machine. At every step it's essentially asking, "given everything written so far, what word most plausibly comes next?" and producing that.

That's why it's brilliant at plausible. Plausible is literally what it optimises for. The problem is that plausible and true are not the same thing, and the model has no separate sense of which is which. A real citation and an invented one feel identical to it, because both are just sequences of likely-looking words.

So when an AI tool is wrong, it isn't wrong the way a person who's unsure is wrong — hedging, mumbling, looking uncomfortable. It's wrong with the exact same smooth confidence it uses when it's right. The confidence carries no information. That's the trap, and once you internalise it, the rest of this is easy.

If you want the bigger picture on using these tools to actually learn rather than to quietly outsource your thinking, I wrote about that in learning with AI without rotting your brain. This post is the narrower, practical companion: catching the errors.

Habit 1: Isolate the checkable claims

A typical AI answer is a mix of three things: genuine facts, reasonable-sounding interpretation, and outright invention. Your first move is to separate them.

Read the answer and mentally underline the specific, checkable claims — names, dates, numbers, quotes, "studies show," cause-and-effect statements. Those are the things that can be true or false. The fluffy connective tissue around them ("this is an important consideration") can't be fact-checked because it doesn't claim anything.

You don't need to verify the whole answer. You need to verify the load-bearing claims — the two or three facts the conclusion actually rests on. Find the claims that, if false, would change your decision. Check those.

Habit 2: Lateral reading — open a tab

Here's a technique borrowed from professional fact-checkers, and it's almost annoyingly simple: when you want to know whether something is true, leave the source and open a new tab.

Don't sit there interrogating the AI about whether it's sure (it'll happily reassure you, and that reassurance is worthless — see Habit 4). Instead, take the claim, open a search engine or a known-reliable site, and look it up independently. Reading across sources beats reading down a single one.

For my wedding-speech friend, lateral reading was a ten-second job: copy the quote, paste it into a search engine in quotation marks, see if any real source attributes it to that author. Nothing came back. Case closed. The fastest fact-check is usually a second tab, not a follow-up question.

Habit 3: Distrust suspicious precision and "it is well established"

Two specific tells are worth training your eye for.

The first is suspiciously precise numbers. When an answer says something happened in "1847" or that "73.6% of respondents" did something, that crisp specificity feels authoritative — but it's exactly the kind of detail a prediction machine fabricates most convincingly. Real precise figures come with a source. Invented ones come with a confident tone and nothing behind them. Precision is not proof.

The second is authority-by-phrasing. Watch for "it is well established that," "studies have shown," "experts agree," "research consistently finds." These phrases do real rhetorical work — they make a claim feel settled — while naming no actual study, expert, or piece of research. Often they're padding draped over a claim the model can't actually back up. When you see one, treat it as a flag, not a fact. Which studies? Which experts? If you can't find them, the claim is unsupported.

Habit 4: Cross-check with a second model

If you've got access to more than one AI tool, here's a cheap and genuinely useful trick: ask the same question of a different one.

This works because the two models tend to fail in different places. If both give you the same answer, your confidence can reasonably go up a notch. If they disagree — and they will, surprisingly often, on facts — you've just learned that at least one of them is wrong, which is exactly the warning you wanted. Disagreement is a gift. It tells you precisely where to go do the real verification.

One thing this does not mean: asking the same model "are you sure?" That's not cross-checking. A model told to second-guess itself will often cheerfully "correct" a right answer into a wrong one, or double down on a wrong one, depending on how you phrased the nudge. Its sense of its own confidence is not reliable. Cross-check across tools, not within one.

Habit 5: Always click the citation

This is the big one, and it deserves its own habit because of a specific, well-documented failure: AI tools fabricate sources. They'll produce a citation that has everything — a real-sounding author, a plausible title, a journal, a year, sometimes even a link — and the whole thing is invented. The link 404s, or worse, leads somewhere real that says nothing of the kind.

How common is this? In legal settings, where the stakes are high and someone bothered to measure carefully, the numbers are sobering. A 2024 Stanford study (RegLab and the Institute for Human-Centered AI) tested general-purpose models — GPT-3.5, Llama 2, and PaLM 2 — with more than 200,000 queries each, and found hallucination rates ranging from 69% to 88% on specific legal questions. On one task — identifying a court's central holding — the models were wrong at least 75% of the time. That's general-purpose chatbots, not purpose-built legal-research products, which do better; but it's a vivid illustration of how readily these systems manufacture authoritative-sounding nonsense when you push them into a domain that demands precision.

The fix is unglamorous and bulletproof: click the citation and confirm the source says what the AI claims it says. Not that the source exists — that it actually contains the claim. An uncomfortable amount of the time, the link is dead, or it's real but irrelevant, or it says something subtly different. A citation you haven't opened is not evidence. It's decoration.

This matters everywhere, but it matters most for students handing in work and professionals putting their name on something. A fabricated reference in an essay or a client report isn't a small slip — it's the kind of thing that quietly destroys trust.

The 30-second checklist

Here's the whole thing distilled into something you can actually keep in your head. Run it on the two or three claims your decision actually rests on — not the whole answer.

THE 30-SECOND AI FACT-CHECK

1. CLAIM      Is there a specific, checkable claim? (name, date, number, quote)
2. TAB        Open a second tab and look it up independently
3. PRECISION  Is the number suspiciously precise with no source? Be suspicious
4. PHRASING   "Studies show / well established" with no actual study named? Flag it
5. CITATION   Click it. Does the source actually say this? Not just exist — say it

Five steps, well under a minute once it's a habit, and it will save you from the great majority of confident-but-wrong moments these tools serve up.

The recap

• AI is a prediction machine, not a database. It's optimised for plausible, and plausible isn't true. Its confidence tells you nothing.
• Isolate the checkable claims and verify only the ones your decision rests on.
• Lateral reading — open a second tab — beats interrogating the AI about itself.
• Be suspicious of suspicious precision and "studies show" with no study named.
• Cross-check across two tools, never by asking one model if it's sure.
• Always click the citation and confirm it says what was claimed. Fabricated sources are common enough that this single habit catches a lot.

None of this is about distrusting AI into uselessness. It's the opposite — once you can catch the errors quickly, you can use these tools far more boldly, because you're no longer one confident hallucination away from embarrassment.

If you'd like to get genuinely sharp at this — telling solid output from confident nonsense in your own field — it's one of the things I most enjoy working through in 1-on-1 sessions, because the best examples are always the ones from your own work.