Hallucination

A hallucination is when a language model produces something false while sounding exactly as confident as when it is right.

Think of a student who has decided that a blank answer is worse than a wrong one. Ask about a book they never read and you get a fluent, well-structured paragraph with a plausible plot, invented characters and a made-up publication year. Nothing in their tone tells you which parts are real. That tone is the dangerous part, not the error itself.

The mechanism is not a bug that someone forgot to fix. A language model predicts the next likely token given everything so far. "Likely" and "true" line up most of the time, because the training text was mostly accurate — but nothing inside the model checks a source, and there is no separate store of facts to consult. A citation that looks correct is generated the same way as a sentence that is correct: it is what usually follows.

Where it shows up hardest#

Safer          : summarising text you supplied, rewriting, translating, code structure
Risky          : specific numbers, dates, names, prices, legal or medical detail
Very risky     : citations, URLs, case law, API methods that sound plausible

Rare facts are the weak spot. Something mentioned a million times in training is well anchored; something mentioned twice is not, and the model fills the gap with the shape of an answer.

You cannot eliminate hallucination, but you can push it down a long way. RAG puts the real documents in the prompt and asks for quotes with sources, which turns a memory question into a reading question. Lowering the temperature reduces creative drift. Explicitly permitting "I could not find this in the provided documents" works better than most people expect. And for anything that carries consequences, a human check on the specific claims is the honest answer — not a better prompt.

Where to go next#

• Full lesson: Hallucination
• Related terms: llm, rag, inference, zero-shot