WHY AI HALLUCINATES (AND WHY THAT WON'T BE FIXED): EXPLAINED FOR OLDER ADULTS

Introduction

You already know that AI makes things up - we covered that in "Why AI Sounds Intelligent But Isn't" - but it's worth understanding just how confidently and convincingly it does this, and why no amount of better technology is going to fix it.

This isn't a temporary problem that'll be solved in the next version. It's fundamental to how the system works.

What hallucination actually looks like

AI doesn't just get vague things wrong; it invents specific, detailed information that sounds completely plausible. It might invent a scientific study that was never published, quote a book that doesn't exist, or give you a legal citation for a case that never happened (several lawyers have learned this the hard way, as we've already mentioned).

It might tell you a historical event occurred on a different date, describe a product feature that isn't real, or attribute a quote to someone who never said it. And it will do all of this with complete confidence - no hesitation, no "I'm not sure about this", just a clear, coherent, utterly wrong answer.

Here's a real example of how this works: if you ask AI "What did researcher John Smith say about AI in his 2019 paper?", and no such researcher or paper exists, the AI might still generate an answer because it knows what academic papers sound like and what researchers tend to say about AI. So it produces text that sounds like a real citation - "In his 2019 paper, John Smith argued that..." - and off it goes, inventing the rest.

Why "better training" won't solve it

People often assume that if AI hallucinates, it just needs to be trained on better data, or more data, or more accurate data. That's not how it works.

Training data determines what patterns the AI learns, but it doesn't give the AI the ability to distinguish true from false - even if you trained an AI exclusively on verified, accurate information, it would still hallucinate because hallucination happens when the AI is asked about something that wasn't in its training data, or when it misapplies a pattern it did learn.

Think of it this way: if you trained a person to memorise thousands of facts, they might still make mistakes when asked about something they didn't memorise. But a person knows when they're guessing and can say "I'm not sure" or "I think, but I'm not certain." AI can't because it doesn't have that self-awareness - it will generate an answer regardless, and that answer might be complete nonsense.

Why it can't just say "I don't know"

You might think the solution is obvious: just program the AI to say "I don't know" when it's uncertain. The problem is that AI has no concept of certainty or uncertainty - it's always predicting what words come next, and every prediction has a probability attached to it.

Some companies have tried to reduce hallucinations by fine-tuning their models to say "I don't know" more often, or by giving AI access to live search results so it can look things up. These approaches help reduce the problem, but they don't solve it because the AI still doesn't know what it knows - it's still predicting, not reasoning, and predictions will always include errors.

If you train it to be more cautious, it becomes less useful because it starts refusing to answer questions it could actually handle. If you train it to only answer when it's "certain", it refuses to answer most of the time because certainty isn't something it has access to. It's a fundamental trade-off: the more useful the AI is, the more likely it is to hallucinate.

Why this is permanent

Hallucination is baked into the design - it's the cost of having a system that can generate fluent, human-like responses to almost any question. You can reduce it through better training, better prompting, or giving the AI access to external tools, but you can't eliminate it because the system is fundamentally built to generate responses, not to verify them.

The only way to completely prevent hallucination would be to fundamentally redesign how AI works, and we don't currently know how to build a system that's both as flexible as current AI and capable of reliably knowing what it doesn't know. (And if I find out how to do this, I won't be posting it here. I'll be selling the knowledge to the highest bidder, believe me. I'd make millions!)

What this means for you

The practical takeaway is simple: never trust AI without verifying what it tells you, especially when it comes to specific facts, citations, or technical details.

If AI gives you a citation, check that the source exists and says what the AI claims. If it gives you a fact, confirm it elsewhere. If it writes you code, test it thoroughly. If it summarises a document, compare the summary to the original to make sure nothing important was dropped or distorted.

This doesn't mean AI is useless - it just means it's a tool that requires oversight. You wouldn't trust autocorrect to write a legal contract or trust a calculator if you didn't understand what calculation you were asking it to do, and you shouldn't trust AI to give you accurate information without checking.

Use AI for tasks where occasional errors aren't catastrophic, or where you're able to verify the output easily. Don't use it for anything where accuracy is critical and verification is difficult or impossible.

And if someone promises you an AI that doesn't hallucinate, they either don't understand the technology or they're hoping you don't.

Browse all topics → Index