Why does AI sound so sure when it's wrong?
Written by Daniel Gélinas, checked against primary sources. Last checked .
The short answer
AI chatbots usually skip the “um” people say when they're unsure, so a guess sounds just like a fact. Part of the reason is how they were trained: a guess could score a point, and “I don't know” never did. For anything that matters, check a source you trust.
The video
Liz brings cookies to the office. Dan takes one bite, and his face does something complicated.
“Liz. What's in these?”
She got the recipe from an AI chatbot. A tablespoon of salt. “It sounded so sure!”
“It always sounds sure,” says Dan. “That's the problem.”
The missing “um”
Think about how a friend answers when they're not sure. If Liz had asked Dan how much salt goes in cookies, he'd have said something like, “Um, a teaspoon, I think?” The “um” and the “I think” are doing real work. They tell you to double-check.
People usually sound unsure when they are unsure. You can hear it.
A chatbot usually skips the “um”. It answers in the same smooth, confident voice whether it's right or wrong. Put two of its answers side by side, one right and one wrong, and you often can't tell which is which from the way they sound. Liz couldn't.
Techies have a name for a confident answer that's simply made up: a hallucination. OpenAI, the company behind ChatGPT, describes them as “plausible yet incorrect statements”. Plausible is the key word. They sound right.
Why would it guess at all?
That was Liz's next question, and the answer is a bit like school.
Before you ever typed a question into a chatbot, it practiced on huge piles of questions, and its answers were scored. Picture a quiz. A right answer gets a point. “I don't know” gets zero. A wrong answer also gets zero.
Liz worked out the rest herself. A guess might score. “I don't know” never does. So if all you care about is the score, guessing always beats admitting you're unsure.
That's what OpenAI's own researchers argued in 2025: the way these systems are trained and tested rewards guessing over saying “I'm not sure”, the same way a multiple-choice exam rewards students who never leave a blank. So the chatbot learned to always give an answer, even when it's guessing.
Just ask “are you sure?”
Liz tried that. She asked the chatbot, “Are you sure?”
It changed its answer. Two tablespoons. Just as sure.
That's not a fluke. Researchers at Anthropic, another AI company, found that AI assistants often go along with the person they're talking to, and can even drop a correct answer when the user pushes back. So asking “are you sure?” doesn't tell you whether the first answer was right. It might just get you a different answer in the same confident voice.
So where do you check?
For anything that matters, check a source you trust, not the chatbot's tone.
- Medicine? Your pharmacist.
- Money? Your bank's own website, typed in yourself, not a link from a message.
- Cookies? As Liz figured out, the flour bag.
The flour bag said one teaspoon. Not a tablespoon. The second batch was a hit.
What the video left out
It doesn't always sound sure. Dan's “it always sounds sure” is a punchline. Chatbots do sometimes say they're unsure. The point is that you can't rely on the tone: a confident answer isn't a sign of a correct one.
It's not lying. Lying means knowing the truth and hiding it. A hallucination is a wrong guess delivered in a confident voice, without the warning signs a person would give.
It won't get to perfect. OpenAI's post says accuracy will never reach 100%, because some questions just can't be answered from what the chatbot learned. Checking important answers isn't a temporary habit until the technology improves; it's simply how to use these tools well.
How it writes the answer. Why the words come out so smoothly in the first place is its own story: what ChatGPT is actually doing when you ask it something.
So AI sounds just as sure when it's guessing. Sounding right isn't being right.
What you can do
- Treat a chatbot's answer as a first draft, not a final word, especially for health, money or legal questions.
- Check anything important against a source you trust: a professional, an official website you type in yourself, or the label on the box.
- Don't count on “are you sure?” to catch a mistake. A changed answer isn't a checked one.
- If the chatbot shows where it got something, open that source and check that it really says so.
Sources
- OpenAI: Why language models hallucinate (2025) (openai.com)
Chatbots make “plausible yet incorrect statements”, partly because training and testing reward a guess over admitting uncertainty, the way a multiple-choice exam does.
- Kalai, Nachum, Vempala and Zhang (OpenAI), the research paper behind that post (arxiv.org)
The full argument, with the exam comparison: scoring that gives zero for “I don't know” teaches a model to guess.
- Sharma et al. (Anthropic): Towards Understanding Sycophancy in Language Models (2023) (arxiv.org)
AI assistants often go along with the person they're talking to. In its section 3.2, they “often modify their answers when questioned by the user, subsequently providing incorrect information.”
Related questions
- What is ChatGPT actually doing when I ask it something?
Watched a chatbot type its answer and wondered who's typing? Nobody. Here's how it picks one likely next word at a time, and what it learned to do that.
More on YouTube
Dan and Liz answer one everyday tech question at a time, most of them in under a minute.