It feels like every single week, there's a new
viral headline about artificial intelligence
just making something up. Oh, absolutely. It's
literally everywhere right now. Right. Like a
chat bot invents a massive corporate scandal
out of nowhere, or it confidently gives you detailed
driving directions to a restaurant that actually
burned down 10 years ago. Yeah, and it almost
always comes packaged with this very specific,
almost medical -sounding label. The media developers,
pretty much everyone says the AI is hallucinating.
Okay, let's unpack this. Yeah. Because we are
going straight to the source today to figure
out what that actually means for you. I really
need to, yeah. We've pulled together a comprehensive
Wikipedia article simply titled, Hallucination
in Artificial Intelligence. And our mission for
this deep dive is to demystify justified the
actual mechanics of what is happening inside
these machines when they lie to us. Right, the
actual gears turning in the background. Exactly.
We are going to explore the hilarious and sometimes
incredibly high -stakes consequences of these
glitches in the real world. And perhaps most
surprisingly, we're going to look at why this
exact same glitch, the very thing causing so
much panic in the tech world, might actually
be the key to Nobel Prize -winning science. which
is just wild to think about. But when I hear
the word hallucination, I instantly picture like
a psychedelic drug trip. Sure. Yeah. Or a severe
fever dream. Right. And framing it that way just
feels completely wrong for a computer program
running on silicon chips. It doesn't have a brain.
It really doesn't. If I had to use an analogy,
AI hallucination feels a lot less like a psychological
break from reality and a lot more like that one
overly confident friend you have at trivia night.
Oh, everybody has that friend. Do you know the
one? The friend who, rather than just admitting
they don't know the capital of some obscure country,
will just invent an entire historically plausible
war just to sound like they know what they're
talking about. That is actually a remarkably
accurate way to think about the behavior. Yeah.
And what's fascinating here is that the word
hallucination wasn't always used to describe
a machine making a mistake. Wait, really? It
wasn't a negative thing? No, not at all. To really
understand why AI does this, we have to look
at the evolution of the term itself because it
used to mean the exact opposite. How so? Well,
if you go back to the 1980s in the field of computer
vision, hallucination actually had a highly positive
connotation. I mean, I would imagine hallucinating
in computer vision means the machine is seeing
things that aren't there, like obstacles in the
road. How could that be positive? So think about
a really blurry, heavily pixelated, low resolution
photo of a human face. Maybe it's from like an
old security camera. OK, yeah, super grainy.
Exactly. Researchers developed algorithms to
automatically add high -resolution details to
that image. It was filling in the gaps to make
it look sharp and realistic for human analysts.
Oh, wow. So, like, enhancing it. Right. And they
literally called this process face hallucination.
The first documented use of the term in this
way was in a PhD thesis by Eric Majolsnes back
in 1986. 1986. That is way earlier than I expected.
Yeah, it goes way back. The AI was mathematically
synthesizing details, pixels, really that weren't
strictly in the original data, but it was doing
it in a helpful, desired way to reconstruct an
image. So it was essentially an enhancement tool.
It was generating helpful filler, not deceiving
the user. Exactly. It was a feature, not a bug.
OK, so when did the tech industry start using
it to describe this modern negative failure mode
that we see today? The semantic shift really
began in the 2010s. Researchers who were working
on statistical machine translation, you know,
getting computers to translate languages, started
noticing a weird phenomenon. What kind of phenomenon?
Well, sometimes the models would just spit out
factually incorrect or totally misleading outputs
that had absolutely nothing to do with the source
text. Just completely making up translations.
Right. The math was prioritizing linguistic fluency
over factual accuracy. It wanted to sound smooth
more than it wanted to be right. That sounds
exactly like the trivia friend. Exactly. And
in 2015, computer scientist Andrej Karpathy explicitly
used the term hallucinated to describe his language
model generating a completely fake, yet perfectly
formatted citation link. A fake link. Wow. Yeah.
And then the term just absolutely exploded in
popularity during the recent AI boom, especially
after OpenAI released ChatGPT in late 2022. Right,
because suddenly millions of people were experiencing
these incredibly confident falsehoods firsthand.
It wasn't just researchers anymore. Exactly.
It was right there on your phone or your laptop.
Reading through the source material though, it's
pretty obvious that not everyone is happy with
the tech industry adopting that word. There is
a massive amount of pushback from the scientific
community. Oh, there is a lot of pushback. Researchers
like Mary Shaw and statistician Gary and Smith
argue that using the word hallucination is deeply
dangerous. Dangerous? Because it heavily anthropomorphizes
the software. It forces us to treat a machine
like it has a biological human mind. Which it
definitely does not. Right. Smith points out
that these large language models, or LLMs, do
not actually understand what words mean. They
don't have sensory perception. They're just math.
Just math. So they physically cannot experience
a hallucination in the perceptual, psychological
sense that a human does. Honestly, calling it
a hallucination seems like it gives the AI entirely
too much credit. It almost feels like a PR spin.
That is exactly the concern Mary Shaw raised.
Because it lets developers off the hook for bad
programming by framing a critical software error
as this quirky psychological human flaw. I mean,
it sounds a lot better to say my AI is hallucinating
rather than my software is broken and producing
garbage. It absolutely softens the blow by spinning
objective errors into idiosyncratic quirks. And
because of this danger, researchers have proposed
several alternative terms in the scientific literature.
Like what? What should we be calling it? Well,
you'll see words like confabulation, delusion,
or fabrication. But my personal favorite comes
from a paper published in the journal Ethics
and Information Technology by Hicks, Humphries,
and Slater. Oh wait, I remember this from the
reading. They used philosopher Harry Frankfurt's
definition of bullshit, didn't they? They did.
They literally defined AI outputs as bullshit.
And they aren't just swearing for shock value,
either. They are applying Frankfurt's rigorous
philosophical framework. OK, I love this. Explain
the framework for us. So Frankfurt argued that
a liar is someone who knows the truth and actively
tries to hide it. Right. There's an intent to
deceive. Exactly. But a bullshitter is someone
who is completely indifferent. to the truth.
Oh, that makes so much sense. Right. The AI model
doesn't care what is real and what isn't. It
is just mathematically generating text. If a
statement happens to be true, it is accidentally
true. If it is false, it is accidentally false.
So it's just the ultimate bullshit engine. That's
exactly what it is. So if it's not a psychological
hallucination and it's just indifferent mathematical
text generation, we really need to look at the
gears turning in the background. We do. If you
take an LLM like GPT at its core, It's essentially
just an incredibly complex next word predictor,
right? That is essentially it. These models ingest
a massive amount of text during their training.
When you ask them a question, they aren't looking
up facts in a database like a traditional search
engine. They're not Googling the answer. No,
not at all. They are calculating the statistical
probability of what the next word should be in
a sequence over and over again. It sounds a lot
like the predictive text feature on a smartphone
keyboard. Yes, that's a great comparison. Like
when you start typing a text message, your phone
suggests the next word to keep the sentence going.
based on your previous habits and LLM is doing
that but on a massive highly articulate scale.
Very articulate. It just wants to keep the conversation
going and it is heavily incentivized to give
a guess even when it totally lacks the underlying
factual information. And that incentive creates
a fascinating tension inside the model between
two competing goals, novelty and usefulness.
Novelty and usefulness. OK, break that down.
Researchers actually draw a parallel to human
creativity here. If an AI system is focused entirely
on usefulness, it will just regurgitate and memorize
safe, boring content. So it would just sound
like a sterile robotic textbook. Exactly. Super
boring. But if the system prioritizes novelty,
you know, trying to generate diverse, unique,
natural -sounding text, it drastically increases
the chances of generating original but entirely
inaccurate responses. It makes things have to
sound interesting. Yeah. But how does that actually
work in practice? Like, how do developers force
a math equation to be novel? They use decoding
strategies. And the most common one is something
called top -case sampling. Top -case sampling.
Yeah. Normally, an AI would just pick the single
most mathematically probable next word. Always.
Which would be the boring textbook output. Right.
So to make it sound more conversational and human,
developers forced the AI to randomly select from
a pool of the top K most likely words. Oh, I
see. So instead of the number one word, it picks
from a group. Exactly. Say the top 40 words.
By injecting that literal dice roll into the
selection process, the AI sounds vastly more
dynamic. But I'm guessing that dice roll is where
the problem starts. You guessed it. The data
shows that this exact injection of randomness
is positively correlated with increased hallucinations.
It branches off into fiction. Because it's trying
so hard to sound interesting and conversational
that it physically overrides its own accuracy.
Exactly. But do we actually know what is happening
on a microscopic level inside the neural network
when it decides to make something up? Like, can
we see it happening? We are actually getting
much closer to understanding that microscopic
level. In 2025, the AI company Anthropic published
some groundbreaking interpretability research
on their LLM clod. Interpretability research,
meaning they are trying to interpret the black
box. Yes. They essentially took a flashlight
under the hood of the neural network. They weren't
just guessing about probabilities anymore. They
mapped millions of artificial neurons to find
specific internal circuits. Wow. And what did
they find? They found a specific circuit designed
to inhibit the AI, to physically stop it from
answering a question if it doesn't have the factual
data. So there's an actual, I don't know, circuit.
Basically, yeah. By default, the circuit is active,
meaning the AI stays quiet. But when the AI determines
it has enough information, that circuit is inhibited,
and it speaks. Okay, so a hallucination must
be a misfire of that exact inhibition circuit,
right? Precisely. They found that if Claude recognizes
a specific token, like say the name of a somewhat
famous person, but it doesn't actually have sufficient
data about that person's life, the circuit incorrectly
inhibits itself anyway. Oh no, so it gets overconfident.
Yeah. The AI essentially says, oh, I recognize
that name. I can definitely answer this. It completely
fails to check if the connected factual tokens
are actually present and just proceeds to generate
a highly plausible, completely untrue biography.
Just totally wings it. Yeah. Knowing those mechanics
makes the bizarre real world outputs we see make
so much more sense. It really does. If the AI
is constantly playing this tug of war between
being mathematically safe and randomly selecting
words to sound creative, you are going to get
some wild swings. It truly is the Wild West of
AI outputs right now. That tension produces some
incredible artifacts. Some of the examples in
the source material are purely hilarious. There
was this data scientist named Teresa Kubaka who
literally made up a gibberish phrase. Oh, the
cycloidal inverted electromagnet. Yes, the cycloidal
inverted electromagnet. She asked ChatGPT to
explain it. Yeah. And instead of the inhibition
circuit firing and the AI saying, hey, that's
not real, the AI invented a deeply scientific,
plausible sounding answer. Complete with fake
academic citations, right? Yes. It sounded so
authentic, she actually had to double check to
make sure she hadn't accidentally typed the name
of a real physics phenomenon. It's a perfect
illustration of the model prioritizing conversational
flow and pattern completion over fact checking.
It saw a prompt that looked like a science question,
so it generated a response that looked like a
science answer. Exactly. There was also the time
someone asked for proof that dinosaurs built
a civilization. I remember that one. The AI just
rolled with it. It claimed there were fossil
remains of dinosaur tools and that they created
engravings on stones. Which is amazing. But my
absolute favorite example from the sources, and
you're gonna love this, a prompt convinced Chet
GPT that churros, like the fried dough pastries,
are ideal cools for home surgery. Home surgery
with a churro. Yes, the AI boldly claimed there
was a study published in the prestigious journal
Science proving that churro dough is pliable
enough to form surgical instruments. Oh my god.
You've been added that the cinnamon sugar flavor
has a calming effect on patients. That is just
incredible. The churro surgery is funny, sure,
but it perfectly illustrates a really dangerous
mechanical flaw called a cascaded error. A cascaded
error. Let's dig into that because it generates
text one word at a time, right? Right. So once
it predicts that first false word about the churro
being a medical tool, it essentially poisons
its own well. Yeah. It has to use its own generated
falsehood as the context for the next word. Yes,
exactly. It locks itself into the narrative.
Once it affirms the absurd premise of the prompt,
it is mathematically forced to double down on
the lie to maintain consistency. Ah, I see. That
is exactly how it ends up inventing fake science
journal articles. It needs evidence to support
the initial random hallucination it just made.
But here's why this actually matters to you,
the listener. We can laugh about dinosaur civilizations
and pastry surgery all day. Right. It's entertaining.
But if these models are fundamentally guessing…
And we know they are prone to these cascaded
errors where they trap themselves in lies. Why
are highly paid professionals blindly trusting
them with critical high stakes tasks? That is
the multi -million dollar question. And the real
world consequences have already been severe.
We are seeing major professional disasters across
multiple industries. Yeah, the sources had some
wild examples. Take the airline industry, for
example. In February 2024, Air Canada was taken
to a civil resolution tribunal. Because of their
chatbot, right? Exactly. Their customer support
chatbot had hallucinated a bereavement fare policy.
It told a grieving customer they could retroactively
request a bereavement discount within 90 days
of ticketing. Which was just entirely false.
And Air Canada actually tried to argue in court
that the chatbot was a separate legal entity
responsible for its own actions, which is just
a wild legal defense. It really is. And the tribunal
completely rejected that defense, ordering the
airline to pay damages and honor the hallucinated
policy. The precedent is set. The company is
responsible for the text its tools generate.
Then there's the legal world, the 2023 case of
Mata v. Avianca. Oh, that was a huge deal. A
lawyer named Steven Schwartz used ChatGPT to
write a legal brief, and he submitted six fake
case precedents that the AI completely fabricated.
Sex of them. Yeah. Schwartz claimed he didn't
know ChatGPT could make things up. He even noted
that when he asked the AI if the cases were real,
the AI assured him they were. Which is another
textbook example of the cascaded error. You ask
the bullshitter if it is lying, and it mathematically
doubles down on the lie. Exactly. Judge P. Kevin
Kestel ultimately dismissed the case, fined Schwartz
$5 ,000, and described the A .I.'s legal opinions
as gibberish that bordered on nonsensical. We
have also seen massive financial waste in the
consulting world. In late 2025, the Australian
government received a $440 ,000 report from Deloitte
that contained hallucinated academic sources
and fake court quotes. Wow, $440 ,000 for fake
quotes. Yeah. And just a month later, a $1 .6
million Canadian health care report, also by
Deloitte, was found to contain at least four
false citations to non -existent research papers.
The sources also highlight the friction these
outputs cause. across other domains. There was
a 2023 defamation lawsuit where a gun rights
activist, Mark Walters, sued OpenAI after Chad
GPT falsely claimed he was accused of embezzlement.
Right. And for context, in 2025, a judge ruled
in favor of OpenAI in that specific case. Yes.
There was also an instance where Google's Gemini
generated images of racially diverse 1940s German
soldiers, which led Google to temporarily pause
the tool's image generation of people entirely.
Yeah. That made a lot of headlines. And we mentioned
these strictly to show the broad spectrum of
areas, from corporate consulting to law to historical
image generation, where unpredictable probabilistic
outputs clash with our fundamental need for factual
accuracy. It's a clash that is happening everywhere.
We have clearly seen the damage this confabulation
causes when we need hard facts. But if we pivot
and look at the broader landscape... what happens
when we want the AI to make things up? Right,
if we actually want it to invent. Exactly. If
we connect this to the bigger picture, the line
between a hallucination and a scientific discovery
can actually be incredibly thin. It really can.
This part blew my mind. The sources detail the
2024 Nobel Prize in Chemistry. David Baker's
lab at the University of Washington intentionally
used AI hallucinations to design 10 million brand
new proteins that do not exist anywhere in nature.
10 million? That is just staggering. Nature only
created a fraction of that over millions of years.
This led to roughly 100 patents and the founding
of over 20 biotech companies. And the Nobel committee
specifically rewarded this work, though notably
they avoided using the word hallucination. They
did. They opted to call it imaginative protein
creation. Which is brilliant, framing by the
committee, honestly. And it is not just proteins
either. At Caltech, researchers deliberately
use AI hallucinations to design a novel catheter
geometry. A catheter geometry. Okay, what did
the AI come up with? The AI hallucinated a design
featuring sawtooth -like spikes on the inner
walls. And it turns out when they physically
tested it, those microstopic spikes actually
prevent bacteria from gaining traction. Oh, wow.
So it stops infections. Yes. That hallucination
has the potential to fight urinary tract infections
globally. We see similar methods in meteorology,
too, where scientists use AI to generate thousands
of subtle forecast variations to discover unexpected
weather factors. So it essentially boils down
to the between brainstorming mode and fact -checking
mode. Exactly. If I want to write a legal brief,
an AI hallucination is a disaster. It's literally
perjury. But if I am in a chemistry lab trying
to invent a new medicine, an AI hallucination
is just a really creative, highly advanced hypothesis.
That is a crucial distinction. The scientific
applications of hallucinations are fundamentally
different from chatbot hallucinations. How so?
Because of the data? Yes. Chatbots operate on
messy, ambiguous human language pulled from the
internet. But the AI models used by these scientists
are, as Caltech professor Anima Anankumar points
out, taught physics. Ah, okay. So they have real
-world constraints. Right. Their hallucinations
are strictly grounded in physical reality, and
they are always validated through rigorous, real
-world laboratory testing. Which brings us right
back to the everyday user. Since hallucinations
can be both revolutionary in a lab and incredibly
dangerous in a courtroom or a business report,
how do we rein them in for the rest of us? That
is the big challenge right now. Especially in
places like schools, where we need the AI to
be a tool, not a cheat code, without breaking
the system entirely. The classroom crisis is
very, very real. A 2024 study at the University
of Mississippi found that 47 % of AI generated
citations submitted by students were completely
or partially fabricated. It's 47%. That's almost
half. Yeah. Featuring incorrect titles, wrong
dates, or totally fake authors. It is causing
massive academic integrity concerns globally.
And we can't just rely on AI detectors to catch
the fabricated text. Right. Definitely not. Because
the Wikipedia article notes that tools like Turnitin
frequently flag innocent papers written entirely
by human students. The detection technology is
so fundamentally flawed that OpenAI completely
shut down its own AI detection software due to
a lack of accuracy. They did. It just wasn't
reliable enough. So how are developers actually
trying to mitigate the hallucinations at the
source? If we can't detect it, how do we stop
it? There are several highly technical methods
being tested right now. Data cleaning is the
most obvious one. Just giving you better data.
Right, ensuring the model isn't learning from
bad information in the first place. But there
are also some really fascinating structural approaches.
Researchers are setting up systems where different
chatbots debate one another. Wait, they have
them argue? Yeah. Model A generates an answer.
Model B critiques the logic and looks for flaws,
and then Model A revises its answer based on
the critique, repeating until they reach a consensus.
That's actually brilliant. It is. Another method
forces the AI to actively validate its own low
-confidence answers by querying the Bing search
API, essentially making the AI Google its own
thoughts before responding to the user. But there
has to be a massive computational catch to all
those fixes. Oh, there is. Every time you ask
a question, if the AI has to generate three different
drafts, have two sub -models argue about the
logic, and then do a live web search. You have
just tripled or quadrupled the electricity and
processing power required for a simple prompt.
That is the ultimate trade -off in the industry
right now. Evaluating multiple possible replies
or checking search engines drastically increases
the computational cost and the financial cost
of every single query. And is that worth it?
In high -stakes fields like medical diagnostics
or chip design, yes, that extra time and money
is definitely worth it. But for a general consumer
chatbot, the data shows an uncomfortable truth.
What's the truth? Customers actually prefer rapid,
overconfident answers over slow, cautious, uncertainty
-aware ones. Oh, wow. So are the tech companies
failing to fix the problem or are we, the users,
the actual problem? Because our behavior rewards
the AI for being a fast, confident liar rather
than a slow, cautious truth -teller. This raises
an important question about the nature of the
technology itself. Many top researchers propose
that hallucinations aren't just a bug we can
eventually patch with a software update. They
think it's permanent. They believe hallucination
is an innate, unavoidable limitation of large
language models. Because of how they are built
as probabilistic prediction engines, they will
always have the mathematical capacity to make
things up. So as you continue to use these generative
tools in your own work, your studies, or just
your daily life, This approach is to completely
stop treating the AI like an omission, all -knowing
oracle. Absolutely. It is not a search engine.
Instead, treat it like a brilliant, incredibly
fast, but over -eager intern. An intern who really
wants to please you, who can brainstorm amazing
ideas, but who requires heavy, constant supervision
and rigorous fact checking before you publish
their work. I think that is the perfect mindset
to have and it leaves us with one final deeply
philosophical thought to mull over. Let's hear
it. If AI hallucination is, at its core, just
the mathematical equivalent of imagination, is
it even possible to cure AI of hallucinating
without completely destroying its ability to
be creative? Oh, that's a wild thought. Think
back to your overly confident trivia friend.
If we somehow build a machine that is physically
incapable of ever telling a lie, do we inadvertently
build a machine that is incapable of ever inventing
anything new? Definitely something to think about
the next time your chatbot tries to convince
you to do surgery with a churro. Thanks for joining
us on this deep dive. We'll see you next time.