Imagine yelling at your smart speaker to turn
off the living room lights, but leave the porch
on. And instead, it cheerfully sets an alarm
for 3 a .m. Oh, yeah. We've all been there. Right.
Like we have computers that can instantly calculate
the precise orbital trajectory to land a rover
on Mars. Yet they completely fail to understand
a basic sentence spoken by a human in a kitchen.
It is surprisingly common. It really is. Yeah.
So why is that? Today, we are diving into that
exact frustration. Welcome to today's deep dive.
Glad to be here. Our mission today is to explore
a comprehensive Wikipedia article on natural
language understanding, or NLU. We're going to
figure out how scientists and engineers have
been trying, honestly, for decades, to teach
computers to genuinely comprehend our messy,
complicated human language. And it is a massive
challenge. You probably interact with some form
of NLU every single day, whether you realize
it or not. Oh, for sure. But, you know, there
is a fundamental difference between a computer
simply recording the audio of your voice and
a computer actually comprehending your intent.
OK, let's unpack this. Because... Natural language
understanding is categorized in the sources as
a very specific subset of a broader field, right?
Yeah, natural language processing. Right, NLP
within artificial intelligence. But NLU deals
strictly with something called machine reading
comprehension. Exactly. That distinction is vital.
I mean, in the field of artificial intelligence,
NLU is considered what we call an AI hard. problem.
AI hard. I mean, that sounds deliberately intimidating.
Well, it's meant to convey the sheer scale of
the obstacle. It means that to truly solve this
problem, you essentially have to make a computer
as intelligent as a human being. Wait, really?
Just to understand language? Yeah. I mean, we've
actually gotten incredibly good at turning spoken
audio into written text, but that's just transcription.
Right, just copying it down. Exactly. NLU is
about automated reasoning. It's about taking
those transcribed words, extracting the actual
intent, and, you know, using it for large -scale
content analysis or complex question answering.
It is deriving actual meaning, not just copying
letters. So let me push back on this a little
bit. Is this just a machine doing advanced keyword
matching? How do you mean? Well, think of a student
hunting for vocabulary words in a textbook without
actually reading the chapter. They scan the page,
find the bolded word, write down the sentence,
but they have absolutely no idea what the underlying
concept means. Is that what these computers are
doing? Historically, and even in many common
systems today, yes, that is exactly what they're
doing. Just advanced keyword matching. Right.
But true natural language understanding requires
something much deeper than that student hunting
for a bolded word. It requires the system to
build an internal representation of the semantics
of a sentence. Let's define that for a second,
because we hear syntax and semantics thrown around
a lot. Syntax is just the grammar. Right? The
rules of how words are ordered. Spot on. Syntax
is relatively easy for a machine. Noun, verb,
adjective. Sure. But semantic is the actual logic
and meaning behind those words. True NLU relies
on mapping that meaning into something called
first -order logic. And what does first -order
logic look like in plain English? Well, it's
a way of translating our messy language into
strict mathematical relationships. OK. So instead
of just seeing the words a dog barks, first order
logic forces the computer to establish a rule.
It says, for all entities, if an entity is a
dog, then that entity barks. Oh, wow. It breaks
it down that strictly. It has to. The machine
has to explicitly understand who is doing what
to whom, the relationship between the objects
and the context. It cannot just rely on the fact
that the sentence is grammatically correct. And
because achieving that level of true reading
comprehension, getting a machine to map out relationships
like a human is so absurdly difficult. Early
computer scientists had to rely on some pretty
brilliant illusions to make computers appear
as though they understood us. Oh, absolutely.
They were forced to work within very strict artificial
constraints. If you can't teach a machine the
whole world, you teach it a tiny fake world.
Which brings us to the 1960s. The early days.
Yeah, the source material highlights this pioneering
program called Student, written in 1964 by Daniel
Bobrow at MIT. This was just eight years after
the term artificial intelligence was even coined.
It's wild to think of. about. And student could
actually take simple natural language input and
solve algebra word problems. Which was hailed
as a massive breakthrough at the time. But it
worked specifically because the world it was
operating in algebra was extremely restricted.
Right. Highly logical and mathematically predictable.
Exactly. So then a year later, in 1965, another
MIT researcher, Joseph Weissenbaum, writes a
program called ELIZA. And this one is famous.
Oh, very famous. Eliza was an interactive program
that carried on a dialogue in English, acting
as a psychotherapist. You would type in your
problems, and Eliza would respond. And people
genuinely poured their hearts out to this machine.
They really did. They felt deeply understood
by it. But the source notes that Weissenbaum
completely sidestepped the problem of giving
the program a real world database. Eliza didn't
have a rich vocabulary. Not at all. So if it
just used simple parsing and substituted keywords
into canned phrases, wasn't Eliza basically just
an interactive game of Mad Libs? Like, it was
just an illusion of empathy. What's fascinating
here is it was entirely an illusion. A brilliant
trick. Just a trick. Yeah, if you typed, I am
unhappy, Eliza would just stand for the syntax
I am, flip the pronoun, grab the next word, and
plug it into a pre -written template. So it would
just spit out, why are you unhappy? Exactly.
It had zero understanding of what happiness or
unhappiness actually - meant. Yet, despite being
a toy project based on keyword substitution,
it became wildly popular. People loved it. They
did. In many ways, Eliza served as the blueprint
for early commercial customer service bots like
ask .com. The illusion of understanding was enough
to fool us. And the history of these restricted
microworlds just gets more interesting. Like
in 1971, Terry Winograd writes a program called
SHRDLU. Right, that's S -H -R -D -L -U. Yes,
SHRDLU. It could understand English sentences
well enough to direct a robotic arm to move children's
blocks around on a computer screen. It was incredibly
visual. Yeah. You could literally tell it. Find
a block. which is taller than the one you are
holding, and put it into the box. And the machine
would execute the physical action. But notice
the constraint again, though. It proved a machine
could parse language and map it to a physical
action, but only within a restricted world of
children's blocks. You didn't need to understand
poetry or politics. Or sarcasm, right? It only
needed to understand geometric shapes, colors,
and spatial relationships, like on top of or
inside. There's also this incredible tridia in
the sources about the lineage of this tech. A
developer named Wayne Ratliff developed a program
in the late 70s called Vulcan. Oh, the Star Trek
connection. Yes. He built it with an English
-like syntax, specifically because he wanted
to mimic the voice -activated computer from Star
Trek. He wanted to talk to his computer like
they did on the Enterprise. The science fiction
inspiration is always there in early AI. And
that program, Vulkan, eventually evolved into
a system called D -Base, which practically launched
the entire personal computer database industry.
It's a huge legacy. Even the massive software
company Symantec originally started out in 1982
as a natural language understanding company.
They were trying to build a natural language
interface for database queries. But they pivoted
away from natural language fairly quickly. Right,
because the computer mouse was invented. Graphical
user interfaces took over. It was just easier.
Yeah, it was suddenly so much easier to just
point and click a folder icon than to struggle
with the friction of typing a perfectly grammatically
correct English sentence to ask the computer
to open a file. That makes perfect sense. I mean,
why fight with early clunky NLP when a mouse
click guarantees the machine understands your
intent? Exactly. But moving past those early
decades, researchers realized that tricks like
ELISA or tiny sandboxes like SHRDLU were dead
ends for real -world application. They couldn't
scale it. Right. To handle actual, everyday human
language, they had to build an incredibly complex
architecture under the hood. Here's where it
gets really interesting. Because the amount of
grueling manual labor required to build these
internal systems is just staggering. To even
begin to understand a sentence, a modern NLU
system needs a lexicon, a massive dictionary
of the language. But it's not just a standard
dictionary like you'd find on a bookshelf. The
source specifies it requires a rich lexicon with
a suitable ontology. Okay, let's pause there.
What exactly is an ontology in this context?
Think of an ontology as a massive, invisible
web of relationships. Like a mind map. Sort of.
A regular dictionary just tells you the definition
of an apple. An ontology tells the computer that
an apple is a type of fruit, which grows on a
tree, which is a plant, and an apple is related
to the action of eating. Ah, I see. It maps out
how every concept in the universe connects to
every other concept. And building that web is
no joke. The source explicitly mentions the word
net lexicon. It took many person years of pure
human effort to construct. Just unbelievable
amounts of labor. Human beings had to sit there,
meticulously mapping out the relationships between
tens of thousands of words by hand. Because that
lexicon is just layer one of the architecture.
Once you have the words and their relationships,
you need a parser and grammar rules to break
the sentences down. I picture a parser like a
diagramming a sentence back in middle school
English class. Remember drawing those lines to
separate the subject, the verb, and the prepositional
phrase. That's a great analogy. But the computer
has to do that instantly, almost in 3D, keeping
track of dozens of overlapping grammatical rules.
That sounds exhausting. It is. And even after
the parser diagrams the sentence, you arrive
at the most difficult layer, the semantic theory.
This is what guides the actual comprehension.
Because without a semantic theory, you just have
a beautifully diagrammed list of words. It's
like handing someone a perfectly translated dictionary
of foreign language. They have the words, but
no meaning. Exactly. They might know what the
words mean individually, but they don't know
the culture, the idioms, or the context. They
wouldn't know if a phrase was meant to be a literal
instruction or a sarcastic joke. Precisely. And
the source outlines that these semantic theories
operate on a spectrum. On one end, you have naive
semantics. Naive semantics. Yeah, this is a very
literal one -to -one interpretation. If you say,
my car died, naive semantics assumes your vehicle
was a living biological organism that has just
passed away. Which obviously causes major errors
in understanding. Right. So researchers try to
push toward the other end of the spectrum, utilizing
something called pragmatics. Pragmatics. Pragmatics
is the ability to look beyond the literal dictionary
definitions and derive meaning from context and
speech. speaker intent. Can you give an example?
Sure. If you are sitting at a dinner table and
ask, can you pass the salt? Pragmatics tells
the system you are making a request, not asking
a literal question about the person's physical
capability to lift a salt shaker. Right. Figuring
out pragmatics is hard enough for humans sometimes.
So we have this massive web of relationships,
the parser diagramming the grammar and the semantic
theory trying to guess the context. Then what?
Then advanced NLU systems attempt to incorporate
logic inference. They map this derived meaning
into a set of assertions using predicate logic.
Okay another big term, predicate logic. It connects
back to that first -order logic we discussed
earlier. Predicate logic is basically the mathematical
machinery used to evaluate if a statement is
true or false based on the variables. So back
to the barking dogs. Exactly. If the machine
establishes the rule all dogs bark and you tell
it Fido is a dog, the predicate logic engine
deduces the new truth. phytobarx. So it's learning.
It is actively deducing conclusions that weren't
explicitly stated. And this is where the underlying
programming language becomes a massive hurdle.
Yeah, the text gets into some heavy specifics
here about functional languages versus logic
-oriented languages. Why does the actual coding
language matter so much if they are all just
running on computers? Well, it changes how the
machine thinks at a foundational level. Take
a functional programming language like Lisp,
which was very popular in early AI. Functional
languages are designed to evaluate mathematical
expressions. They are not naturally built to
handle logical deductions like like Fido is a
dog. So how do you make it understand that? massive
subsystem on top of Lisp just to represent and
process those logical assertions. That sounds
clunky. It is. Think of it like buying a standard
car and trying to bolt airplane wings onto the
roof so it can fly. Right. It's heavy, complex,
and wasn't built for that purpose. But the source
contrasts that with logic -oriented languages
like Prologue. Right. Prologue is like buying
an actual airplane. Its core foundation is already
built on logical representation. Oh, I see. Yeah.
In Prologue, you don't write complex functions
to evaluate a phyto barks. You just declare the
fact. You literally write dog phyto. The logic
is baked into the foundation of the language
itself. OK. So if we have all of this today,
these massive word net lexicons that took years
to build, 3D parsers, pragmatic semantic theories,
and predicate logic mapped out in languages like
prologue, why do our smart speakers still completely
fail to understand us? Because of the ultimate
trade -off in this entire field, every system
built has to make a sacrifice on the matrix of
natural language understanding. The matrix of
breadth versus depth. Yes. This is the core bottleneck
of the technology. When we evaluate an NLU system,
we plot it on a chart with two axes. Okay, let's
visualize this. is the sheer size of the vocabulary
and the grammar it can handle. How wide is its
knowledge of the world? And depth. Depth is the
degree to which its understanding actually approximates
that of a fluent native human speaker. How well
does it comprehend the nuances, the pragmatics,
and the hidden logic of what it knows? So if
we picture this matrix, where do the things we
use every day sit? Like my earlier example of
telling a smart speaker to turn on the living
room lights. That sits squarely in the narrow
and shallow quadrant. Narrow and shallow. Yeah,
it's narrow because it only knows commands related
to timers, weather, and smart home devices. It's
shallow because it doesn't understand the nuance
of your request. It's just hunting for the keywords
turn on and lights. Minimal complexity. But what
about those old MIT programs? The ones stacking
blocks like SHRDLU? Those occupy the narrow and
deep quadrant. They were built primarily by researchers
to explore the actual mechanisms of human understanding.
They are incredibly deep. SHRDLU deeply understood
the spatial relationships and semantics of its
environment. but their application is impossibly
narrow. It only knows about blocks. So if SHRDLU
was so good at deeply understanding blocks, why
couldn't researchers just add more words to its
dictionary? Why not just expand its ontology
to include cars and restaurants and emotions
and make it understand the whole world? Because
the moment you expand the vocabulary, the amount
of ambiguity explodes exponentially. Oh, I didn't
think about that. Yeah. Block has one meaning
in a sandbox. In the real world, block could
mean a toy, a city street, a mental block, or
to obstruct someone's path. The deep systems
collapse under the weight of real -world ambiguity.
Which forces companies to build systems in the
broad and shallow quadrant. Exactly. This is
where massive commercial enterprise systems live.
Imagine a program that automatically reads and
routes millions of corporate customer service
emails to the right department. Like reaching
billing instead of tech support. Right. It has
to deal with a massive real -world vocabulary.
It is very broad, but it lacks true comprehension.
It's not deeply analyzing the customer's childhood
trauma or sarcasm. It's just classifying text
based on statistical patterns. It is shallow.
Which leaves the final Holy Grail quadrant broad
and deep. A system that knows every word and
topic in the human experience and understands
them with the pragmatic depth of a fluent native
speaker. The dream of AI. But the source material
is very clear on this. Systems that are both
very broad and very deep are currently beyond
the state of the art. Completely beyond it right
now. So what does this all mean? Because I have
to push back here on the idea that broad and
deep doesn't exist. What about a system like
IBM Watson? Ah. The source brings up Watson using
machine learning for text classification. When
Watson went on Jeopardy and beat the human champions,
didn't that prove machines can understand broad
and deep topics? It was answering incredibly
complex, pun -filled questions about history,
science, and pop culture. If we connect this
to the bigger picture, that specific question
is the philosophical debate tearing the AI field
apart right now. Really? Yes. Did Watson actually
understand those Jeopardy clues, or is it just
incredibly good at statistical prediction? The
source cites the famous philosopher John Searle,
who argued explicitly that Watson did not understand
the questions it was answering. How so? It gave
the right answers. It gave the right answers
by computing probabilities based on vast amounts
of data, not by understanding meaning. I'm not
sure I follow. Think about the word bank. When
Watson reads a clue with the word bank, it doesn't
vividly imagine a brick building with a vault
and security guards. It simply calculates that
in its massive database of text, the word bank
appears next to the word money, 85 % of the time,
and next to the word river, 15 % of the time.
Oh, wow. Yeah, it uses statistics to guess the
most probable missing piece of the puzzle. So
we are right back to the student hunting for
vocabulary words. Watson is just doing it at
lightning speed across a billion textbooks simultaneously.
Exactly. And the source includes the perspective
of cognitive scientist John Ball, who invented
Patom Theory. He strongly supports Searle's assessment.
What does he argue? Ball argues that NLP has
largely succeeded today by secretly narrowing
the scope of the application or by relying on
those statistical parlor tricks, not by actually
achieving broad and deep comprehension. Right.
Because there are still thousands of ways a human
can request something that completely defies
conventional statistical NLP. Because human language
isn't just a math equation of probabilities.
It relies on shared physical context, body language,
cultural shifts, and shifting definitions. The
article quotes an executive named Wide Wagemans
on this exact point. He says that to have a truly
meaningful conversation with a machine, the computer
must match every single word to the correct meaning,
based entirely on the meanings of the other words
in the sentence without relying on statistical
guesswork. Which, based on everything we've discussed
about parsers and ontologies and breadth versus
depth, sounds technologically impossible. But
as Wagemans points out, that impossible standard
is exactly what a three -year -old human does
naturally. A three -year -old? Yeah. A toddler
can understand shifting context, tone, and intent
without running massive statistical probability
calculations on a server farm. They just understand.
A machine currently cannot. That is incredibly
humbling. We have these massive supercomputers,
and they still can't match the true semantic
comprehension of a toddler. This debate about
whether machines truly understand our logic or
just predict our syntax really highlights where
we are in the history of computing. It is a profound
reality check on our technological progress.
So let's recap this journey we've been on. We
started in the 1960s with Eliza, proving that
simple keyword substitution could create a powerful,
albeit fake, illusion of empathy. A brilliant
trick. We moved through the decades, discovering
the grueling, multi -year manual labor required
to build vast ontological lexicons like WordNet,
and the necessity of establishing pragmatic semantic
theories and predicate logic systems. We witnessed
the evolution from clever parlor tricks to incredibly
dense structural architecture. And yet to Today,
we are still caught in that ultimate matrix of
breadth versus depth. We have systems that are
broad enough to scan millions of emails and systems
that are deep enough to perfectly understand
a tiny sandbox, but combining them into a machine
that can truly converse about the whole world
remains the holy grail currently beyond our reach.
The three -year -old standard remains unbroken.
It certainly does. So, for you listening, the
next time an automated system magically routes
your customer service email to the perfect apartment,
or completely fails to grasp a simple spoken
command to set a kitchen timer, you now know
exactly what is happening under the hood. You've
seen the matrix. Yeah, you know exactly where
that specific program sits on the breadth versus
depth chart. You know it's missing the complex
semantic modeling of a fluent speaker and why
it is still struggling to do what a toddler does
effortlessly. This raises an important question
and it's something to think about the next time
you interact with one of these systems. What's
that? Well, if achieving that true broad and
deep machine understanding is currently beyond
our engineering capabilities and computers constantly
struggle with our messy context -heavy language,
will we eventually stop waiting for the machines
to learn our language? That's interesting. Right.
Or will we subconsciously standardize and simplify
the way we speak, smoothing out our own messy
human quirks just to ensure the algorithms finally
understand us? Wow. We started by saying we like
computers because they are exact and predictable,
like a rigid calculator. But maybe, in our desperation
to be understood by them, we are the ones who
will start talking like the calculators. Thank
you for joining us on this deep dive. We will
see you next time.