You know, when you snap a picture of your dog
and save it to your phone or when you type just
a quick grocery list into a notes app, there
is this fundamental illusion at play. Oh, absolutely.
We tend to think the computer actually knows
what a golden retriever is or what a gallon of
milk is, but it doesn't. No, not at all. To the
machine, it's literally just storing ones and
zeros. It's essentially just a highly organized
filing cabinet. Yeah, exactly. Which brings up
a really massive question for you. to think about
today as we start this deep dive, how do we actually
teach computers to understand the world rather
than just passively store data about it? It is
arguably the defining question of artificial
intelligence. It really is. And to answer it,
we are going to look at a highly detailed source
today. It's this comprehensive breakdown on knowledge
representation and reasoning. Right, which you'll
often see abbreviated as KRR or KR squared. Exactly,
KR squared. So our mission for this deep dive
is to completely demystify how AI uses structured
information to actually solve complex tasks.
We're going to explore the history of this field,
the massive, honestly, almost comical hurdles
of trying to program common sense. Oh, the common
sense stuff is wild. It's so crazy. And we'll
get into how all of this is secretly shaping
the architecture of the Internet you use every
single day. Yeah. And what becomes clear very
quickly when you read through this material is
that Knowledge representation isn't simply about
writing lines of code. It really merges psychology.
Like how we as humans naturally solve problems.
Exactly, how we categorize our reality. It merges
that with mathematical logic, which is how we
automate reasoning. So by the time we finish
today, I really think this will fundamentally
change how you view every interaction you have
with a search engine or an AI assistant. Okay,
let's unpack this. To understand how a computer
thinks, we have to define what knowledge representation
actually is. Our source brings up a foundational
paper from 1993 by an MIT researcher named Randall
Davis, and he outlined five distinct roles that
define a knowledge representation framework.
But the very first role he mentions is the one
that really anchors the whole concept. Yeah,
it's the core of it all. He says that a knowledge
representation is fundamentally a surrogate.
It is a substitute for the thing itself. And
the core justification for doing this is that
you want an entity, in this case a computer system,
to be able to determine consequences by thinking.
rather than by acting. Right. You want it to
reason about the world rather than, you know,
taking physical action within it to see what
happens. So is that basically like running a
physics simulation in a video game before doing
a stunt in real life? Like the computer gets
to think about the crash and calculate the trajectory
and the force without actually crashing a real
million dollar car. Close. But there's a crucial
distinction to make there. A video game is mimicking
physics. Gnarly representation is actually about
mimicking reasoning itself. Oh. It's less about
calculating gravity or wind resistance and more
about the computer deducing why the car crashed
based on facts it holds. The computer builds
a surrogate reality of logic, not just a surrogate
reality of physics, but building that surrogate
introduces what early AI researchers realized
was the ultimate balancing act in this field.
It's this massive trade -off between expressivity
and tractability. Expressivity and tractability.
Let me try another analogy here to see if I grasp
that trade -off. It feels a lot like designing
a map. Okay. If you use something like First
Order Logic, which our source mentions is the
gold standard for high expressivity because it
can formalize basically all of mathematics. It's
incredibly powerful. It's like having a one -to
-one scale map of a city. It includes every single
blade of grass, every pebble on the street. It
is perfectly accurate. It is perfectly expressive.
But it is entirely useless to actually carry
around and read. It's intractable. That captures
the problem perfectly. First Order Logic is amazing.
But because of its high expressive power, it
allows for countless different ways of expressing
the exact same piece of information. Wow. So
when you feed that into a computer, the machine
spends a massive amount of computational power
just translating and managing all those different
expressions. Instead of actually solving the
problem. Exactly. It becomes intimidating for
human software developers to write and incredibly
inefficient for computers to process. You simply
don't need the full power of first -order logic
to get most jobs done. Right. So many of the
early AI knowledge representation formalisms
were basically just design decisions. They were
trying to figure out how to balance that expressive
power with actual workable efficiency. So if
first -order logic was too heavy and intractable
for computers to actually use back then, what
did early AI pioneers do? I mean, they couldn't
just abandon the idea of a digital brain, so
they must have started out with just massive
sweeping ambitions to try and solve this. Oh,
absolutely. If we go back to 1959, You see the
earliest work focusing on exactly that, general
problem solvers. Like universal brains. Yeah.
You had Alan Newell and Herbert A. Simon developing
a system literally called the general problem
solver, or GPS. VPS, right. It featured data
structures designed entirely for planning. It
would take a large goal, decompose it into smaller
sub -goals, and then construct strategies to
accomplish each one. And in that exact same year,
John McCarthy proposed the Advice Taker, which
attempted to implement common sense reasoning
using formal logic. But reading through the history
here, the field was incredibly fractured on how
to actually achieve this. Like in the 1970s,
there was a massive turf war. Huge. On one side,
you had purists who wanted AI to solve problems
using pure mathematical logic. But on the other
side, researchers at MIT argued that knowledge
isn't just math. It's procedures. It's a set
of instructions. Right. The conflict between
logical representations and procedural representations.
The MIT researchers completely rejected the idea.
that a computer should just look at a giant database
of facts and mathematically deduce an answer.
They thought that was the wrong approach. Totally.
They advocated for the procedural embedding of
knowledge. To put it simply, they believed knowledge
should be encoded as a set of rules or instructions.
Right. Think of it like baking a cake. A logical
representation would just give the computer the
chemical properties of flour, sugar, and heat
and expect it to deduce a cake. Which is insane
for a computer to do. Right, whereas a procedural
representation says, step one, mix flour and
sugar. Step two, apply heat. That makes so much
more sense. And this debate was eventually settled
by the development of logic programming languages
like Prolog, which managed to kind of blend these
two approaches. But wait, I have a pointed question
here. If early AI pioneers were trying to build
a literal general problem solver, why did they
suddenly pivot in the 1970s and 80s to hyper
-specific expert systems. Wasn't that a massive
step backward in ambition, like abandoning the
dream of a general AI to build something that
only does one specific task? What's fascinating
here is that it wasn't a retreat at all, though
I know it might look like one on the surface.
It definitely looks like one. Sure, but it was
a highly practical realization driven by the
cognitive revolution in psychology. Researchers
like Ed Feigenbaum in North America realized
that if you actually wanted an AI to match human
competence on a specific task, like diagnosing
a medical condition, general purpose reasoning
just wasn't enough. Exactly. Humans don't solve
complex medical problems just by being generally
smart. They solve them because they went to medical
school and acquired thousands of highly specific
facts. Right. So to make an AI useful, broad
and shallow rules had to lose out to narrow and
deep knowledge. So the AI field shifted its focus
from building a universal brain to building highly
educated specialists. Yes. And this pivot to
expert systems gave us the architectural terminology
that AI still uses today. They divided these
systems into two distinct parts. First, the knowledge
base. which holds the raw facts and rules about
a specific problem domain. And second, the inference
engine, which is the mechanism that takes the
knowledge in that base and applies it to answer
questions or solve problems. So they literally
split the storage of facts from the logic of
how to use those facts. Precisely. That makes
complete sense for something bounded like diagnosing
a blood disease. You give the knowledge base
all the symptoms of blood diseases and the inference
engine matches a patient's symptoms to the right
disease. But the source points out that while
expert systems were amazing in a laboratory for
those specific tasks, they failed completely
at everyday human life. Well, yeah. Medical diagnosis
is complex, but it's bounded by the walls of
a hospital and the biology of the human body.
Everyday life is completely unbounded. Right.
Which brings us to the challenge of encoding
common sense. Here's where it gets really interesting.
In the mid -1970s, Marvin Minsky developed a
concept called frames to try and tackle this
exact problem. Yes, frames are so important here.
A frame is described as an abstract description
of a category. It has these slots for data and
constraints on what can actually fit in those
slots. And the example in the source that really
helped me visualize this was ordering food in
a restaurant. The restaurant frame is just a
fantastic way to understand how we try to take
computers context. So a frame is basically like
a social Mad Libs template. If I walk into a
restaurant, the restaurant frame activates in
the AI. It automatically knows to expect a menu,
a waiter, and food. Exactly. It instantly narrows
the search space so the computer isn't suddenly
preparing for a medical exam or trying to solve
a calculus problem. But what happens if something
unexpected occurs? What if the waiter drops a
plate? That is exactly why frames are so revolutionary.
When the waiter drops the plate, the AI doesn't
have to relearn what gravity is or figure out
if the waiter spontaneously exploded. Because
the restaurant frame is loaded with default expectations
about physics and human behavior, it can constrain
the possibilities. The AI understands that the
waiter slot is still filled by a human, and the
plate slot just changed from intact to broken
due to gravity. Wow. And by the early 1980s,
communities realized they could combine the real
-world representation of frames with the complex
logic of rule -based systems. Merging them together.
Yeah. In 1983, a system called the Knowledge
Engineering Environment, or KEY, did exactly
that. It had a complete rule engine to handle
logic, but also a frame -based knowledge base
to handle the messy reality of categories and
objects. But even with all that power, they kept
slamming into the common sense reasoning problem.
The source mentions the frame problem, which
is just mind -boggling when you think about it.
It really is. How do you teach a computer that
an object maintains its exact position from one
moment to the next unless it is moved by an external
force? Right, this raises an important question.
How do you explicitly code the infinite unwritten
rules of human existence? It seems impossible.
As humans, we regularly draw on an extensive,
invisible foundation of knowledge about the real
world. We understand basic physics. We know that
if you put a cup on a table, it stays there.
We know that water flows downhill. We understand
causality and human intentions. We take it entirely
for granted because we've been interacting with
the physical world since we were babies. But
to a computer... To an artificial agent, a digital
surrogate that has never felt gravity or dropped
a cup... None of that is obvious. It all has
to be explicitly stated. And someone actually
tried to write it all down. Doug Lean it with
The Psych Project. The Psych Project spelled
C -Y -C. It was one of the most ambitious programs
in AI history. They created their own language.
cycle and hired armies of analysts to manually
document various areas of common sense reasoning.
Just taping out common sense. Literally. Yeah.
They tried to manually type out models of time,
causality, physics, intentions. They were writing
rules like you cannot be in two places at the
same time or once you consume a liquid, it is
no longer in the glass. That is why. It was a
monumental, seemingly endless task to type out
reality. But here's my logical next step on this.
If one guy and his team spent decades just trying
to type out the basic physical rules of the world,
how do different systems or even different human
researchers ever agree on what the rules actually
are? Like if I build a database for my company
and you build one for yours, how do we ever share
this knowledge? That exact problem gave rise
to the discipline of ontology engineering. As
knowledge bases scaled up, researchers needed
them to be modular. They needed different systems
to communicate with one another. But defining
the boundaries of concepts is incredibly subjective.
The source includes a quote from Tom Gruber that
summarizes this beautifully. He says, every ontology
is a treaty, a social agreement among people
with common motive and sharing. He argues that
a truly general purpose universal ontology is
impossible. So what does this all mean? I have
a hard time wrapping my head around that. If
every ontology is just a treaty or a social agreement,
does that mean there is no objective truth in
computer logic? Let me give you an example. Is
a tomato a fruit or a vegetable? If I'm building
a system for a botanist, the tomato is objectively
a fruit. Right. If I'm building it for a chef,
it's a vegetable. Are we literally just teaching
computers our subjective human perspectives?
If we connect this to the bigger picture, mathematical
logic, the ones and zeros, the inference engine
has objective truth. But categorizing the world,
deciding how to represent reality within the
knowledge base, that requires perspective and
bias. Interesting. The source gives two stark
examples of how the exact same objective reality
can be represented in completely different ways,
depending on what the user needs the system to
do. Oh, the electronic circuits example really
highlighted this for me. Yeah. If you were representing
an electronic circuit, you might use what's called
the lumped element model. In this view, you see
the circuit purely in terms of components and
connections, and you assume electrical signals
flow instantaneously between them. It's a simple,
highly useful treaty for most basic electronics.
But what if you need more detail? Right. If you
need to pay attention to electrodynamics, maybe
you're designing a high -speed computer processor,
that ontology is totally useless. In the electrodynamic
view, signals propagate at a finite speed. A
simple resistor is no longer just a component.
It's an extended medium that an electromagnetic
wave flows through. But the physical reality
of the circuit on the table hasn't changed at
all. Exactly. Copper wire is still copper wire.
But the representation, the treaty we agreed
upon to make the math work has completely shifted.
And the exact same dynamic played out in early
medicine. We talked about expert systems earlier.
A famous early system called amycin viewed medicine
as a set of empirical rules. OK. It created simple
associations connecting a specific symptom to
a specific disease. But another system called
internist view the exact same medical field as
a set of prototypes. Wait, how is that different?
It tried to match the patient's entire case against
prototypical diseases rather than just symptom
to disease rules. There are two totally different
ways to encode the same objective reality of
human biology. And this isn't just ancient history
for massive room size mainframes. This concept
of ontologies and treaties is literally the architecture
of the modern Internet. Yes. The source talks
about the semantic web, and this is where it
really connects to your daily life as a listener.
The semantic web is the ongoing attempt to add
a layer of actual meaning semantics on top of
the current internet. Because right now it's
mostly just text matching, right? Exactly. Traditional
search engines primarily index websites by matching
text keywords. If you type the word Apple, the
traditional search engine looks for web pages
that contain the letters A -P -P -L -E. Just
the string of characters. Right. But the semantic
web uses large ontologies of concepts. It relies
on markup languages like RDF, which stands for
Resource Description Framework. And what does
RDF actually do? RDF creates a structure of subject,
predicate, and object. It tells the computer,
Apple is a fruit. or Apple is a technology company.
Oh, wow. So when I search Apple, the computer
isn't just looking for letters anymore. It looks
at the ontology and effectively asks, wait, is
the user's current frame technology or fruit?
It uses logic to deduce the context. The semantic
web... also uses OWL, the Web Ontology Language,
to add complex logic to those relationships.
And it uses automated reasoning engines, called
classifiers, to constantly update the Internet's
ontology on the fly as new information gets added.
So the internet is slowly moving from a giant
filing cabinet of text strings into a structured
web that actually understands the relationships
between concepts. Exactly. But that entire web
of meaning, all those classifications, are governed
by those human treaties we talked about. The
computer only knows a tomato is a fruit if the
treaty we wrote says so. It's all built on the
perspectives we choose to code into it. That
is just incredible. To summarize our journey
today for you listening, we started with the
massive dreams of the 1950s, trying to build
general problem solvers that could figure out
anything. Then we saw the practical computational
trade -offs that forced the field to pivot to
expert systems, trading broad logic for deep
specialized domain knowledge. We explored the
quirky, incredibly complex world of teaching
computers common sense using frames so they know
how to navigate something as simple as a restaurant
without breaking down. Exactly. And finally,
we arrived at ontology engineering, realizing
that the very fabric of the semantic web is bound
by digital treaties, social agreements on how
we choose to represent our world to the machines
we build. Which leaves us with a really profound
implication to consider. If every knowledge representation
is fundamentally a treaty, a specific lens chosen
to view the world, like viewing a circuit as
simple connections rather than complex waves,
what happens to the human knowledge that doesn't
fit neatly into these digital treaties? Oh, that's
a heavy thought. As our machines become the primary
way we store, retrieve, and interact with information,
Are we slowly reshaping human thought to only
fit the structures that computers can easily
process? That is a brilliant and slightly terrifying
question for you to mull over as you go about
your day. Thank you so much for joining us on
this deep dive into the architecture of artificial
thought. Keep questioning the digital world around
you.