You know, when you think about cutting edge artificial
intelligence today, there's this expectation
of these sleek, sterile server farms. Oh, absolutely.
Just glowing racks of silicon everywhere. Right.
Massive cooling fans, endless streams of code,
just something completely divorced from the messy,
you know, organic world of biology. Yeah, it
definitely feels like a purely mathematical achievement.
like something born entirely inside a machine,
completely disconnected from the natural world.
But then you trace the family tree of the very
technology that gave machines the ability to
see. And suddenly, you aren't in a server farm
at all. No, not at all. You're actually in a
biology lab. Exactly. A biology lab in the 1950s
looking at the brain of a cat. It really is the
ultimate technological plot twist. I mean, the
blueprint for modern computer vision was actually
drawn by Mother Nature long before anyone ever
typed a line of code. Welcome to the Deep Dive.
Today, we are taking a stack of research on convolutional
neural networks, or CNNs, and figuring out how
a biological discovery in the 50s became the
technology that lets your phone recognize your
face. And powers self -driving cars. Yeah and
even discovers new medicines. Our mission here
is to demystify how we actually taught machines
to see and to do that we have to look at how
nature taught us to see. Which is fascinating
all on its own. Okay let's unpack this because
the idea of computer vision can feel totally
overwhelming but human vision is actually built
on a surprisingly simple foundational trick.
Think of it like looking at a massive complex
mosaic on a wall. A mosaic is a really great
way to frame it. Right, because when you stand
right up close to a mosaic, you don't process
the entire wall at once. Your eye catches a single
dark blue square. Just one tiny piece. Yeah,
then maybe you notice a light blue square right
next to it. You're just looking at tiny, localized
patches of color. And they don't mean anything
on their own. Exactly. But as you step back,
your brain starts combining all those individual
meaningless tiles. The dark blue and light blue
merge into a pattern and eventually you realize
you were looking at a giant picture of an ocean.
You build the whole from tiny overlapping pieces.
And that maps perfectly onto the biological research
that started all of this. Back in the 1950s and
60s, there were these two researchers named Hubel
and Riesel. Okay. And they were studying the
visual cortices of cats. They were trying to
figure out exactly how the brain translates light
into shapes. So what did they do? Well, they
showed these cats various shapes and lines on
a screen, and they literally measured the electrical
spikes in specific neurons in the cat's brains.
Yeah, and what they discovered fundamentally
changed our understanding of sight. They found
that individual neurons in the brain, they don't
process the entire visual field at once. Like
staring too close to the mosaic. Exactly. Instead,
they respond to stimuli only in very restricted
tiny regions. They called these regions receptive
fields. So a single neuron is only responsible
for its own little patch of reality. Right. It's
completely blind to what's happening on the other
side of the visual field. Precisely. And neighboring
cells have overla - receptive fields, so together
they map out the whole visual space, just like
your mosaic tiles overlapping to create the ocean.
That makes a lot of sense. Hubel and Weisel even
identified two specific types of visual cells
operating in this system. First, they found what
they called simple cells. Okay, what do simple
cells do? These cells get really excited, like
they maximize their electrical output only when
they see a straight edge or a line at a very
specific angle within their little patch. So
they are very picky. Incredibly picky. If the
line is tilted even slightly wrong, the cell
stays totally quiet. Then they found complex
cells. And I'm guessing they are less picky?
Yeah. They take the information from the simple
cells, but they are a bit more flexible. They
don't care exactly where that edge is located
within the patch, just that the edge exists somewhere
in their neighborhood. Oh, I see. So the simple
cells are the strict rule followers looking for
exact angles, and the complex cells are just
looking at the general vibe of the area. That's
a really good way to put it. And here is where
the jump to technology happens. In 1980, a computer
scientist named Kunihiko Fukushima took this
exact biological discovery and translated it
into a mathematical model. In 1980. Yeah, 1980.
He introduced something called the neocognitron.
Neocognitron. Sounds like a sci -fi villain.
It really does, but what he did was directly
mimic. those cat brains by creating two alternating
layers in his computer model. He made an S layer,
which acted like the simple cells, looking at
specific receptive fields and identifying basic
patterns. Exactly. And he made a C layer acting
like the complex cells, which consolidated the
information and helped classify objects, even
if they shifted position slightly. So this neocognitron
is essentially the origin of the whole CNN architecture
we use today. It is the foundational blueprint.
Yes, it is wild. that the blueprint was mapped
out in 1980. But the source material notes that
when early computer scientists actually tried
to build standard neural networks to process
images using these ideas, they immediately hit
a massive mathematical brick wall. A huge wall.
They ran into something called the curse of dimensionality.
Right. And looking at the numbers in the research,
I see exactly why. Let's do the math on a simple
100 by 100 pixel image. To us, that is incredibly
tiny. Yeah, it's just a low resolution thumbnail.
But for a computer, a 100 by 100 grid is 10 ,000
individual pixels. Which doesn't sound too bad
until you look at how a traditional neural network
is wired. Right. In a standard fully connected
network, every single artificial neuron in one
layer has to connect to every single neuron in
the next layer. OK, so if you have those 10 ,000
pixels in the first layer, And let's say your
next hidden layer has a thousand neurons. Every
single one of those 1 ,000 neurons needs a dedicated
connection to all 10 ,000 pixels. Do the math
on that. That is 10 million individual weights,
or connections, that the computer has to calculate.
For a tiny thumbnail. And if it is a color image
with red, green, and blue channels, you multiply
that by three, suddenly you have 30 million parameters
for a blurry little picture. Exactly. And if
you try to process a modern high -resolution
image, say from a smartphone camera, the math
explodes into the billions or even trillions
of connections. The computers of the era just
choked on the data. They completely choked. I
see. It's kind of like the difference between
a micromanager and a smart auditor. A traditional
neural network is the micromanager. I like that
analogy. It demands a direct individual line
of communication with every single employee in
a global company, asking for an update every
single second. Right, checking every single pixel
at all times. There are millions of emails flying
around, it's too much noise, and the micromanager
just collapses from exhaustion. But a convolutional
neural network, a CNN, is like a smart auditor.
Right. Instead of talking to everyone at once,
the auditor takes a little rubber stamp, let's
call it a filter, and walks down the hall, checking
small 5x5 teams one at a time. That little rubber
stamp is what we call a convolution filter or
a kernel. Okay. Instead of a single neuron needing
10 ,000 weights to look at the whole image at
once, a CNN uses a tiny 5x5 filter. That filter
only requires 25 weights. Wow, just 25? Just
25. And the secret sauce that makes this whole
thing work is that it uses shared weights. What
does that mean, shared weights? As that 5x5 filter
slides or convolves across the image, it uses
the exact same 25 parameters to check every single
patch of the picture. Oh, because if a pattern
like a vertical line or the curve of an ear is
important in the top left corner of an image,
it's probably just as important if it appears
in the bottom right corner. Exactly. The rule
doesn't need to change just because the location
changed. That is the core insight. This massive
reduction in parameters from 30 million down
to just a few dozen per filter does something
crucial for the learning process. It keeps the
computer from crashing. Well, yes, but also in
early, fully connected networks, scientists struggle
with the learning signal either dying out into
nothing or blowing up into infinity as the system
tried to adjust millions of weights at once.
So it was unstable. Highly unstable. By forcing
the network to share weights over fewer connections,
the CNN regularizes the math. It stabilizes the
whole learning process. That makes sense. The
filter slides across the input, calculates how
closely each patch matches its specific pattern,
and generates an abstracted map of where it found
that feature. We call that output a feature map.
OK, so the auditor has walked through the building
with the rubber stamp and created a feature map.
But according to the source, once the CNN has
scanned the image with its filter, it suddenly
has way too much data again. It does. It generates
a lot of maps. Right. And it needs a way to separate
the signal from the noise. And this is where
I really had to challenge the logic of what happens
next. Oh, what's the issue? The text talks about
a process called max pooling. It says, a standard
two by two max pooling layer actually discards
75 % of the data it just looked at. It does do
that. But if we are throwing away three quarters
of the information we just worked so hard to
extract, how does the AI still have any idea
what it's looking at? This raises an important
question because it seems entirely counterintuitive
to throw away data. Totally counterintuitive.
But let's look at the mechanics of how max pooling,
which was introduced by a researcher named Yamaguchi
in 1990, actually works. OK. Walk me through
it. Imagine your feature map is divided into
a grid of two by two squares. That's four pixels
per square. Got it. Max pooling looks at those
four numbers, takes the single highest value
of the maximum, and literally deletes the other
three. Just deletes them. It seems so reckless
to just dump 75 % of the resolution. But alongside
max pooling, the network usually applies something
called a ReLU layer. That's a rectified linear
unit, first used by Fukushima back in 69, actually.
OK, what does ReLU do? The ReLU function acts
as a strict bouncer at the door of the next layer.
It looks at the data and says, if your mathematical
value is negative, you are out. You become zero.
Just flat out zero. Yep. It only lets positive
values through. This removes negative noise,
adds necessary mathematical complexity without
bogging down the system, and massively speeds
up training. So you have real you cleaning up
the noise and max pooling shrinking the image
down. OK, I understand the cleanup part. but
I'm still stuck on throwing away the location
data with MaxPooling. To understand why it works,
we have to talk about something called Translation
Equivariance. Translation Equivariance? Which
is just a fancy way of saying flexibility in
position. Okay. Intuitively, the exact pixel
-perfect location of a feature is far less important
than its rough location relative to other features.
Like, it matters more that an eye is above a
nose. rather than being at pixel coordinate 45
by 60. Yes. Think about your phone's facial recognition.
When you wake up in the morning and hold your
phone slightly off center or your head is tilted
on a pillow, the image your camera sees is completely
different on a pixel by pixel level than the
one you originally scanned. Ah, I get it. The
phone doesn't care that my left eye moved down
three pixels on the camera's grid. No, it doesn't.
It just cares that the eye pattern is still roughly
above the nose pattern and next to the other
eye pattern. Exactly. If the neural network memorized
the exact pixel coordinate of everything, it
would be useless in the real world because real
world objects move. Right. By taking only the
maximum value in that two by two cluster, max
pooling essentially says... I found a strong
signal for an edge in this general area. I don't
care exactly which of the four pixels it was
in. It's focusing on the big picture. It grants
the network that flexibility, making it robust
against variations in position. And as a bonus,
it drastically reduces the memory footprint the
computer has to handle. I am looking at the timeline
of all this, and it is driving me crazy. How
so? We have the biological blueprint in the 50s
and 60s. Fukushima builds the neocognitron in
1980. The math for training these networks is
figured out in the late 80s. Yamaguchi brings
in Max Pooley in 1990. It was a long buildup.
Here's where it gets really interesting. The
text even points out this wild fact. A researcher
named Jan Lacoon developed a CNN called Lynette
5 that was successfully reading millions of handwritten
bank checks a day for the NCR corporation back
in 1996. Yeah, Linnet 5, it was incredibly successful
for that specific constrained task. Right, so
if they had the math, the layers, the pooling,
and commercial viability in 1996, why did we
wait until the 2010s for the massive world -changing
AI boom we are living in now? It's a great question.
The math was clearly there. The hardware must
have been failing them. That is exactly the bottleneck.
The Mac required to run hundreds of convolution
filters across thousands of images is extraordinarily
computationally intense. Because even with the
shortcuts, it's still a lot of math. Tons of
it. In the 90s, we were trying to run these networks
on standard central processing units, or CPUs.
Think of a CPU like a brilliant mathematics professor.
The professor can solve incredibly complex problems,
but they do them sequentially, one after another.
They read a problem, solve it, and move to the
next. So they are smart. but slow if there's
a huge stack of simple problems. Right. That
is great for running a spreadsheet or an operating
system, but it is terrible for the massive simultaneous
calculations required by a deep CNN. They were
trying to dig the Panama Canal with a collection
of really nice teaspoons. Exactly. It doesn't
matter how good the teaspoon is, the sheer volume
of dirt is the problem. So what changed? The
breakthrough didn't come from AI researchers
changing the math. It came entirely by accident
from the video game industry. Video games. In
the mid -2000s, video games were getting highly
realistic, requiring millions of pixels and polygons
to be rendered on screen at the exact same time.
Oh, to draw all the 3D graphics. Yes. To do this,
companies built graphics processing units, or
GPUs. If a CPU is a brilliant professor doing
one problem at a time, a GPU is an army of 10
,000 middle schoolers doing basic arithmetic
all at the exact same time. Parallel processing.
Instead of sliding one silter across an image
step by step, a GPU can calculate the entire
feature map in an instant. And researchers started
to realize this was the key. In 2011, a team
at an institute called IDSIA managed to accelerate
CNN training by an astonishing 60 times compared
to a CPU by using GPUs. 60 times faster. That
is the difference between an experiment taking
two months to run and taking one single day.
It's a game changer. That fundamentally changes
the entire iteration cycle of science. You can
test a hypothesis, fail, tweak it, and try again
before the weekend. It changed everything. And
the catalytic event, the moment the AI boom officially
started, was the 2012 ImageNet challenge. What
was ImageNet? It was a massive competition to
see which software could categorize millions
of images best. A researcher named Alex Krzyzewski
and his team unleashed a GPU -based CNN called
AlexNet. And how did it do? It used all the building
blocks we just talked about, convolutional layers,
max pooling, ReLU activations, and GPUs, and
it absolutely crushed the competition. It wasn't
even close. Wow. It was an earth -shattering
moment for computer science. We've spent a lot
of time establishing how CNNs conquered images
and visual data. But one of the most surprising
revelations in this source material is that once
researchers realized how good CNNs were at finding
spatial relationships, they started using them
for things that aren't visual. because when you
get right down to it, a convolutional neural
network doesn't actually know what an image or
a photograph is. It just sees numbers. Right,
it just sees a grid of numbers and looks for
local patterns between those numbers. So if you
could figure out a way to turn something, anything
in the real world into a grid, the CNN can look
at it just like it looks at a picture of a cat.
It's like translating reality into a spatial
map. That realization opened the floodgates.
If we connect this to the bigger picture, think
about a go board. The board gate. Yeah. It is
literally a two -dimensional grid of black and
white stones. Researchers fed the board states
of millions of games into a CNN, allowing the
network to find spatial patterns in winning strategies.
Oh, that makes total sense. That was fundamental
to AlphaGo, the system that famously beat the
world's best human player at a game once thought
too complex for machines. But the text mentions
drug discovery, too. How do you turn a disease
into a grid? Well, a chemist looked at this technology
and asked, wait, isn't a molecule just a three
-dimensional grid of atoms? Carbon is at this
coordinate. Oxygen is at that coordinate. In
2015, a system called AtomNet used a 3D CNN to
look at representations of chemical interactions.
So instead of a 2D picture, it's a 3D space.
Exactly. Just like a 2D CNN learns to compose
small visual edges into the shape of an ear or
an eye, AtomNet learned to discover the shapes
of chemical features like hydrogen bonding. That's
brilliant. By sliding 3D filters through the
molecular grid, it was used to predict novel
treatments for the Ebola virus and multiple sclerosis.
That is incredible. And what about time? The
text mentions forecasting financial markets and
weather patterns. How does a CNN look at time?
Time is just a one -dimensional grid. I never
thought of it like that. It's a line of data
points moving forward. Researchers found that
if you treat time dependencies like visual patterns
using 1D convolutions sliding along a timeline,
CNNs can actually outperform traditional forecasting
networks. Because they just look for shapes in
the data. Yes, they spot local patterns in stock
dips or temperature changes, they are computationally
more efficient, and they don't suffer from the
learning signals dying out during training. So
what does this all mean if we step back and look
at the journey we've taken today? We started
by peering into the visual cortex of a cat in
the 1950s, watching biological cells react to
straight lines and overlapping fields of vision.
A long way from where we are now. We watched
that biological mosaic get translated into mathematical
layers. We saw how shared weights and sliding
filters solve the curse of dimensionality, stopping
the computers from collapsing under millions
of connections. And we threw away the exact coordinates
with max pooling. Right, which gave the system
the flexibility to recognize a face even even
if it moved, focusing on the big picture. We
traced how this math laid dormant for decades,
quietly reading bank checks, until the video
game industry accidentally built the perfect
parallel processing engine to unleash it. Those
GPUs really saved the day. And now, we are using
that exact same architecture to design life -saving
drugs and forecast financial markets simply by
turning reality into a grid. It is a remarkable
synthesis of biology, mathematics, and hardware
engineering. At its core, you have to remember
that a CNN is a system designed to discover what
is truly important in a massive sea of data.
In the early days of AI, humans tried to hand
code every rule. If you see two circles in a
triangle, it is a face. But with CNNs, we don't
hand engineer the rules or the filters anymore.
The machine does it. We just set up the architecture,
feed it the grid of data, and the network learns
how to see the patterns all on its own without
human bias. But as much as we talk about how
CNNs replicate human vision, I want to leave
you with a fascinating contradiction pulled right
from the source material. Let's hear it. We've
built these incredibly powerful networks, and
they're capable of instantly classifying incredibly
fine -grained categories that totally confuse
humans. can distinguish between two nearly identical
breeds of dogs in a fraction of a second, spotting
spatial relationships in the fur patterns that
our eyes completely glaze over. They're completely
superhuman in that specific regard. Superhuman.
Yet the text points out that these exact same
advanced algorithms can easily get confused by
things that make no sense to us. Like what? A
CNN might confidently classify a school bus as
a completely different object. just because a
tiny ant is resting on the camera lens. Just
a few pixels out of place. Exactly. Or gets completely
thrown off by an image that has been slightly
distorted with a simple digital camera filter.
A human toddler wouldn't be fooled for a single
second by a grainy photo of a bus. But the AI
is baffled. It's a totally different way of processing
the world. Which leaves us with a profound question
to ponder long after you finish listening today.
What does it actually mean for our future? that
the machines we specifically built to replicate
human vision are rapidly developing an entirely
alien way of seeing our world. Keep diving deep.