Welcome to The Debate. Today we're dissecting
the machinery of the modern mind, or maybe a
better way to put it is the digital mirror we've
built to reflect it. We're talking about the
neural network. Right. And it's a term that has
become, you know, completely synonymous with
artificial intelligence, yet it carries this
biological passport. It suggests that we've successfully
reverse -engineered the human brain. But when
you peel back the layers of code, what are you
actually looking at? a digital cortex, or just
advanced statistics wearing a biological costume.
And that is the central tension we're navigating
today. The neural network is the architecture
underlying everything from, well, chat GPT to
self -driving cars. But its origins, and I would
argue its fundamental nature, are rooted in neuroscience.
The question is, have we built a model that functions
like a brain? Or have we simply borrowed the
terminology to brand a mathematical calculator?
I'm taking the position that the neural and neural
network is really a relic of history, not a description
of function. We're dealing with mathematical
models designed to approximate nonlinear functions.
These are tools of engineering, not biology.
And I'm here to argue that the biological comparison
is not just a metaphor. It is the blueprint.
From the architecture of interconnected units
to the very way they learn from error, artificial
neural networks are the first technology to successfully
implement the theories of the mind proposed by
19th century psychologists. We haven't just built
a calculator. We have digitized the fundamental
principles of thought. That is a bold claim.
Let's verify it. To understand my stance, you
have to look at where this concept came from.
We tend to think of AI as a product of the Silicon
Age, but the theoretical framework actually predates
the computer. We're going back to 1873, to Alexander
Bain and later William James in 1890. The fathers
of modern psychology. Exactly. And what was their
radical proposition? They suggested that thought
or consciousness wasn't some singular, indivisible
thing. They proposed that it was an emergent
property, something that arises from the interactions
among a massive number of neurons. This is the
definition of a neural network, a group of interconnected
units where the intelligence isn't in the unit,
but in the connection. I don't dispute the historical
lineage. I mean, Bain and James were brilliant
in hypothesizing how the biological brain might
work. And yes, in 1943... Warren McCulloch and
Walter Pitts used those ideas to create the first
computational model of a neural network. They
were connectionists. They wanted to mimic the
brain. And they succeeded. They demonstrated
that simple units acting together could perform
complex logic. But intention is not the same
as execution. Just because McCulloch and Pitts
wanted to build a brain doesn't mean the modern
result is a brain. If you look at the definition
of an artificial neural network in machine learning
today, the biology is notably absent. It's defined
as a mathematical model used to approximate nonlinear
functions. Let's pause on that term, nonlinear
functions, because I think it scares people off,
but it's actually the strongest argument for
the biological connection. How so? A linear world
is simple. If I push a box twice as hard, it
moves twice as far. That's linear. But the real
world, and biological decision -making, is non
-linear. You can tap a person on the shoulder
and get no reaction. Tap them slightly harder,
they turn around. Tap them a tiny bit harder
than that, and they might punch you. The output
isn't directly proportional to the input. The
brain is a non -linear processor. Artificial
neural networks are the only mathematical tools
that handle this non -linearity the same way
biology does, through thresholds. I think you're
conflating the what with the how. Yes, both systems
handle complex, non -linear tasks. But the mechanism
matters. In engineering, we say form follows
function. But here, the form has diverged radically.
The source material highlights a critical shift.
Early models, like the Perceptron, built by Frank
Rosenblatt in 1957, were physical hardware. They
were machines you could touch. with wires acting
like axons. And today, there's software. Right.
And that transition to software is where the
model becomes an abstraction. When you move to
software, you're no longer constrained by the
physics of the brain. You're optimizing for math.
Modern networks are used for specific utility.
Predictive modeling, facial recognition, handwriting
recognition. They're statistical engines. To
claim they're still biological models is like
saying a submarine is a model of a fish. Yes,
they both swim, but one uses a propeller and
the other uses a tail. The mechanics are fundamentally
different. I love the submarine analogy, but
I think it fails because a submarine doesn't
try to replicate the muscle structure of a fish.
A neural network does replicate the information
flow of a brain. Let's look at the actual transmission
of data. This is the mechanics of transmission.
Okay, let's get into the weeds. How do you see
them as identical? In a biological brain, you
have neurons. A single neuron receives signals
from its neighbors through dendrites. If the
total signal is strong enough, the neuron fires,
sending a signal down the axon to the next neuron.
Now look at the artificial version. You have
nodes, or artificial neurons. They're arranged
in layers. Input, hidden, and output. Information
comes in, it's processed, and it's passed forward.
The hidden layers are doing exactly what the
mass of gray matter in your skull does, processing
information in stages. Processing is doing a
lot of heavy lifting in that sentence. Let's
look at what is actually happening at that node.
In biology, you're talking about electrochemistry.
A neuron receives excitatory or inhibitory signals,
chemicals like glutamate or GABA. These open
ion channels. The cell membrane voltage changes.
If it hits a threshold, boom, an action potential.
It's a spike. It's a dynamic, temporal, physical
event. And the math simulates that. The math
replaces that with something totally different.
The text is very clear on this. The artificial
neuron calculates a linear combination of the
outputs of the connected neurons. Which is a
summation. It's a specific kind of summation.
It takes a value x, multiplies it by a weight
w, adds a bias b. It's algebra. y equals wx plus
b. Then, to your point about non -linearity,
it runs that number through an activation function,
like a sigmoid or remove function. This squeezes
the number to determine the output. You're describing
the exact mathematical translation of the biological
process. The weight is the strength of the synapse.
The summation is the accumulation of neurotransmitters.
The activation function is the firing threshold.
Just because we write it in Greek letters instead
of squishy tissue doesn't mean the logic is different.
It is a high fidelity abstraction. I strongly
disagree that it's high fidelity. It's a caricature.
In the brain, timing matters. The rate of firing
matters. The chemical soup matters. In the artificial
network, it's just a static number passing through
a static function. You're stripping away all
the chaos and complexity of biology to get a
clean mathematical equation. That's not a model
of a brain. That's a spreadsheet that looks like
a brain. But does the complexity of the substrate
matter if the emergent behavior is the same?
If I build a heart out of titanium and plastic
and it pumps blood, it's a heart. If I build
a network out of code and math and it recognizes
a face, it's a neural network. But does it recognize
the face the way a human does? Or does it just
find a statistical correlation of pixels? This
brings us to the most contentious point, learning.
How does the system actually improve? This is
my strongest evidence. We have to talk about
Donald Hebb. Hebbian Learning, 1949. Right. Cells
that fire together, wire together. This is the
foundational rule of biological learning. If
neuron A repeatedly helps fire neuron B, the
connection between them gets stronger. The synapse
physically changes. This is exactly, exactly
how artificial networks learn. Explain that connection.
In an artificial network, the behavior is determined
by the weights, those numbers we multiply the
inputs by. When we train the network, we are
simply adjusting those weights. We're strengthening
the connections that lead to the right answer
and weakening the ones that lead to the wrong
answer. That is heavy in theory and pure distilled
code. It sounds like heavy in theory, but the
mechanism for how those weights change is radically
different. And the text points this out explicitly.
Artificial networks are trained using empirical
risk minimization and backpropagation. Now those
are just the algorithms we use to adjust the
weights? Just the algorithms. is a massive understatement.
Backpropagation is the engine of modern AI, and
it has no biological equivalent. Think about
how it works. The network makes a guess. It calculates
the error, the difference between its guess and
the right answer. Then it uses calculus to go
backwards through the network from output to
input, calculating the gradient of the error
and adjusting every single weight to minimize
that error. It's an optimization technique. It's
a way to find the right weights faster. It's
a cheat code that biology doesn't have. Your
brain cannot freeze time, calculate the mathematical
error of a thought, and then send a correction
signal backwards through your axons to adjust
the synapses. Biological learning is local. It
happens at the synapse, in real time, based on
local signals. Artificial learning is global
optimization based on a static dataset. I think
you're getting hung up on the implementation
details again. Backpropagation is simply the
most efficient mathematical way to achieve the
state of learning. The end result is what matters,
a network where the connection strengths encode
knowledge. The process dictates the capabilities.
Because we use backpropagation, we need labeled
data. We need a pre -existing dataset. The text
says we train these networks to fit. a data set.
We show it 10 ,000 pictures of cats and say,
minimize the error in identifying these. That
is statistical regression. That is curve fitting.
But humans learn from data sets too. We call
it experience. But we don't need 10 ,000 labeled
examples to know what a cat is. We learn adaptively,
physically, continuously. We don't perform empirical
risk minimization on a static batch of data.
We survive in an environment. The text makes
this distinction clear. Biological networks are
large -scale brain networks integrated into a
nervous system that drives muscle cells in motion.
Artificial networks are isolated software loops
minimizing a loss function. You're acting as
if these networks are stuck doing simple regression.
We need to talk about the evolution of purpose.
We aren't just building simple perceptrons anymore.
We are building deep neural networks. Deep just
means more than three layers. usually two or
more hidden layers. But that depth creates emergence.
The text mentions generative AI and general game
playing. When you add those hidden layers, the
network starts doing things that look less like
statistics and more like cognition. It creates
art. It writes poetry. It plays Go better than
any human. It's impressive, I grant you. It connects
back to a discovery by Sminyat Chakan in 1956
regarding retinal cells. He found that you couldn't
understand the function of the eye by looking
at a single cell. You had to understand the network
of horizontal cells. The interaction created
the capability. That is what deep learning is
doing. By stacking these layers, we're mimicking
the deep hierarchical structure of the cortex.
We're moving away from curve fitting toward feature
extraction and representation. The constraints.
Even with deep learning, The text notes the divergence
from biology. These systems are often brittle.
They can be fooled by noise that wouldn't fool
a human. They require massive energy to train.
They are approximating nonlinear functions, just
extremely complex ones. I think you're underestimating
the general in general game playing. A system
that can learn the rules of chess, then shogi,
then go, without being reprogrammed, that is
approaching generalized intelligence. That isn't
just a calculator. That is a malleable learning
substrate. just like the brain. But it learns
those games to maximize a score. It's still an
optimization problem. It doesn't know it's playing
a game. It doesn't have agency. The biological
network is designed for survival. It's chemically
connected to a body. The artificial network is
designed for accuracy on a test set. What about
self -driving cars? That is adaptive control,
which the text mentions. That is a network navigating
the physical world, making life -or -death decisions
in real time. That is the closest parallel, I
admit. But even there, the car is seeing numbers,
not the road. It's calculating probabilities
based on LIDAR point clouds. It's a simulation
of perception. You keep using the word simulation
as a pejorative, but all models are simulations.
My argument, and the argument of the biological
roots perspective, is that we have found the
fundamental algorithm of intelligence. It just
so happens that you can run that algorithm on
wet biological tissue or on silicon chips. The
weight is the universal unit of memory. The layer
is the universal unit of processing. And my argument
is that you've found a mathematical trick that
produces results analogous to intelligence, but
through a fundamentally different route. The
linear combination is not an action potential.
Back propagation is not heavy in plasticity.
And empirical risk minimization is not survival.
But we're getting closer. Every year, the networks
get deeper. The architectures get more complex.
We're adding attention mechanisms which function
like human focus. And yet, the text reminds us
that even as they get more complex, they become
increasingly different from their biological
counterparts. To solve the engineering problems
of AI, we've had to abandon the biological constraints.
we stopped trying to build a brain and started
trying to build a machine that works. Maybe that's
the ultimate irony. To build a machine that thinks
like a human, we had to stop copying the human
anatomy and start focusing on the human mathematics.
Or maybe we're just seeing what we want to see.
We look into the black box of a neural network
and we see a reflection of our own minds, when
really it's just a very shiny mirror made of
calculus. That is the question we leave on the
table. We've traced the arc from Bain and James'
theories of the 1870s, to McCulloch and Pitts'
circuits of the 1940s, to the massive deep learning
models of today. A journey from biology to math,
and perhaps back again. We encourage you to look
at the source material and decide for yourself,
is the deep neural network a sibling to the human
mind, or is it simply the world's most impressive
approximation machine? Thank you for joining
The Debate. See you next time.