This is the Convergent Science Network podcast. Leading researchers in the domain
of neuroscience, brain theory and technology are interviewed by Paul Verschure and Tony Prescott.
Paul Verschure with the Convergent Science Network podcast.
I'm here with my colleague Tony
Prescott at our Barcelona Cognition Brain Technology Summer School 2018.
Where our guest Bjorn Merker, welcome to the podcast, Bjorn.
Has given a tutorial on the different systems of the brain and how to organize,
especially from an anatomical perspective, right?
And then also what possibly the functional consequences of this organization would be.
So Bjorn, how many systems would you distinguish in the mammalian brain?
How many subsystems would you consider as being relevant?
Well, what I did in the tutorial was try to, first of all, raise some problems
about what we mean by systems and how to define them.
Because anatomically, I made one example. For example, take midbrain and diencephalon.
In our anatomical nomenclature, we divide them up into two clear compartments,
diencephalon and midbrain. brain.
But if you look, I showed horizontal sections, and it's an absolutely uniform
griseum, you know, gray matter with various nuclei, and you can't see any distinction between them.
And then you look at embryology and molecular markers for developmental processes,
and you see that midbrain and diencephalon in its origin is one unit,
and it's the unit that receives terminals from the optic tract,
the midbrain does, and so I call it the optic brain.
And so that was to try to make a little problem out of having to find a system.
And anatomically, when I was looking at brains myself, you took the posterior
commissure, was the dividing line.
Behind that is the midbrain, in front of that.
But functionally and in terms of circuitry, you can't see a border between them.
So I tried to make a little problem out of defining systems.
And then I said, well, we can use an approach from, for example...
Systems theory or cellular analysis of cell function
there you have clear division of labor between the different
organelles they're each doing a specific thing the
mitochondria are doing metabolism and the endoplasmic reticulus reticulum is
doing is doing dna work and goldie apparatus and so each has its function and
they all work together and when they stop working together the cell is dead, the system is gone.
So if you look at this in terms of the division of labor, what would you like
the parts, the system parts of the total system, what would you like them to do?
Well, you would like them to have rather clear-cut generic functions.
And if you think about that, you would also expect that if there is a system
and it's anatomically visible, it would have a pretty redundant structure.
And the classical example of course is cerebellum which
has this crystalline structure redundant throughout and you
know you would be have to be pretty much out of your mind not to think that
wherever in the cerebellum you are that circuit is doing something similar across
the whole cerebellum so a generic function and reflected in in a in a clear-cut structure the cortex,
Neocortex is similarly very uniform in its structure.
The basal ganglia is also, but they are all very different in their structures.
So by that kind of reasoning, by showing lots of pictures of these things,
you can start saying, well, maybe there is a reason to divide this thing up
into the higher functions, into systems, division of labor between basal ganglia,
cerebellum, and neocortex, Or actually, cortex as a whole,
because the hippocampus, for example, is simply the top of the hierarchy,
in a sense, of the cortical hierarchies.
Then as you go down, midbrain, you have this clear-cut, very well-defined,
which I wouldn't hesitate to call the system, which is the colliculus.
You have the extensions of the basal ganglia down into the midbrain.
And then, of course, you have the brainstem, which is just this massive collection
of very, very specific circuitry having to do with all the basic functions,
from respiration and locomotion to vocal control and so on.
So once you're up in the higher reaches, I find it easy to give at least a conjectural notion of a system.
How many systems do you divide the brainstem up into? That's your free choice.
So I wouldn't hesitate to give an explicit number, but I would say that there
are very clearly defined systems.
And the challenge would be two challenges.
What is their generic function, each one of them? can we define a global function
for each one of them that is reflected in the circuitry and secondly how do
they work together and but we could also as a heuristic,
go to ethology or we could go to psychology and say well systems are defined around motivation,
attention perception yeah memory right would you would you find it a helpful
heuristic to Yes, absolutely. It would come in.
For the brainstem, you would obviously have to deal with motivational systems
built in at a very basic level,
survival-related on all kinds of dimensions, and psychology,
attention, volition, and so on, abstraction, cognition, surely.
And then now the question comes, how do you interface them with this other way,
this division of labor notion?
And much of what psychology looks at, of course, is consciously accessible information.
You can ask people in psychophysical experiments, or you can do behavioral experiments, and.
You extract some kind of functional conception from that,
and so you get a class of your attentional mechanisms play across
the different modalities so that's probably some system
that is doing that and so on uh now it would play it could conceivably cut across
several of these division of labor defining systems attention there is a theory
of attention that actually involves the basal ganglia isn't there,
Somebody wrote something about the base of ganglia and attention.
Now, normally we don't think of it that way, but where did I see that?
Anyway, absolutely, yeah.
So, this sort of overall systems decomposition, can we find some.
General principles that are even above the level of modules of subsystems.
For example, sort of the Jacksonian principle of layered control.
Would you agree with that? Is there you have redundancy, or maybe not redundancy,
but you have similar functionality at different levels of the brain,
but the differences? Very much appealing.
I saw one example this afternoon with Giovanni's presentation where the eye-bling
conditioning with cerebellar involvement, the eye-bling reflex,
which is the one without the cue, when just an earpuff goes to your eye, that one keeps working.
That one keeps working. And what you get is you move the beginning of that up.
So when you saw the detailed videos of the eye closing, eye-bling conditioning,
when when the animal was conditioned, you see this slower thing,
which is the anticipatory blinking, and then a real squeezing of the eye, which is the old reflex.
But here they are both two levels, two Jacksonian levels, if you please,
both working at the same time in parallel.
In that case, there is no reason whatsoever to eliminate the lower one.
Let it run. And that to me, by the way, brings up the very appealing notion
that sometimes we think of the systems working together that you have to switch
back and forth between them.
But the default procedure ought to be that every system does its generic thing
all the time in parallel.
And when no other system is sending it input, it idles.
And when it gets something significant, it does its thing on it and sends it
back wherever it's going to send it.
So that means that they are running in parallel all the time,
and that there would then be some behavioral situations in which you would have
to turn off actively the lower level because it interferes with something,
but those would be special cases.
The default would be multilevel, parallel, running, each one doing their own
thing, and like there is the old notion that visual function moves from the
colliculus up to cortex.
No, a new layer is added with completely new capacities, object vision,
and the colliculus keeps doing its old orienting thing and actually some salient things and so on.
So let it do its thing. Why?
It's perfectly home from billions of years of evolution.
Let it do its thing and just add new things. And then the new things have to
be tied into the old ones.
And they are usually tied in by some kind of fiber projection.
So absolutely, yeah, air levels thing, I have no problem.
But now we have a bit of a mess, right?
Because we start with anatomy, and it looks like, although you already said,
be careful with drawing borders, because it's not so obvious.
But we could think about, okay, but there are structural divisions, right?
I have a brain and brain is my midbrain, forebrain. but now if we take this
Jacksonian view together with this more ethological psychological functional perspective,
it starts to look like at each of these structural systems.
All these function elements are actually represented in some form but as I move
forward along the neorexis.
It is starting to become more dependent on let's say on memory it starts to
become more dependent on learning properties of the control of these different
psychological functions.
So how should we now square that circle? How do we get the structure and the function aligned?
Because the standard heuristic was always, well, function follows structure.
If you understand the structure, you have to head around the function.
But maybe it's not that straightforward.
In other words, your question is, at each of these levels, even at the most
primitive, is there an instantiation at that level of each of the fundamental things?
And then you replicate them, but they get more and more sophisticated and more
and more dependent on advanced kind of operations.
And naturally, in a lamprey, you know, you have, yeah, but yeah, that's,
Maybe we don't have to go that far because what Grillner is showing is that
lamprey in this, you know, compared to our brain or a mouse brain is quite primitive.
But it has a basal ganglia and it's divided up pretty much.
It sends its abendular for avoidance, projection and so on. And when he takes
a lamprey and legions the basal ganglia, they get tardive dyskinesia.
They suck on to rocks and they won't let go, like a Parkinson's patient who
can't get going. Now they get stuck.
So even a lamprey has, of course, a telencephalon. on.
So there is presumably never a stage at which only a brainstem works by itself.
The brainstem itself is a bit hierarchical.
And I think the big idea... Perhaps a developmental state. Yeah.
Something like the Monodelchus domesticus opossum, where the infants are born extremely juvenile,
essentially just of brain stem and what they can do at that age is just cling
to the mother and find and they would be very different from the beginning.
I think they're born around two weeks after gestation.
And you can watch the other parts of the brain begin to kick in.
And I think you can probably see some similar things in rats in development.
There is a mutant mouse where there's a...
A gene which controls dopamine and
if this this this gene
is absent then dopamine cells don't develop
properly and those mice are finally their first form but by by two weeks so
the dopamine system and the basal ganglia presumably which you know which has
a veto on any kind of voluntary movement if once that comes in if there's no dopamine it won't work.
To that they will be they can generate behavior deficits yeah
that stage but not their own yeah which means that
in a sense that system comes it comes in a bit late yeah
so maybe in a guinea pig which which
is born and runs around uh with incredible
competence because i was said to watch a guinea guinea pig birth the woman was
away and i had them in a cage and i didn't realize that the newborns could get
through the meshes of the cage within one hour of birth,
they were running around the place and hiding in places, and I had to chase
them down, and it was just as bad as chasing a full-grown rat.
So they would be very competent.
So there may be developmental schedules that keep these things in for precocity
versus altriciality. Yeah.
Interesting. And it's interesting in humans that we are precocial in terms of
our sensory development before with our eyes open, but our motor systems are really altricial.
So really altricial. What is interesting in your mouse example,
Tony, maybe that's a riddle also you're going to solve, if you sort of perturb
the neural mental trajectory, you get severe deficits, right?
But if you take an adult animal and you would lesion the forebrain.
Then often from the outside, they're pretty much functional, right?
If they live in a cage, they can take care of themselves, they can take care of their young, right?
So it's interesting to see that if you disrupt the developmental pathway,
the system apparently is really going towards some attractor that it cannot
reach and it collapses, it's pathological, dies.
Well, it wasn't reached that developmental stage, and then you just remove it
discreetly, it has to fall back on these earlier layers in its architecture.
So why is this bootstrap system to me so sensitive to perturbation as the adult
system is actually very robust to massive lesions? How would you explain that?
Well, I thought sometimes about the possibility that there are certain structures
whose essential function is to educate the system early on.
And once I have done their work the deficits will be less and that's approximately.
Rephrasing what you're saying now I have thought about whether the cerebellum
is like that that it's real,
use is very early of course later on we know that you have to recalibrate your BOR when your,
eyeballs grow and so on So there are always adjustments that are needed for
fine-tuning during adult,
so it has a function then also, but maybe it's real work that's being done early. Exactly.
So, okay, so we haven't really solved the problem yet, right?
Because we have a structure, we have a brain structure, there is heterogeneity, little uniform.
Form but we cannot really draw borders very clearly necessarily right it's not clearly modular.
But we cannot necessarily map it in some clean way to the to different functions
you might want to see either and the possibility might be that all functions
exist at multiple stages of the system but still when we when we progress along
anorexias and we go to the telencephalum the forebrain,
There are distinct features that you learn. Would it be fair to say that actually
the key thing that's happening is just we're building more advanced memory systems.
As we move forward, we build more advanced memory systems, and we have to add
features to just control these memory systems.
Would that be enough? I think that's absolutely right. Anyway,
I said in my tutorial that as soon as you see a structure in the brain that
has big volume with millions and millions and maybe billions of neurons,
you should think learning structure, memory structure, something that stores.
Because I have this heretical view that in every learning system there is this
tradeoff between stability and plasticity.
So, if you make the synaptic weight change, it's very rigid,
you're going to not be able to use those for anything else.
Well, that has been solved in neural network context by adding new units to the network. work.
And my alternative to that is you start out with a huge information storage
capacity and you use it up sequentially as you learn.
So there would be some kind of a learning front going through it, using up the tissue.
It would be equivalent to the ovaries stocked with all the eggs a woman is going
to produce in her entire lifetime, and they're successfully released.
Here you would start with your full learning capacity and use it up,
so it would sort of burn up as you learn and as you commit more and more synapses
in various ways to specific content.
When you see, so I have a chapter in my 2004 paper in Cortex,
a long introduction about the volumetrics of information storage.
And the best evidence is in bird vocal learning, where you have this very,
very clean correlation between the size of these nuclei and the learning capacity,
the complexity of the song that they will acquire. required.
And this goes within species for individual with different capacities of different
size, and it goes across species.
Species with very complex repertoires have big nuclei and a big system,
and the other ones have smaller.
So if you extend that to all learning, you would expect...
So if you extend that to all learning, you would say, looking at the brain,
just like that, you would say, where are the learning structures,
cerebellum, basal ganglia, neocortex? Those are the three voluminous, big chunks.
And the nice thing in evolution is that they go hand in hand.
They are sharply, the regression of volume between cerebellum,
neocortex, and basal ganglia should raise a sharp correlation.
A bigger cortex requires bigger basal ganglia, bigger cerebellum.
Well, the colliculus stays pretty much the same, you know?
So if you assume that they are storing information continually through a lifetime,
and for everything thing that you have to adjust to and using up their storage
group, but they have to start out big and sequentially use it up.
But in the story, you didn't include hippocampus, why is that? Dr. No, it's on the top.
I include that with the neocortex, not neocortex, but with cortex.
In my tutorial yesterday, I just said cortex general, and I simply put the hippocampus
at the top of the hierarchy. When you have an alternative learning system that
accounts for the rest of the cortex. Yeah, it's specialized.
One of the reasons is it's at the top of the hierarchy and it actually converts,
it's very elegant, it converts the.
Supragranularly dominated feed-forward path to the infragranularly dominated feedback path.
So from entorhinal up into the hippocampus, you get the feed-forward path and
it comes back in the deep path, in the feedback path.
So there's this nice hinge, but of course, the hippocampus preceded neocortex
by millions of years, by a long evolutionary history.
So it had a separate function long before it became the hinge of the cortical
counter-curve, which I… Are you sure about that?
I mean, I think… What are you saying about the illusion of hippocampus?
It preceded that only mammals have neocortex, and a lot of vertebrates have…
The medial pallium is some hippocampal catalogue.
It's the unlogger of the hippocampus. It grows is big only in the neocortex
because it serves it in very intimate ways.
But there is hippocampus and an unlogged amygdala prior to man.
Well, there is a sort of three-layer sort of cortex in reptiles,
and the hippocampus is more sort of, I guess, reptilian.
Yeah, so it's really cortex, so it's a three-layer, and then you get this mammalian
invention of five-layer cortex, which birds don't have, though their forebrain expands vastly.
And they all have a hippocampus.
In fact, hippocampus is one of the very good examples in birds,
not only vocal learning, but these food caching jays and scrub jays and stuff
somebody up in Siberia they've studied them they hide seeds in tens of thousands
of different places one animal,
tens of thousands of places. They remember, they know where they are.
And the ones who have different species, the ones that hide in fewer places
have smaller hippocampus.
The ones that hide in more thousands and thousands of places have bigger hippocampus.
Very nice volumetrics. Otherwise, the animals are pretty similar.
But there's something interesting about that, Bjorn, also relative to the architecture
of the neocortex, because you indeed see then the hippocampal hinge.
This is where feed-forward switches back to feedback, but there's another pathway
like that which runs over frontal cortex, which is receiving more attention in some sense.
You say, okay, I'm at forward to frontal cortex, and then the feedback projections come back as well.
So now I have two orthogonal, if you want, hinges in some sense.
So how should I think about that? They are enmeshed. If you look at the connectivity
instead of where they are physically,
and you look back at Malcolm Young's beautiful graph theory plot from the macaque, Kokomak, in 1995.
He shows there is this beautiful visual part of the graph, one big thing hanging out to the left.
There is the somatosensory in the middle and the auditory coming off on the right.
And then there is the gustatory.
And they all converge in what he calls the frontal limbic cap.
And when you look at the cortical areas, they are completely enmeshed with reciprocal connections.
The hippocampus sits in the middle of the frontal things like a spider in a web.
When I saw that, I said, now I understand finally. They are all the highest level of the system.
So that hinge is a broader hinge than hippocampus.
It's enmeshed. And when simply looking at this graph, you see the densest number
of connections are up there. They're all crowded together.
Limbic frontal and hippocampal is all meshed together.
And then the modality-specific streams, the hierarchies, are hanging off it.
So that's the head of the system. And the hippocampus is connectively very close
to frontal area, much more distant to visual, for example, than frontal.
So it's the next-known neighbor to those guys.
So if you just collapse, if you just shrink the fiber lengths.
So sometimes the physical positioning of an area can be very deceptive.
Here you have at the tip of the temporal, middle of the temporal lobe is one
of them, and the other one is in frontal cortex, and they are connected just like that.
And the other deceptive one is the frontal eye fields. from the light fields
is placed frontally but connectively in the young diagram, it's in the visual system.
It's much farther back because his connections, of course, are like rubber bands.
So they pull and push till they settle to an equilibrium in his modeling of this.
So there you see that really frontal life films is high in the visual dorsal stream.
Right. But now more recently, there was another proposal by David Van Essen
and Henry Kennedy where they spoke of this sort of bowtie structure.
I think in classical architecture, it means there's a core at the center of
the bowtie that is very densely interconnected.
And then there are these two wings of the bowtie that are feeding into it from
the periphery or have received inputs from it.
And that will give you the bowtie structure. Is it that your mind is still compatible
with the older Malcolm Young picture, or is there any change?
Did anything change in that picture?
What has changed is that at the time of Malcolm Young, they interpreted it as a small world system.
And it was by the evidence they had then. What has happened since,
which was updated in this Vanessa, I love that paper.
I mean, it's a tremendous synthesis they had done there. and what
they they found so many more connections that hadn't
been reported back then and when you add those
connections it's it's it's more connected than
a small world network should be and that's
how they got the bow tie structure and it doesn't conflict with the old view
it's simply another version of having an efficient network and it's to have
have this sort of super hub of in the middle and having sensory things feed
into it and things where it projects coming out.
But if you look at their diagram, their sort of.
Conceptual sketch of the boat type you see one double huge arrow you're going
from the feed forward and the feedback is intact that whole and that's the thing
that's essential about about,
malcolm young and all the old versions of this is that feed forward and feedback are streaming.
Against each other all the time that's sort of the basic right and and but now
what you also did yesterday which i found very interesting uh so so we have
now this outline of of the anatomy.
We have a feeling maybe for for how this mapping to
function but you went a step further we're really formulating at
least as a hypothesis distinct functions of them sort of major subdivisions
of the architecture right yeah so could you could you step us through those
and I only did my guess work on,
because it is, as far as I can tell,
there is no consensus on what these things do in the sense of how to describe
them at the very abstract generic functional level.
For neocortex, my guess is that it's doing veridical source reconstruction across all afferents.
You should remember things like
the cellular system has a cortical representation. Everything is up there.
It wants to make sure it has access, rather direct access, to all the basic
sensory systems, including the stimulus, which is quite spectacular, actually.
So what do I mean by virilical source reconstruction across all afferents?
Its task is to tell us in a sense what reality is, and to do that,
it needs help from the base of ganglia, mostly in terms of how to deal with
the world, and from the cerebellum in terms of maybe.
Decorrelation, maybe something else, but there are these very strong anatomical
connections from cortex into cerebellum, from cerebellum back,
same thing with basal ganglia.
So, and of course, basal ganglia is serving much more than motor cortex,
it's serving vast areas of the cortical mantle or projecting into the basal
ganglia and then and they get down through the funnel and shit back through
the thalamus up to cortex.
So division of labor...
Source reconstruction means because sensory afference is not only noisy,
but loaded with so much ambiguity in terms of ill-posed and inverse problems.
In vision, for example, there is a whole catalogue. Computer vision has been
a catalogue of working through all the inverse problems that you have to solve
to make sense of what's on the retina.
If you have a circle on the retina, it could be an ellipse at the tilt,
it could be a close ellipse at the tilt, or a small circle close to you,
or a big circle far away from you. It could be any number of things.
It could be an infinity of stimuli that generate the same retinal image.
So how do you tell? Well, in a monocular view, you can't tell.
You need some additional cues. fuse. So you open two eyes or you move your head
and suddenly you see, well, it can't be far away because parallax is little and parallax is much.
There's lots of parallax when I move my head, so it must be close.
And so you disambiguate.
And then you take the help of other systems.
If you're really unsure, you might try to touch
the thing and see what what's actually there uh not
not as an adult but early in development um
so so it has
to reconstruct the the my my bet is that cortex is there in order to give us
a realistic model of reality it has to reconstruct from the senses the auditory
system of course forget about it you know All this noise is binging on the cochlea,
and the first thing it does is a huge Fourier transform.
So it has to reconstruct what's out there from this very indirect,
very noisy, very ill-posed and inverse problem riddle sensory afference.
And how does it come to the brain?
As chattering of action potentials in millions of fibers.
So it's a huge task. and, of course, the lower levels that we talked about,
they solve these things by simply throwing away information.
The colliculus doesn't reconstruct objects. It just knows whereabouts they are
when they move and when they shine brightly over there.
That's all it cares about. It doesn't have to reconstruct anything that way.
Location in space is enough for it.
So once it has that, it can direct the eyes there. Its job is done.
The cortex wants to know what is the object like, and therefore it is confronted
with all these ambiguities and inverse problems.
So it has a huge job. And for me, the hierarchy, the hierarchical organization
of cortex with feedback and feedforward is to extract priors from its experience
with the world to make the next experience clearer,
to clarify the view.
And so our perception of the world goes from, it's sort of an irreversible gain in clarity.
We see the world clearer and clearer from infancy and up.
And we understand more and more about it. By our age, the perceptual lessons
are long gone. They have been handled long ago.
We just go about the world. That's the most self-evident thing.
So it's so easy to overlook how difficult those problems are. Size constancy.
The thing that's small in the retina can be just as big as this if it's far
away. That has to be acquired.
And it's acquired first for the proximal space and then extends out.
Size constancy, the progress of size constancy is simply extending the constancy
farther and farther out.
So some people living evidently in rainforests with dense foliage all around
them, with never open spaces, they have deficient, distant fact size constants.
They have very good clothes. But if you take them out from the rainforest to
the plains and they see a gazelle, they've never seen such a small animal. They're surprised.
I was making this one up. No, sure, but how does that scale?
Because, okay, I can imagine that this is a really good summary if I will be
just a passive observer of the world and I want to have accuracy about this world.
But in some sense, the brain wants to act. Yes. Otherwise, you're toast, right?
We want that. So how does this model now of neocortex map into the ability of
making a decision and matching your goals and building strategies?
So how does it generalize to this
more active, goal-oriented part of the system? Tony and others come in.
But you have to answer the question. Frontal, frontal, basal,
ganglia, circuitry, going back to cortex.
So first of all, the desirability of an accurate view of accuracy in the reconstruction,
the vertical, the vertical part of it is a big thing.
And cortex, instead of like colliculus or other subcortical things throwing
information away and saying, I don't care, I just want a fast solution,
they get instant certainty at the price of permanent ignorance.
Okay? Okay, so if you really want to know what's out there, you've got to be accurate.
So cortex steps back and says, I'm neutral.
I'm not making any commitments initially. I simply make it on a probabilistic basis.
It's probable that this is a slanted line oriented 45 degrees to the right.
Okay, it's a probability. So a tuning curve for orientation in the visual cortex
is like a probability density distribution.
So the cortex keeps doing that and then it starts merging in the hierarchy with
feedforward and feedback, the incoming information with the priors,
and at some point it has to collapse these things to an estimate.
And that's when we get the best estimate we can get of what's out there.
But we have to act like you say what do we do about it we're
not just some big eye viewing the world contemplatively
gazing at our navel so how do we get about it then you have what you said goals
motivation we have to have we have to accomplish the things we have to accomplish
we are we are basically a metabolic engine that converts food into energy and in dividing up that
for all the tasks that need to be accomplished, we have trade-offs.
So you can spend all your time searching for a mate.
If you don't eat along the way and allocate enough time for eating,
you will starve to death before you find a mate.
So trade-offs, protecting your body, finding food, finding shelter,
finding the right environmental circumstances, finding the right social environment.
So trade-offs constantly, dividing up the energy to different tasks,
and evolution has taken care of a lot of that, saying there are some fundamentals
that you have to accomplish.
You have to eat, you have to protect your body integrity, pain system,
you have social motives, you have fear, protect yourself from danger,
and you have curiosity to explore.
Yeah, but I'm not sure you're solving the problem.
Well, you're putting the solution to that part of the problem goes back into
the brainstem, areas like the hypothalamus. Yeah, yeah.
But then there's also a sort of led architecture of motivational systems that
you would then trace up into cortex.
Yep. And how does that link to areas like the hypothalamus? I would say that
it's just like you say. They are up there.
The larynx system is the motivational layer, in a sense, the motivational compartment
of the cortex for some of the motivational systems.
Not only that, but the frontal cortex, the orbitofrontal system is shot through
with motivational things like social and so on and it's all gonna now the problem
about cortex and action is,
that the cortex is doing this big sort of
objective reconstruction of the world but it's doing it in parallel and action
goes serially one action at the time essentially and how do you convert this
huge parallel display of of incredible
amounts of information, including motivational information, in frontal and limbic,
how do you get it down to behavior?
You have to have something that converts that, in a.
Sequences, action, based on the best evidence that all those parallel areas are supplying to it.
It seems to me that that's done by piping it into the striatum and into the
basal ganglia, which looks to me like a big funnel, which whittles things down
to a narrower and narrower compass.
That's the final one, but the globus pallidus externa is bigger than the interna.
And Niagara is smaller than that.
So it's a big funnel. It gets narrower and narrower because parallel things
are now competing and you're selecting the most urgent thing along the way until
finally there is a Niagara signal that says, okay, that one, release it.
It has won a competition along the way in a sense through the base of Yangon.
It's like an obstacle race.
There are all these other, I think at the core of entertainment,
there are all these other guys competing.
You know, there's a topography there from Cortex. All these other guys competing.
And the further down you get, the more they have knocked out the competitors.
And one is going to win. That's the one that right now is going to be acted on.
And next comes the next runner-up who is busy.
He still wants to get to control the system.
And that's the next one. So now that this one is accomplished,
urgency for this one is lessening, the next guy wins, and so on.
How that's worked out in circuitry, I can't tell you.
But that's my intuitive sense of this funnel, this narrowing thing down to the little naiva in the...
So that means, in your mind, the whole process of actually comparing,
deciding, evaluating, so on, is a process playing out at the Bezo-Gange level.
Frontal basal ganglia yeah but the cortex is only dealing with building little
models yeah of whatever states you might care about but it's also hierarchical
so so uh super ordered goals are in frontal and so they are gonna and and if
you look at the weighting of input from cortex,
to the to the striatum you have the the tail of the car date that's where visuals
you know Where sensory stuff gets in.
And the farther frontal you get, the more massive is the input to the striatum.
So the striatum is biased towards frontal input.
And that's because it's high level. It's a big thing. It cares about things
that are talking to the motor system.
Yeah, it cares. The bits of cortex that directly or indirectly talk to the motor
system. So only visual cortex.
It doesn't speak to the cortex. It doesn't matter much. But there still is a
little bit, it still keeps an eye open a little bit, but massively for the motor-related
things, and which is… We just talked about sort of early and late attention.
So in a sense, the basal ganglia is late attention,
it's you already identified some candidate action plans and you're now down
to choosing, you know, which of a small number of things to do,
rather than cortex calculating everything that you could possibly do and having
to choose between the two.
So there's a lot of ways of resolving those competitions that don't require
maybe secretive basic anglia, but operate through maybe a tractor,
like dynamics and then cortex.
Yeah, I perfectly agree. It makes eminent sense. And there was something that it reminded me of.
Which is that the reinforcement learning thing makes perfect sense in that context
of the whittling down of competitors from a parallel thing to a serial thing.
Because now, if there is this successive obstacle race across narrower and narrower competition.
Then you can go in with a reinforcement signal, the outcomes that were successful,
you can say mark that one that's a good one let that one slip by faster next
time that was a good one so you can start marking in the base of ganglia marking
synapses or whatever constellation of activity it is,
by reinforcement blurring outcomes because that's what action tells you I know
you threw in the base of ganglia to pacify Tony and it seems to be working so that's problematic
because you swing
something else that you haven't accounted for because
the starting point is and it's very meditative brain
i care about having very accurate
reconstructions of the the sources of my center stimulation in the aphid world
and i use a form of predictive coding for that right okay fine i can buy that
for for let's say everything up to the central of sulcus from the occipital cortex. Yes, exactly.
So you have delta in that part. Yeah. Which is not talking about the basic area.
Now you say, okay, if you go more frontal, then goals come.
But you haven't already taught me how I use that same hardware,
that cortical hardware, to get this goal representation.
So how do the goals come in? Where do they come from?
High levels on the sensory hierarchy are ideas and concepts,
high levels of the motor hierarchy are plans, goals, and intentions. Mm-hmm .
But can't we say everything that is more, let's say, parietal looks at the outside world.
Everything more frontal looks at the inside world.
It looks towards action. No, but if you go intention.
No, but that's what are they for? They are tying you up.
They are tying up your trade-offs tremendously. As soon as you set out the goal
to be the greatest guitarist in the world. That's Tony. Tony is.
Oh, I know you. So that wasn't by a pick.
You try again, right? Now guitar. What else? Soon there'll be Sheffield.
World's greatest guitars.
That eliminates a lot of stuff right off.
You're now going to spend eight hours a day. you know
you're narrowing your options by setting up
such a goal so what was that an answer to
something you just said i was saying would it
be fair to say that all the everything in front of the central circus
characters in all states of the internal they are yeah and they you were not
convinced by that yeah i objected and that the reason for that is that that
goal you're setting up is really not so much you feel it and in that sense it's
internal but it really has to do with your faith in the world,
your trajectory through this world that you have built up objectively.
How am I going to get to that place?
And it now eliminates, once you have set up the future goal,
and goals are always in the future, once that's set up, you are constraining
action in very, very concrete and basic ways.
Because now you can't play all day.
Now you have to practice and so on. so it
so even so though they are internal in
the sense that they're that you you feel an
impulse to do something or a desire to be
that great artist or an ambition that's the internal part and of course that's
related to what I said about motivational systems being meshed in meshed with
the trompo that's why young calls it frontal limbic frontal limbic is a very
nice concept so so So naturally there is this and there is the insulin for proprioceptive,
for interoceptive.
So naturally, but ultimately I would say goals and planning is the big topic,
the overriding topic is action.
Okay, so now we have an idea of the core hierarchy.
And now in some sense we have a more, let's say at the microscopic level,
we have to now Now start to think about how all this information is represented
and passed along between all these circuits.
And some time ago, you wrote a fantastic review on gamma responses in the brain,
which I would think you wouldn't also see as a possible substrate for this kind
of information exchange.
This is baiting me.
Oh, you're off the main line. I'm on to you. So, I said that easy.
But the point is, because now we talked in abstracto about these cortical hierarchies,
but now signals have to pass between all these different stages.
So how should I think about that? How does that happen? My answer to that is,
what passes is signal energy, not messages.
And how does it pass? it always passes through fibro-bundles that are coherently organized,
essentially most of the time in point-to-point topographies they may be broken
up by interleaved columns but across the columns you're still maintaining the topography,
from map to map there are these transformations from map to map and there are
examples of actually twisted fibro-bundles like from cortex apparently ground
it down to the pre-survival nuclear in the ponds,
there's this weird topology and then it pipes up to the Ceylon.
My take on the interaction, how they work together, would be that each system
is doing its specialty all the time.
But the signal exchange was the question. How do they exchange signals? By their projections.
What is the signal? Where is the message?
There isn't any message. Well, you're setting traps for me. Rate coding, yes.
Really? Yes. That's a fundamental question. Because you said it last year,
what the computational issues are.
It's energy, not messages. But you've also talked very much in sort of representational
terms about what's happening in cortex.
These are two things which are quite hard to square, because if you want to
go down a sort of really hardcore dynamicist route, you wouldn't mention the
word representation in your language either.
You would say the brain is just in tuned with its body and the environment in
such a way that, you know, it balances between appropriate protractors so that
the animal does the right thing in the right time.
But all of your language is actually about modularity and decomposition.
It all sounds like the kind of thing that a computationalist would love.
I am definitely not in the dynamicist camp and the dynamic systems camp,
but in terms of my representational talk, it's because what I see
those cortical areas as are two-dimensional maps, and those maps are topographically organized.
Neighborhood relations are preserved in projections from area to area.
They form a hierarchy, and there is a pattern.
On each map a certain as a certain you've
seen that the famous to tell the picture of when
they showed when they showed a macabre bullseye with 2d oxyglucose and they
developed the films and they showed it the beautiful transformed image and visual
cortex a lot of people misunderstood you by a weasel when they said that the
stimulus is taken apart into orientation color and so on They said that locally,
and people started thinking globally,
but they also said equally emphatically that as you move the electrode across
V1, you are crossing the columns and.
Over distance you are maintaining the retinal topography.
So there is an image in V1, there is a physical pattern of different activation
states, and I treat the neurons as pixels in a picture.
Essentially, that's my metaphor, a very abstract metaphor.
There are pixels in a picture, on a screen, and what happens across the hierarchy
is it gets transformed, the screens get smaller and smaller,
they get more and more abstract, and they are interacting.
This sounds a little bit like what people have described as morphological computation.
Morphological computation in the context of the brain, you're taking advantage
of the the geometries and the physical hardware that they end up to do computation for you.
Are you trying to describe me as a complete disaster now?
I mean, we were doing so well, yes. It sounded so convincing.
And we could agree. What you're saying, the morphological computation fits very
well with how I think, and I call it analog computing. Right,
but that would not pacify me.
Because now suddenly we completely switched perspectives, and we talk about
maps, 2D maps that are wired, conserving some sort of topography between them.
On the other hand, you talk also about hierarchical feed-forward top-down relations.
Yeah. So that means I have a forward and backward topographically preserving
projections between all these maps. So you're going to run out of wires very quickly.
Not at all. The wires are fixed,
That doesn't solve my problem. The interaction between feedforward and feedback
simply changes the synaptic weights and fills those areas in the algorithm. How can it scale, Bjorn?
How can it scale? Because here you have this sort of Buddha brain that just
contemplates the world and tries to estimate the sources of its sensory stimulation.
The world is all practically infinitely variable.
And in your case I have to capture all that content by really labeling individual
connections with a certain meaning right?
There's no way to escape from that there are no labels you have labeled lines,
you must because you have topography preserving maps so if you want to encode
any combination of features It is a combination of features expressed by activities
and locations in such a map.
And locations in the map must be conserved as you move through such a hierarchy.
So if I now want to have, let's say, a location invariant representation of
your hat, and I want to also have a rotation invariant representation of your
hat, there are a lot of wires going between these maps.
That's way up in the hippocampus, or way up next to the hippocampus,
in the perihippocampal, and that's where those recognition things are.
Your knowledge of a rotation and abstract hat is not this hat.
This hat is located in a specific place on your maps, all the way up through
the infratemporal cortex.
They still maintain topography all the way down through infratemporal cortex.
When you get up to HT and parahippocampal and entorhinal, that's when you start
breaking down topography.
And once you are in hippocampus, you are in sparse mass, which is,
you know, people, a lot of people don't understand that.
That the place cells that are responding to
a single location in a rat's cage
are spread all over the hippocampus through
the cross-section and scattering it looks like pepper you know like scattered
you know and the guys next to them are is another subset of hippocampal cells
so there is no spatial map in the colliculus or the visual one sense in the
hippocampus that's an abstract
map that's for storing the maximum amount of information in a finite number of cells.
But I'm not running out of pixels or wires because these screens,
the maps, the two-dimensional maps, just metaphorically think of them like a TV screen.
You're not running out of TV pixels because you're showing a movie on them.
They are scintillating and changing all the time. and what they are displaying
is every one of the areas has a different content depending on where it is in
the hierarchy and that has to do with the priors.
For instance, okay, an example. And atomically, it's a fact that inter-aerial
synapses landing into a volume of cortex,
but the process coming from another part of cortex or the long-range interactions is.
Less than 3-2%.
It's minimal. It's very, very small.
I love it. But in your proposal, in your proposal now, I think you will need
way more than this 2% to wire your system together to be able to be this meditating
Buddha and say, ah, it's a box here.
I don't quite see why. We may be.
Kawato did a very nice well that was
this is a test so we should make a prediction what kind of wiring ratios you
would get local global and whether they match what you find in the real thing
I like when I read Young back then his numbers.
For density of interconnection but remember those two, three percent,
they are not scattered randomly. They are into part. They are in register.
They're always in register. Even though they're... We will finish this over dinner.
I think that because we need maybe some wine to go along with this,
because we're speculating.
And we know you don't like that. But,
So, Björn, with your long career in many fields, which is amazing, right?
You've been in many fields. You have impact in many fields from music to neuroscience
to humans' understanding of mushrooms.
You've been all over the place, right? And it is really astonishing.
Vedic exegesis, I have solved puzzles that Vedic translators have not managed to solve in the Riveda.
And I published a paper in Mongolian studies
where I solved four puzzles
in the translation of nobody had solved and the
reason was I had studied the step nomads religious ideas and there's the step
has a common culture there no mass are going back and forth the ring made over
step people came down into India and they carried some of their lore from the
step with them and you knowing what
they thought about the stories and so on, I could solve problems.
So, re-veila exegesis without knowing Sanskrit?
But now you do know Sanskrit. No, I don't. No, okay. Never did I read it. Oh, okay.
But I heard you sing the girl from Ipinema once in Sanskrit.
Oh, yeah. But that was, I un-memorized it. Oh, okay. Memorized again.
So, it was summertime. Oh, it was summertime, sorry.
But, okay, look, Look, this broad experience that you bring to the table here,
what is Bjorn's law that we should follow in order to understand how the brain works?
First of all forget about language that's the
last thing you want to look at once you have solved all
the rest of it you can add language probably easily so forget about language
and messages and so on are language based metaphors look at analog computers
how you would solve these things analog
wise which would be morphological computation in a sense and third one,
I think that's enough okay so then um totally convinced you soon in Christian's thought,
about three years from now to to check whether you actually have falsified or
verified your key hypothesis on how the brain works so what hypothesis would
you like to see really tested in a three year time frame,
that's a good question That's a nice question.
Finally, we have a question he liked. Well, I liked all of them.
You tried to trick me on Ghana and the what was the other one?
Yeah, you tried to trick me on Ghana and the one on the No, you weren't trying
to trick me, you just wanted to ask a question.
Let's see. Prediction.
Prediction for the future.
And it's also one that Tony can appreciate. He's going to come to Christian
and start to check whether it actually happened.
In winter, it shouldn't be too pleasant.
I'm trying to get out of this field. So you're going to prediction and walk away, right?
Not me alone, but Tony rings the doorbell. Let's see.
My dream experiment my dream research line three years from now with an answer what would that be,
hell I can't think well I think given our discussion an obvious one would be
that you're going to prove that you're topographically conserving architecture
that you can scale. I know now.
I mean to support you. I know now.
It's not that. I likely touched upon the fact that cortex works probabilistically,
but you need an estimate. And.
There is work that shows that as long as cortex works on the probabilistic basis.
Its transmission is fast its its operations are fast as soon as you want try
to constrain cortex to precipitate an estimate things not crash but slow down
and i forget now the name of the people who suggested that.
So probabilistically cortex is fast and
Mumford back in the 90s had a
couple of papers on corticothalamic relations and in
the abstract he says estimates are made
in the thalamus and I am on that line I think that cortex is working on a probabilistic
basis and when it is time to make an estimate for actual action things these
are these those are made in in subcortical locations and like basal ganglia for action,
colliculus for prioritization and orienting.
And I think the thing I would like to be tested is is there a global best estimate of the whole.
Where the whole probabilistic panoply of cortical areas where all that information
is collapsed to one single estimate of my best now in the moment estimate of
what's happening around me.
But this sounds like your brain, your cortex, which already had a problem with
representing all these topographic maps, now has to support a full panoply of Bayesian.
Probabilistic estimates of different interpretations of the world and maintain
those at the same time. Yeah.
So, I mean, do you really want to say that that's what the brain is doing?
I want to test whether that happens in the pool or not.
That it collapses there is massive convergence of multiple higher order cortical areas,
uh most of the the v1 projects involved now but
but the the dorsal pulmonary which is what
i'm looking at it has this massive conversion from
a lot of higher areas both frontal and parietal and temporal okay so there is
this converted place and it has it's invested with a special inhibitory internal
that exists no other place in the thalamus and their long-range inhibition in
the dorsal discovered by Kathy Rockland.
And that place to me looks like a place that might be ideally placed to make
a global best estimate in the moment of what holy cortical information,
amounts to in terms of one global estimate.
And you need sort of a global estimate because some of the things are not disambiguated
until you have put them all together.
McGurk effect, you know, You look at somebody's mouth and they say one syllable
and another one comes into your earphones and so on. Actually,
it's all the time with Tony.
But that's also in your conscious sea. The conscious sea is that.
That's the conscious sea.
So if that could be tested in three years, I would be very happy.
But your prediction would be that it will be true. It will be consistent.
There will be a global-based estimate in the dorsal pulmonary.
That's my prediction. A clearer prediction you can't get.
Very good. John Marker, thank you very much for this conversation.
Wonderful. I enjoyed it.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Program.
For more interviews, recorded lectures, or upcoming conferences in the field
of biometrics and bio-hybrid systems, go to csnnetwork.eu.
And thank you for listening.