This is the Convergent Science Network podcast. Leading researchers in the domain
of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
So this is Paul Verschoor with the Convergent Science Network podcast.
And in this episode that we recorded as part
of the CSN Barcelona Cognition Brain and
Technology Summer School series I'm talking
to Dimitri Sklosky and Dimitri
you come out of engineering now of theoretical physics and from theoretical
physics you went into neuroscience and also in your in your presentation this
morning you also show how let's say this interest in theory is translating towards
how you understand brains.
So how do you see that link exactly between theory and the practical sides of neuroscience?
So historically, neuroscience has been rich on a number of experimental observations
and facts that have been assembled through a large number of techniques
on a variety of different levels and animals.
But what has been lacking is a theoretical framework that would allow to put
these facts into a unified perspective and to make future predictions and eventually
to understand how the brain works.
Neuroscientists always are interested in finding an appropriate framework to put the facts in.
In our case, we had a lot of success with doing it by borrowing the ideas from
electrical engineering, specifically from the field of adaptive signal processing. Right.
So what you emphasized a lot there was how very specific ideas about signal
processing, as developed within engineering for a long time by now,
might help us to get some leverage in how we can understand the brain.
And so what do you see as some promising starting points there when you talk
about adaptive filtering?
Because in some sense, there are quite a number of people who go around saying,
well, the brain is an adaptive filter.
Common filters have been very successful in engineering, so the brain also must
operate like one, et cetera.
But to make that now more specific, where do you really see leverage in these
kinds of more normative approaches towards the brain? Yeah.
Well, I think to make such ideas successful and of practical value is to make
a connection between theory and experiment on a very specific level,
so that we could predict, for example,
response properties of individual neurons given a certain stimulus presentation,
which could be compared with electrophysiological recordings, for example.
And I think one of the places where this could be done easiest are sensory systems,
like visual, for example,
where you have complete control over the stimulus and you also have access to
the neurons by recording in the retina or further down in the vertebrate pathway in the LGN.
And the reason I think that the engineering principles should apply there is
because both systems have to deal with the same kind of limitations that are
presented by the physical world,
such as limitations on the dynamic range and the bandwidth of the communication
from the retina, say, to the LGN and from the LGN to the cortex.
But now, can you give me an example? Could you talk me through an example of
how such a filter would help us to understand what happens in the retina and LGN?
So one example which goes
back in time is the receptive fields of the retinal or LGN neurons that have
been known to be biphasic in time and center-surround in space.
Both of those observations can be explained based on the predictive coding framework
that originates in engineering,
where you say that the system is trying to compress the incoming signal by subtracting
a prediction from the actual signal value.
So, in terms of biphasic temporal response, for example, you would use the previous
values of the signal to predict the current one, and that's how you get a biphasic shape.
In the case of the spatial response shape, the center-surround shape,
you use the surrounding values of the signal, like the surrounding values of
image pixels, to predict the value of the pixel, a central pixel in the image,
and subtracting that prediction.
And that's how you can explain both the biphasic and the center-surround shape of the response.
But now, as a start, the intuition to look at encoding by neurons,
certainly in sensory systems, from this perspective, goes back quite a long way.
I mean, you mentioned yourself, Barlow, for instance.
That's right. When he tapped into this. So what does progress mean if we compare
it to the intuitions of Barlow and where we are today?
So Barlow and Atnav particularly borrowed the ideas from engineering.
And I think Barlow's point of view is usually summarized by the maximum redundancy reduction,
where the idea is that you just transmit the part of the signal that is non-redundant
or new or surprising or couldn't have predicted.
And then it has been followed by the introduction of predictive coding framework,
which is a concrete quantitative framework that allows to want to generate the
predictions for the receptive fields like biphasic and center-surround responses that I talked about.
And that was done by Srinivasan Laughlin and Dubs in 1982.
And then this line of work was continued by several people, most notably Attick and Van Hatteren.
And what we did recently is just try to take this work to its logical conclusion
and derive a normative theory in the most direct and transparent way.
And in that way, we could compare the predictions to the actual measurements
of spatio-temporal receptive fields that were done in catangion and insect visual system.
Okay, so what would now be the key parameters of this model?
So if I would like to take the model as a filter to look again at the brain
itself, what are the key parameters I should be sensitive to?
Right, so the model actually has no free parameters in a sense that once you
define a natural stimulus ensemble,
um so that's a statistics of the
input and um you specify
the signal to noise ratio then there
is a unique prediction for the filter shape and that
can be compared with the um electrophysiologically measured um receptive fields
by say a reverse correlation method but now um in some sense if you take something
like the signal to noise ratio of these neurons this might not be a constant necessarily, right?
This could vary depending on, let's say, the presence of neuromodulators or not.
So how well would it generalize beyond this fixing of these kinds of parameters?
Right. I was actually referring to the signal-to-noise ratio in the input that
would have to do with the absolute intensity level.
But there is also internal noise, of course,
and the impact of that noise depends on the specific circuit implementation of the filter,
which we have one proposal for that I discussed earlier this morning,
which is based on the lattice filter idea.
Before we go to the lattice filter, let's look at the simpler case first.
Because In some sense, the key signature that you see as confirming the physiology
is biphasic response, which essentially means I would have, let's say,
some onset-driven response,
and then I would have, let's say, an orthogonal or an opposite response in turn,
like I might have, let's say, a depolarizing response to something,
and then I'm hyperpolarizing, right?
This would be a quick characterization of, let's say, the simplest form of such a filter.
And so so what i'm curious about is how
specific can you make these responses right so
also what i post you the question post in the morning one way
to to look at these these the negative side of
the response the the negative tail could be to say like well after hyperpolarization
is a standard feature of all neurons whether they're encoding anything or not
so this is just a non-specific component and now you're saying no no it's a
specific component because that's exactly what I need for my predictive filter.
So how could we make this experimentally testable? Or do you think the data
is already out there to allow us to make a decision on this?
I think the answer is partially yes. And the reason, I mean,
of course, there is a physiological mechanism like after hyperpolarization that
has to underlie this response.
But I think the evidence to say that this response is there to implement productive
filtering comes from the change in the filter shape in response to the change
in the stimulus statistics.
So, for example, at high contrast, at high signal-to-noise ratio,
you get very strong biphasic response with a strong and sharp negative component.
When you go to low contrast, to low signal-to-noise ratio,
the filter changes and it carries most of the weight in the first peak,
which gets wider, and the negative rebound actually gets much weaker.
And that would be consistent with the filter doing more of low-pass filtering
rather than high-pass filtering.
Such change would be expected from the predictive coding framework,
but would require a different physiological implementation.
And so the fact that the neuron response follows this prediction suggests that
the predictive coding framework has value.
Yeah, but that would also mean that for the predictive coding framework,
you should see a stimulus-specific modulation of both the positive and the negative
phase, right? That's correct.
And is there sufficient evidence for that? If we go to just,
let's say, standard physiology of the visual system, and we start to look at,
let's say, the encoding of more or less complex scenes as an example,
do you see examples of this?
Only very few. As I mentioned, there is a change in contrast that people looked
at, and lately they've been adding more and more noise to the images and looking at the response.
But I think that this line of work actually should become a bigger project right now,
and we're trying to make connection with experimentalists to actually test this
idea more exhaustively by playing with the different statistical ensembles of
natural scenes and seeing if the filter would change accordingly.
So it is in some sense work in progress. Right, okay, understood.
Now my other question is also, so before we go to the complex version of the
model, which is this lattice filter system.
Something funny happens in the argument about adaptive filters, right?
Because, as we also discussed earlier, originally the intuition is like, well,
adaptive filters were actually a near optimal way or an optimal way,
if you want, to show how you can transduce information through some channel, okay?
And this is also developed as a technique, given the limitations and the possibilities
of the engineering that we do. And there, indeed, bandwidth is always an issue.
But now for brains themselves, for the brain, maybe bandwidth is not an issue in the same way.
Like, for instance, you could argue that actually the whole principle of the
brain is to do massive IO, but all these connections that you have available
to you with actually minimal local computation,
because biophysically that's more complex.
So the whole design might actually be exactly orthogonal to what the engineers
were thinking of when they designed these adaptive filters.
So maybe then it's the wrong metaphor to apply to this system.
Yeah, it's actually a very astute observation, Paul, because when we started
this work, our main motivation was a so-called communication bottleneck,
as I think Barlow referred to it, where he said, well,
you know, there are many more photoreceptors in the vertebrate retina than there are ganglion cells.
And so there is a bottleneck for transmitting information to the rest of the
brain, and therefore there must be compression, which could be done by redundancy reduction.
And that's the philosophy that we started with.
But I think as the work progressed, especially after looking at other systems, such as, for.
As explicit as it is in the vertebrate pathway, we started questioning this
assumption of the need for compression.
And at the same time, what started to become clear that there could be computational
advantages to implementing this predictive coding framework and decorrelating the incoming signal,
especially when it's done in a stage-wise fashion like it's done in the lattice filter.
Because when I looked up in signal processing textbooks,
it turns out that lattice filters, in addition to performing the correlation
of the original signal stream, They also are used for feature construction,
because if you output signal from each stage of the lattice filter,
you get a set of orthogonal features,
which are very convenient to be used as a set for predicting or training or
learning a correct response to another input.
Input, which presumably is what the brain is trying to achieve in associative
learning or something like that.
So now we move to this lattice filter. What makes the lattice filter interesting
and how is it different from the filter we just talked about?
Right, so lattice filter is a specific circuit implementation of a predictive coding filter,
and the defining characteristics is that decorrelation is done in stages, and,
where each consecutive stage decorrelates the signal on a different time scale.
Which seem to correspond to electrophysiological
observations of receptive fields in the retina and LGN.
Specifically, it has been measured that the temporal receptive fields in LGN
are longer than the ones in the retina.
And that's why it
seems that the lattice filter may be a good
model for the system so the key
thing is a lattice filter is hierarchical at every
state it performs the same operation roughly which essentially is to just decorrelate
the image is that the reasonable way to interpret it that's right on a different
time scale okay so that means you you decorrelate at varying time scales as
you go through the filter as you go through this cascade of filters essentially
that's right but With the local operations, it would be the same.
Similar. In the simplest model, they're exactly the same, but the lattice filter
can be modified to have slightly different operations.
So what does the lattice filter now solve that your previous linear filter did not solve?
So I think a better way to put it is that the general linear filter is a mathematical
concept that performs optimal prediction and therefore widens the incoming stimulus.
Leitz's filter is a specific circuit implementation of that filter.
And in particular, it's the one which allows you to do decorrelation in stages
by using biologically realistic elements such as neurons that have relatively short time constants.
Distance, but giving you the ability to de-correlate the signal over a longer
time scale due to the cascade hierarchical structure that you mentioned. Okay.
But now, so if it's getting close to implementation, it also means that you
must be able to make more specific predictions about how a biological system
could implement such a filter.
So what would be these specific predictions that would come out of that?
That's exactly right. Right, that's the reason to go for this specific circuit implementation,
and I think the strength of the lattice filter model is that you can map the
specific units in the circuit to the specific neurons in the brain.
And in particular, then, we could predict the responses of the retinal neurons
versus the LGN neurons, according to what the lattice filter tells you, but also we.
Can predict that there should be at least two different types of responses in LGN, for example,
which correspond to the so-called forward and backward prediction error filters in the lattice filter.
And those responses actually have been discovered electrophysiologically,
and they could be identified with the classes of cells that are called lagged
and non-lagged cells in the LGN.
And the non-lagged were discovered by Mastro Nardi more than 20 years ago,
and they have a distinct property that, although they also have a biphasic response,
but the second phase is greater in amplitude than the first,
which is rather unusual.
Is that still a negative phase, or that can be a positive phase, or it doesn't matter?
Right. So the important thing is that the two phases have the different signs,
It doesn't matter whether the first one is plus and the second is minus,
or the first is minus and the second is plus.
In this way, the classification into lagged and non-lagged cells is separate
from the classifications of cells into on and off.
So both lagged and non-lagged cells can come in on and off varieties.
What's the duration of this lagged response?
So what lagged cells do is in response to a step stimulus, they respond with
an initial delay which could be a few tens of milliseconds and that's why they're
called lagged. Right, okay.
But then how would your filter cascade account for that kind of lag given that
you would only have, let's say, three or four synaptic steps to get there, right?
If we go from the photoreceptor to our ganglion cells and then to your lag cells,
actually would, well, let's say two to three synaptic steps.
So how would you now relate that to this cascade of filters,
which are positive and your negative prediction components?
So, it is a little bit hard to explain without showing any diagrams,
but I can say... But for that, we recommend people to look at the video lecture. Exactly.
But the basic idea is that the photoreceptors at the very front of the cascade
perform low-pass filtering,
thus introducing the delay of various frequency components.
And then each stage of the pathway invokes a so-called all-pass filter,
which transmits all the frequency components equally, but introduces differential
phase delays depending on the frequency, which could be thought of as delays,
as just pure delays.
And as those delays accumulate from stage to stage, they lead to this electrophysiologically.
Notable lagged responses.
But it would mean in your case, the prediction would be that somewhere in this
network of ganglion cells or so, this buildup of delays would then happen.
Is that the logical consequence? That's right.
I think those interactions could happen already on the bipolar cell level.
For example, it is known that although off-bipolar cells have fast response
and they use ionotropic ion channels.
The on-bipolar cells have a delayed response because they use a metabotropic ion channels.
And that delay response I think is about 20 ms or so.
So that could be the original source of the delay.
But then there is further processing, of course, in the bipolar to ganglion
cell synapses and in the amacrine cell network interacting with bipolar cells.
So what I liked also in this model that you presented is that you actually purposefully
want to apply it both to, let's say, vertebrates and invertebrate systems, right?
Because you do believe, or wish at least, that the same principles will hold.
Right. This is correct. Correct.
So as an example, you talked about a specific system in the fly brain, right?
So how well did the model map onto that system? How well did that work out?
Yeah, I think what you said is really important that, you know, really good.
Powerful and correct theory has to apply across species, has to be general enough.
And so we're particularly pleased that this theory works for invertebrates as well, like in flies.
And in particular, in flies, the photoreceptors synapse on the cells called,
large monopolar cells, which have two biggest classes that are called L1 and L2,
and they're very similar in their initial response and the part of their anatomical
features, which led us to think about the dual pathway communication,
which is a hallmark of the lattice filter itself.
And that's how we got to the idea of the lattice filter, thinking about L1 and
L2 as being those two pathways, the forward and the backward prediction error filter.
So what evidence did you find that they could indeed exchange information in
a way that would be consistent with that model?
So that evidence is still somewhat sketchy because it is very hard to do electrophysiology in flies.
And what electrophysiology has been done was based mostly on recording from
cell bodies, although there was some in the axons.
But those measurements initially did not show difference in response properties between L1 and L2.
However, more recent measurements of the calcium dynamics
in the axon terminals of L1, L2, which is the output of those cells,
showed a different response between L1 and L2.
And in fact, the kind of response that has been reported by the clandinin lab
shows features indicative of the forward and backward pathways of the lightest Right.
So, now you're in a very unique position, right?
Because you work at Janelia Farms, and you also have been very much involved
in a very detailed reconstruction of the brain of these flies, right?
Now, the data set that you have for playing there, and I hope that you can explain
to me a little bit what you guys have been doing there, this is also giving
you now a grounding again to look at this more theoretical model, right?
So what's the data you have in your hands now on that fly brain at the anatomical
level that would help you understand this kind of filter model?
So what you're referring to is another direction in my group,
which is a high-throughput reconstruction of the connectome or the wiring diagram
of the brain on the synapse level.
And we've been doing that in the fly visual system, and this project is now bearing fruits.
And in particular, we were able to assemble the visual pathway in fly through
the first two neuropills, lamina and medulla.
And medulla was done for the first time. And what we have now is a kind of idealized processing column,
which repeats itself in the visual system in parallel.
And that column contains about 50 neurons, and we have attempted to map out
all the synaptic connections between those 50 neurons.
How many synaptic connections do they have?
So it's of order of 10,000. Okay.
That's including the gap junction, so that's only the synaptic connections?
The current imaging techniques that we're using based on electron microscopy
doesn't allow us to see gap junctions clearly in our data set.
So what we're reporting is just the chemical synapse. Right, okay.
So now the wiring diagram that you now have extracted from that column How does
it map onto this model of an adaptive filter?
So parts of that wiring diagram are consistent with the lattice filter,
but what we are also seeing is that the circuit is more complex,
and in particular, it seems that it isn't just focused on the decorrelation,
as one would expect from the compression and redundancy reduction point of view,
but also on the feature extraction that we mentioned previously,
using, of course, the correlation, but on feature extraction that can be used
to build other features for more specific purposes, such as,
for example, motion detection.
But how would I see that at an anatomical level? So I have my photoreceptor
projections, and I have my lobula, and I have my lamina and medulla.
So what are the specific wiring templates, if you want, that you can now extract from this?
Okay, this wiring template is more the predictive filter, and that wiring template
is, let's say, feature extraction.
Yeah, so, of course, just from anatomy, it might be hard to know that conclusively.
So what we
know already is that you know l1 and
l2 both are postsynaptic
to photoreceptors which is what you would
expect in the first stage of the lattice filter we know that they interact by
means of gap junctions which have been reported by the bores lab so they could
potentially be the two pathways forward and backward pathways of the lattice
filter and their output as measured by calcium imaging seems to support that interpretation.
What happens afterwards in the medulla is.
Are rather somewhat unclear and still work in progress, but it is already known
that L1 pathway and L2 pathway are involved in motion detection.
And so what we are doing,
we're tracing through the cells postsynaptic through L1 and L2 to see if they
would be consistent with the interpretation of feature construction or decorrelation.
Okay, but now if you have 50 neurons a column you have about 10,000 connections
so I could argue that in that setup
you can find basically any type of connection pattern you would like.
So in some sense the question is more about which obvious connection patterns are absent.
So which pattern of connectivity is absent that would support your hypothesis?
So, let me say it this way.
So, it is true that there is a zoo of connections, and we have to orient ourselves in it.
But there are a few things that help us. The first one is that each connection
has a multiple number of synapses in parallel.
So if neurons A and B have synaptic connection, there are usually tens or sometimes
more than a hundred synaptic contacts in parallel.
And so we can order connections in terms of their strength by using the number
of contacts as a proxy for connection weight.
And of course, initially we focus on the strongest connection.
So in some sense, we first look at the scaffolding of that network.
That's the first thing that we use.
The second is how that circuit is divided in between columns.
So the fly visual system, if you think about looking at a fly eye,
starts with about 800 so-called amatidia, which correspond to basically pixels
of the image that the fly sees.
And then each of those pixels is initially processed independently from the other.
And that forms the basis of the so-called column that has about 50 neurons.
That's the one I described.
And the processing structure is rather periodic.
So it's almost a crystalline structure of 800 units. Okay.
And we focused initially on the connection within the unit.
But if the structure were to perform motion detection, it has to correlate signals
from adjacent pixels, at least.
And therefore, there must be connections between the columns.
And so we know then that the connections that are necessary for motion detection
should span multiple columns.
And that's how we can determine which connections would have to be involved in motion detection.
So if we did not see any connections between columns at this stage,
we would be very surprised because then the system couldn't do motion detection.
Fortunately, we found such connections, and they are a natural substrate for motion detection.
Did you find any evidence for the famous Reichardt detector that sort of tries
to extract motion by correlating different input signals?
So I think this is a million-dollar question, of course,
and I think philosophically the Reichardt detection is correct in the sense
that the system is comparing a signal from one pixel with a delayed signal from an adjacent pixel.
But we seem to favor a slightly different form of such comparison,
which actually does not involve multiplication,
but contains another non-linearity that is necessary for forming a motion signal.
Which is what, the threshold? Erectification. Okay.
So this is pretty amazing, right? So sometimes you guys have it all now because
you have access to an exquisite data set of this brain. You have a theory.
Now you're trying to match. But in some sense, I could say, but maybe you're
barking up the wrong tree, right?
Because maybe the fly brain or any brain did not evolve to optimize signal transduction.
It just got optimized to generate behavior.
And all you're telling me now is how in this complex set of connections in the
fly brain, I can optimize a signal.
But at some point, this fly just has to go left or right or up and down the
land or whatever, or, you know, pursue the sugar.
So where does this mapping take place? How do I get behavior out of this and
also functionally relevant responses?
Right. So this is, of course, the hologram of neuroscience. How do you get from
sensory inputs to behavior?
And we think that we're moving in the right direction.
And there are two arguments that I can make.
Well, the first one is the idea behind those redundancy reduction and predictive
coding approaches is to come up with some theoretical framework which is not
based on the specific task that the animal has to perform, right?
So you say, well, I want to communicate information to the rest of the brain
domain as fully and as quickly as possible.
And then whatever task is needed to do will be possible to do if we preserve all the information.
So initially, of course, this requirement was viewed as the strength of a theory,
because you can come up with the predictions that were task-independent,
which I think is completely appropriate for the front end of the visual system.
As we are moving further into the visual system, of course, this is not good
anymore and we have to come up with task-specific computations.
And I think motion detection is the first step in that direction.
The second part of my answer is, you know, I would put it in the fallen way,
which I learned from talking with Sydney Brenner, who spearheaded the reconstruction of the C.
Elegans conictome about 30 years ago.
The explanation is the following. Whenever you come up with a model, there is no way to prove.
Fully and rigorously that the model is correct. You can only disprove models
by saying that they don't fit experimental facts, and if the model is not disproven,
it's still in the running.
But you always get questions, you know, how do you know that your model is the right one?
How come there couldn't be some other wire that sends information directly from
the photoreceptors to the avoidance response neurons in flies.
And that's exactly why we are doing a complete conic-tome reconstruction.
Because once we reconstruct all the connections in the fly brain,
we can answer that there is no other wire.
And so I think that if we combine the theoretical approaches with electrophysiology
and with behavioral tests with the conic-tome,
that's when we can say well no we
are not barking up the wrong tree um this is
a necessary condition the model would have to
use this particular pathway but now
that but there's one aspect that i'm missing because you could
also are you look certainly if you look at the insect case we know
that that after the medulla we start
to hit these whitefield neurons that we know physiologically show
responses that are highly adapted to the behavior of
the animal like you might have specific optic flow patterns you
might have approaching obstacles like time to contact type responses so in that
sense you just want in your in your model and in your reconstruction just one
synapse away from that response so isn't another test of your model to show
that which is one single synaptic step,
you can then generate these kinds of established and also behaviorally relevant
physiological responses of your wide-field neurons.
So how would that work in your model?
Yeah, so I think that I cannot give you a complete answer now because we haven't
done that part of the work,
but this is something that people have thought about in the motion detection
context because once you have elementary motion detection that detect motion
in a local part of the visual field,
then the inputs of those local detectors can be combined to produce a global
motion response in a given neuron.
And one example of those neurons are, for example, the neurons that are supposed
to reflect rotations around different axes,
in the flight of a fly, which require a very particular map of the motion response direction,
and they can be, of course, built by using motion detectors in different parts
of the visual field with different directional selectivity.
Right. But then, in your case, to extract those features, you would have to
tap into the filter cascade at multiple levels.
You cannot just read it out at a single level, if I understood it right.
It depends on how complicated a temporal sequence you need to predict the response.
I think for the motion detection, you need to just compare two points in time,
at least if I take a correlation-based framework of Reichert and Hassenstein seriously.
So I think that you should be able to do that relatively simply.
But of course, for more complex predictions, then you would have to tap into
the lattice filter on many stages.
Right. Okay. So that might be not a testable prediction that would come from this framework.
That's correct. Okay. So that means if in your reconstruction of this fly brain,
you do not find a very wide multi-scale readout from these wide-field neurons
in this whole column or this set of columns,
then the cascade filter might not be the way the problem is solved.
That is true, of course. But I have to say that our observation is that,
unlike engineering application of lattice filters, for example,
in speech processing, it's not uncommon to have a hundred or more stages of the lattice filter.
We think in the brain, there aren't as many stages.
And just having a few stages can accomplish a lot because the processing in
the brain is applied not on a single channel level,
because those columns actually interact with each other the further you go into the system.
Then the filter can accomplish a lot, actually, with just a few stages.
Okay, but that's the kind of compression of such a filter that the engineers
haven't really tried yet.
Well, I cannot say that they haven't tried, but it's certainly not just a textbook version of a filter.
I don't know, it may exist in the literature on some level.
I think one thing that it seems that the brain is using all the time that is
rare in engineering is the use of nonlinearities.
And I think that's one thing we can learn from brains. But I guess engineers
have good reasons not to use them.
Because it complicates their lives. Exactly. It's difficult to analyze and understand. Exactly. Right.
And nature doesn't have these kinds of scruples.
So we also touched upon this whole issue of, let's say, optimal encoding frameworks
versus finite capacity models, right?
So the adaptive filter is more finite capacity, because you start to squeeze
as much information as you can through a channel that's this bottleneck.
But in optimal coding frameworks you start to sort of you're not too worried
about that problem right you just worry about how do I sort of compress my information
in an optimal way in sort of information theoretical terms,
so do you see this as contradictory approaches or do you see this as complementary
do you see this column of 50 cells in the thigh brain maybe doing both or is
it really sort of more an exclusive choice that we have to make here.
I'm not sure I got the exact... Well, in the engineering literature,
these would be seen as different approaches, right?
It's possibly also contradictory, whether you deal with optimal coding or with
finite capacity channels.
Oh, yeah. Yeah, so I think that at this point, the experimental evidence is
just maybe barely sufficient to make such a fine distinction.
But that's currently, you know, we're investigating that currently and see if
we need additional experiments to make that distinction clearly. Right.
So the other thing is, if we now go back to the vertebrate case,
we have the retina, now you have the LGN, you give us a model of how we can think about this.
There's of, let's say, optimally decorrelating these inputs.
And now we hit the cortex. And then you could argue, well, now the cortex has
this perfectly massaged signal.
So that means from a signal processing perspective, the game for cortex should
become a different game, right?
Because now it's It's optimally decorrelated. It's a perfect signal.
So from the perspective of an adaptive filter, your cortex, so this higher level
processing story, will be playing a different game.
What would that game be if you would have to guess today?
Well, I guess I would have to speculate at this point.
But I think the way I would put it is that if the goal of predictive coding
is to de-correlate the signal as much as possible,
then the stages in the retina and the LGN take you as far as possible with linear filters.
Yes, there are non-linearities in neurons along the way, but there is also evidence
they can conspire in a way to generate a linear response.
Now, when you get to the cortex, the responses of the cells are decidedly non-linear,
as for example exhibited by the complex cells in V1.
And so what I think is happening is that the cortex may be continuing the job
of the decorrelating of the input stimulus and the feature construction,
which the lattice filter stages were doing before,
but invoking additional non-linearities to make even better predictions and
make the outgoing signals even more independent than is possible with linear filters.
Okay. Or possibly actually start to correlate again, to group features together in meaningful ways.
To predict function. Yeah. Predict behavioral output, yes. Yeah.
Okay, that's great. So we made progress here in understanding sensory systems.
So, but now, Dimitri, so to finish up, we always have two questions,
right? So you come from theoretical physics.
You have been working very hard on, let's say, the anatomy of these brains,
so you know really how hard that is.
And then you try to combine that now with also theoretical work,
which I think is actually the only way forward, right?
Look at all this detail within anatomy.
So on the basis of experience, what would be Dimitri's law that we should all
follow in our aims to understand the brain and behavior?
That's a tough question.
So, I come from a tradition in theoretical physics which favored starting with a simple,
and intuitive model of even the most complex phenomena that one could be studying.
And the reason for doing this is, I think, because our brains are better suited
at analyzing the simple models.
And by simple, I mean models involving very few relevant parameters.
And if you build such a model, it kind of prevents you from overfeeding experimental
data. And also, it makes all the assumptions very transparent.
And I think that my.
My style of work in theoretical neuroscience is strongly based on that tradition
in theoretical physics,
where the simplicity and the clarity of the model is the main driving force.
And so, even when studying such complex things like the brain and how it computes,
I would like to start with the situations that can be broken down on the very simple level involving,
very few variables and few or no adjustable parameters.
And that's why we focused on the sensory system.
And only once we get a foothold there, I think then we can move on further.
But again, chipping away a small and digestible part of the problem that we
can solve in a clear and intuitive way, and then move on.
Okay, so Dmitri's Law is keep it simple. Pretty much.
Okay, very good. Good. And the second one, so if I'm going to get you back here
five years from now, since I'm such an unpleasant person, I would like to remind
you of the predictions you have been making.
So what's the one prediction that you feel most strongly about today?
I should remind you of five years from now to ask you, does that really come out?
What's that one prediction I should remind you of five years from now?
So I think that our strongest predictions are the shapes of receptive fields of the LGN neurons,
and although some of them have been seen already, I think that there are more
details there that the lattice filter model contains,
but haven't been completely verified experimentally, and in particular on the
circuit level, I think we have a model for computation,
but how it is implemented in terms of individual neuron synaptic properties.
For example, in LGN, there is a structure called triadic synapse and so on.
On how that structure builds up the receptive field, the lattice filter predicts, is not clear.
But the prediction would be
that it is exactly the computation that is proposed by the lattice filter,
which is an all-pass filter, which has a frequency-depending phase delay.
Right. Very good. Good. So, Dmitry Slotsky, thank you very much for this conversation.
Thank you very much, Paul, for having me here. Great.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European 7th Research Framework Program.
Music.