This is the Convergent Science Network podcast. Leading researchers in the domain
of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
This is Paul Verschoor together with Tony Prescott for the Convergent Science
Network podcast. Here's the 10th edition of the Barcelona Coalition Brain and
Technology Summer School. And we're here with Ton Koolen.
And Ton, in your talk this morning, you were dealing with this,
the analogy or the similarities you might find between neural networks and immune networks. Yeah.
So how did you come to study or to raise that specific question?
What kind of mileage do you think we can get in addressing it?
Yeah, how did I come to it? It was really a case of being asked,
being lucky to be asked by a group of Italian researchers who had recently developed
some really nice new models.
And in the development of these models, they find that at some point,
solving these models required mathematical technology which had only been in
the public domain for a couple of years and they were not having the experience
to do this calculation and I happened to have been working on similar problems.
And in addition, we knew each other because one of these Italians had done a
PhD in London. So one thing led to another, and I was asked to join.
And I really enjoyed that. It was really very interesting. The model was interesting,
and it allowed me to do research in a field that was really very close mathematically
to the borderline of what was then the state of the art.
So this problem had everything you would wish for in a problem.
It has clear relevance, understanding the immune system has clear medical relevance.
And it also was very interesting scientifically as someone who likes to do a
calculation, to be involved in those calculations.
So what were the specific mathematical techniques that they needed?
In the analysis of systems with very large numbers of variables,
the traditional methods that were being used until around 2000.
Work only when the number of components that each element interacts with is
either very large or very small.
But in the intermediate regime where you have components that interact with
a finite number of other parts, there were no methods available.
And this is called finite connectivity analysis and there are different mathematical
medical tricks for dealing with it but they'd only been developed in this domain
around 2000 by various people including mainly Rémy Monasson in France Giorgio Parisi in Rome.
And this turned out to be needed to solve these particular immune models.
And these techniques were already developed looking at specific natural phenomena,
or these were being developed within theoretical physics or applied mathematics
just as a technique that people thought was important?
Well, these methods tend to come from a community of solid-state physicists,
theoretical physicists, is, but it is specifically the sub-part of that community
that look at interdisciplinary problems.
So I think the motivation for looking at finite connectivity problems might
have even come from optimization problems in computer science that were being
analyzed by methods from theoretical physics.
At least the names that come to mind at this moment are typical type of people
who've been working on optimization problems.
They convert these optimization problems into theoretical physics problems and
then use the technology.
What I find instructive about that is that what you see here is how applied
problems are driving the basic science. Oh, yes.
Right? Which is actually… That's very much so. It's counterintuitive.
Usually, there's this idea like, okay, the basic science happens because we
have this genius sitting somewhere in a dungeon somewhere, and then it comes
out and becomes applied. But here, you see it exactly the other way around.
Yeah. I mean, this has been going on for about 25, 30 years at least in that community.
Initially, physicists started looking at heterogeneous systems in the context
of some obscure magnetic materials that later turned out to have no use whatsoever
anywhere, and were called spin glasses.
Since then, the main application domains of the mathematical methods that came
from these obscure materials have been ranging from computer science to economics
to biology to medicine to statistics.
And now there is a very fruitful flowing back and forth of ideas and problems
between theoretical physicists who are only theoretical physicists by virtue
of the salary they draw from the physics department.
But they work largely on non-physical problems.
So this has gone on for quite some time. So one thing I found really remarkable
in the model you presented from these people in Rome on the immune system in
this case is that they actually stumbled, if you want,
into a description of this immune system in terms of an energy landscape,
if you want, or a dynamical landscape that you could call energy.
That was identical to what people like John Hopfield have proposed now 30 years
ago as effective macroscopic descriptions of the spin glass models initially
and then of neural networks.
So is this convergence? Was this happenstance, luck, bias because of the techniques
people had? How did that come together and why is it meaningful?
Well, I think it's a mixture of many things. First of all, you have to have
seen certain domains to recognize certain approaches or models or methods for what they are.
So it demands that you are someone who is willing and able to look over the
fence away from their own discipline.
And the people who did this in Rome had been involved in various projects related
from physical systems to neural networks and so on.
So while doing the calculations, while building the models, at some point you
recognize that models are developing into a certain form that resembles something else.
But you'll only recognize it if you've actually been reading papers or preferably
been carrying out research in these other disciplines.
So breadth helps you. And also the other thing you mentioned is mathematical technology.
Mathematical technology constrains what you will be writing down in the first
place because Because you know.
Only certain types of models will be solvable. So you steer away from constructions
that you know will lead you into the morass anyway.
So all these things work together.
From your talk, I understood that both the neural networks and the immune networks
have this beautiful abstract description in terms of statistical mechanics,
which you've explored in the talk.
But I'm interested in, I think you also pointed out in the talk to some other
sort of direct analogies between immune networks and neural networks that could
provide further levels of interaction between those two fields.
Can you say a bit more about what you think those are?
Yeah, so the similarities are at different levels.
You have the mathematical level, it was already mentioned just now,
but also even if you didn't buy these particular models, if you thought,
well, this is just a choice by a couple of researchers and we could have written
models in very different ways, although qualitatively perhaps similar.
But at a higher level you see that you are looking at two systems which share
various fundamental ingredients.
A, they have large numbers of variables. B, these variables interact with each
other in a heterogeneous manner.
And C, this system is not a static system.
The forces that all these objects exert onto each other are evolving over time
on the basis pieces of experience in response to the environment that they work in.
And these are ingredients that are exactly the same in neural networks and in the immune model.
There are also differences, that's very clear.
But the way you can exploit these similarities is first in terms of intuition.
If you have, as is the case here, one field that has already been studied for
decades, like neural networks,
you develop a pretty good intuition for what to expect, pretty good intuition
for what would happen if you added certain ingredients or took certain ingredients away.
And that intuition is probably the most powerful thing to carry over to other
fields, because the details may differ.
But statistical mechanics teaches us that as soon as you have a very large system,
microscopic details of how exactly the model is built or what exactly you choose
for the types of variables or the parameters become less and less important.
So when at some microscopic level of description what is built into the model
is similar, then you also expect the behavior to be really similar.
And that helps you. That helps you in designing models. And there is never a
perfect model. Every model has a typical shelf life.
But it helps you in being effective in selecting models, in selecting directions
in which to to push your models.
Because at the end of the day, certain models are enormously powerful and helpful,
and many others will not be so for various reasons.
And this guides you, this analogy guides you. And I think for some of the neuroscientists
in the audience, it was also surprising to,
hear you talking about immune networks and the way that we talk about neural
networks as structures that have learning and memory, perhaps in quite a different way.
Way, and in some senses the mechanisms may be similar and other ones might be different.
But I guess that's one of the paths by which this research in immune networks
may come back to influence the neuroscience.
Yes, that is something that is not guaranteed to happen, but it is something
that I certainly think is worth thinking about.
I think in science generally it's very rare for one to make an interesting discovery,
for that discovery not to have
implications outside of the initial domain where the discovery was made.
And the main short-term lesson that I think neuroscience can take from this
immunological way of working is the possibility of having attractor neural networks
in which one is not recalling one attractor at a time,
an attractor which contains a very large number
of bits of information, one image at a time,
but where you see that there are multiple attractors which can be recalled independently
at the same time, even though you are looking at a system that is fully interacting,
not directly, but there are no subsystems that are disconnected from each other.
But if you then change parameters, all of a sudden you can get a transition
where all these subsystems are no longer acting independently.
Now, many of the things that you think about in neuroscience are somewhat reminiscent
of that, especially in making recognition on the basis of recognizing subparts of images.
Whether or not you combine them, or whether you want to create bigger objects
that are composed of things that you've seen before, but never in that combination.
May be relevant, may not be relevant, but certainly worth exploring.
These models operate in a slightly different regime in terms of parameters and
scales and numbers of bits of patterns and things like that.
One thing I find interesting here is also how you described the development
of these techniques. We start with spin glasses.
Those techniques made certain assumptions about symmetry and homogeneity of
the system. So those techniques would not scale to complex biological phenomena
like the immune system or nervous systems. For that, you need new techniques.
Would that actually help us to also distinguish living from non-living systems?
Because apparently, mathematically, there's a unique set of tools you need to
describe living systems as opposed to a pure physical system like a spin glass.
Would you go that far? Yeah, well, one thing I've learned over the years working
on biological systems is that when it comes to modeling, biology is much harder than physics.
And there are some really very clear reasons for that.
One very simple reason is that very often in biology, you look at systems in
which the number of elements involved is neither really small nor really big, the mesoscopic area.
And the problem with statistical methods from physics is they only work when
you have really large numbers.
In biology, that's not always the case. That's the first obvious thing.
The second thing is that many physical methods rely on the systems that study going to equilibrium.
Well, that's never happening in biology, at least not until we are permanently horizontal.
So that rules out about three quarters of the methodology of statistical physics.
But I think the main one, and that is also the most interesting one for a modeler
like like me, is that in biology, we always have heterogeneity.
We have heterogeneity in interactions between elements, heterogeneity in molecular structures.
And physics started out, at least statistical mechanics started out,
looking at systems in which every particle was interchangeable with every other particle.
Two electrons are exactly the same. The forces between particles would always be the same.
Physicists tended to look at pure systems, and they saw heterogeneous systems
as, say, impurities of pure systems, perturbations of pure systems.
Biology is not like that. Now, in the 70s, there was a whole area opened up
in statistical mechanics to look at heterogeneous systems. This is where the spin classes came in.
But one fundamental difference has remained, and that is, in using the physical
methods, you always have to be able to represent the heterogeneity as randomness.
You write a probability distribution for the particular parameters or forces
that are heterogeneous.
And that's exactly what is different in biology. Biological systems are heterogeneous,
but the heterogeneity is never random.
It has always been a consequence, at least partly, of evolution and selection.
And that rules out immediately all these methods. And the best example to think
about is probably the modeling of proteins.
Proteins are these large molecules assembled of little parts that are called amino acids.
And nature uses proteins as the building blocks of cells, as messengers.
It's the basic hardware of cellular biology.
If you look at randomly selected macromolecules of amino acids,
they behave completely differently from proteins.
That means that it is essential that, in order to understand how these biological
systems operate, you must study the evolution that gave rise to these structures.
But that is a very complicated thing, because it is a two-time-scale process
in which the processes on the slowest timescale influence the fast ones,
and the fast ones feed back to evolution.
You will never be able to write a heterogeneity in terms of a probability distribution.
That is really interesting because it means that here is a type of problem that
we cannot handle the way we would handle a heterogeneous physical system.
There are ways, there are openings, but it is a completely different field.
You mentioned this in your talk that you were continually discussing
with the biologists just about which aspects of the system they consider to
be important and in your models you have a noise term which i guess at the moment
captures a lot of that detail yeah and there may be aspects of that detail that
they think has to be pulled out and put into the model,
is is what you're saying that the mask gets hairier and hairier as you pull
more of those details out and put into the model or is is there some sort of
possibility of actually going into
that space and it's not going to become increasingly difficult.
There are going to be ways of making models that are more biologically rich,
sort of things that you can work with. You can still show interesting results.
I think there's no golden and perfect answer to this.
It's always the case that in the development of models, you have these three
distinct stages as I see them.
So stage one is where you have a problem from the real world and you have really
no idea how to model it in such a way that you can make progress.
Now, I think we are beyond that point in immunology now.
And the second stage is that there are some models that seem to make sense,
but they are really very simplified, extremely abstract, and one has sacrificed
a lot of the realism in order to get something that works.
That's where we are now. And the third stage is, as you say,
where you make these models more realistic.
And that means having a good intuition for knowing in which order to bring these ingredients in.
And there will be some ingredients that have a serious impact on the methodology
that you will need to still solve the models.
And there are some ingredients that are relatively easily incorporated.
To give an example in these immune models, the initial models of Adriano and Elena,
had the unwelcome element that a T-cell, a single T-clone, could be simultaneously
excitatory to some B-cell and inhibitory to another.
It's just like, and it's a nice analogy, the same thing was assumed in neural
network theory in the very beginning by Hopfield.
We know that the hardware is different. The hardware doesn't allow for that.
And in neural networks, we could remove this and we could still solve models. The same happened here.
So that happened to be a relatively easy ingredient to bring in.
There are other things that are going to be much harder. We can see that coming.
And that is when you include the evolution of antigen.
Because then you have to study in great detail the dynamical processes of hypermutation.
Mutation, because it is a rat race between the virus or the bacterium that has
come in and that will multiply, and the immune system trying to catch up.
So when you start working on those models, you can forget about equilibrium
studies, you can forget about energy functions, because these systems are no
longer going to equilibrium.
So you're going to move away from statistical mechanics to another modelling approach?
Well, one would move away from equilibrium statistical mechanics, which
is the part that has been used so far on these particular models
then you would use non-equilibrium statistical mechanics and
that is a slightly more specialist area
of statistical mechanics it is a small subset of the broader area most of statistical
mechanics but this non-equilibrium system mechanics has been developing by looking
at biology or it came from other domains of physics oh it came also from from
standard domains of physics,
because even if you know what the equilibrium behavior is, you might still be
interested in studying how the system would get there.
So they were developed already within the physics community,
but you just find that in biology, that will turn out to be your only toolbox,
because most biological systems don't go to equilibrium.
Okay, the analogy between neural systems, biological neural systems,
and immune immune systems is that they can learn and have memory, right?
So, in the nervous system, and there's also many talks on this,
the dominant view on that is that this memory has something to do with the connectivity
in the network and how it's regulated.
It might not be literally the
synapse, but it is the regulation of the synapse that's key for memory.
How should we think about memory now in this immune system? What's the substrate
of memory in the immune system?
Yeah, that has been a very tricky discussion in the biology community in the
sense that there are various competing views.
Some are fashionable in certain eras and then get ditched a couple of years
later, and that has happened in this community.
For instance, Jern's model of idiotic networks envisaged a mechanism of memory
that was very different from what most biologists now assume is the case.
Most biologists now assume that bee clones, after they've been activated,
after they've been producing antibodies, then go into a state where they become
so-called memory cells.
So somehow you want to retain and memorize the particular shapes of antibodies
that were very successful in attacking the antigen that came in.
So the thinking is that those B cells that produce those antibodies,
they convert into a state where they can live for 25 years. I find that intuitively slightly.
Not maybe unrealistic, but slightly far-fetched. Is there any empirical support for that idea?
Well, I'm not an immune biologist, and I'm sure there is some support.
But I would be surprised if that was a clear-cut and completely closed case.
I think there is still room for alternative. But that would then predict that
there is some reservoir of these memory B-cells that reside somewhere in the
body. Yes, that's the problem.
That's expanding, right? That's called ditty expanding. Well,
the thinking is they don't expand.
The thinking is that you have this group of B cells that have been activated,
and they go and sit somewhere, and they live 30 years, just in case at some
point that particular virus reappears.
I don't buy that. But I cannot argue that there is evidence to reject this.
It just seems a bit far-fetched to me. Okay, but it means for you,
this means the memory problem for immune systems is still not resolved.
If we don't actually really know?
I think there will be an explanation which is a bit more natural and elegant than the memory cell.
So your modeling approach, would that lend itself to building models that contest
the alternative hypothesis?
I think so. Yes, I think so. I think so. But at this moment,
we're not there yet because the models are relatively primitive.
I mean, compared to, if you look at, that's why I find the history of neural
networks here is so interesting because we've gone through many of those things.
And you can look back and see, yeah, it was very similar then.
Hopfield models were primitive, really very primitive. Lutz was wrong with them.
And the interesting thing was, at that time even, Hopfield models were still
solved using equilibrium statistical mechanics, and in these models,
one sacrificed biological realism in order to be able to do that.
And when you talked to people in those days, it was very often suggested,
well, we will never be able to do the dynamics, so this is why we do it that way.
Turns out to be not true. People are sometimes too pessimistic about what is
possible, and that is very dangerous, because you miss opportunity.
Turned out to be nonsense.
Doing the dynamics was, in retrospect, relatively easy, which was very good
because then we could remove all these biologically unrealistic model ingredients.
I expect the same to happen here.
But the other thing is that for memory, it's mysterious.
In some sense, we could argue memory is equally mysterious for the brain,
but there is consensus on what that substrate would be, more or less.
But now let's look at the learning component, right?
So for the immune system, you would have hypermutation and then selection upon
this set of hypermutated B cells as your core learning mechanism, right?
And now in the brain, yes, people like Jerry Edelman have proposed that similar
mechanisms would work in the brain to select synapses and in that way have specificity
of connectivity and learning.
But it's not the only way in which we can think about learning in the nervous system.
It might also just be the differential strengthening of synapses that allow
you, as in a Hopfield network, right, to sort of bias the network towards certain
responses away from other responses without necessarily selecting across a repertoire.
You remove certain possibilities completely. Completely.
So do you believe that selection is then the only mode of learning in the immune network?
Or is there actually more subtle than that?
It's an interesting question. I think that over the years we find that a lot
more is going on than we presently think.
I would also, for instance, expect that at some point there is some interface
between the nervous system and the immune system. And even today I had some
very nice discussions with some of the people here where these ideas came up too.
If the nervous system, for instance, would be able to predict or anticipate
certain invasions of antigen, then you can prepare the immune system.
I think there's a lot going to be found that we don't know present.
Well, in that sense, for instance, if you look at sepsis as an inflammatory
problem, it's extremely interesting to look at.
If you look at these broad organ systems that are affected in sepsis by inflammation,
it's a rather complex dynamics in which, let's say, the inflammatory load to
organs can be very specifically changing or transiently changing during such a disease process.
And actually, also many patients who suffer from sepsis, some of them might
appear in all standard biomarkers healthy.
They're sent home, and a few months later, they still die, right?
So that would also suggest there's much more active regulation going on of the global response,
response yeah with the possible anticipatory component to
the immune response that might or
might not reside in the immune system itself so
how do you see that coupling exactly between the brain and
the immune system do you expect the brain to really model let's say the health
status of organ systems and then bias in some way the immune response and if
so what's the substrate of that you have to think really about these things
in in terms of the coding that the different system use.
Brains think in terms of electric potentials and the immune system thinks in terms of chemistry.
It communicates by docking and sending cytokines and things like that.
And there are certain constraints if you think about possible mechanisms and possible interfaces.
For instance, immune cells cannot pass the blood-brain barrier,
so they cannot directly enter the brain.
However, various molecules that they produce could.
We also know that in the brain, as I understand it, there is an interface between
communication by spikes and chemistry.
The release of all kinds of chemicals happens also in the brain.
So I don't think it's far-fetched. I don't think it's far-fetched.
There are all kinds of opportunities, but we will never be able to find out
until we have at least a satisfactory description of the immune system itself.
There's so much we still don't know about the immune system.
Going to an interface now is premature.
But if I were a betting person, I would put my money on there being one.
It just seems to be giving you an enormous evolutionary advantage.
And therefore I think it will be there.
So the algorithms that are involved in learning, presumably,
I mean, you didn't talk about them in any detail in the talk,
but there's something analogous happening there to natural selection,
and is it involving both the B-cell and the T-cell populations in some interesting
ways that alters the population dynamics so as to give you some form of optimal search?
Yeah, so it's interesting if you look at the literature on immunology that most
studies that have been carried out tend to have been carried out on B-cells
and on evolution of receptors of B-cells.
A lot is known about them, much less is known about T-cells.
And partially this is due to the fact that T-cells were only discovered later.
The regulatory T-cells were only discovered in the 1990s.
So there is a good reason why we know much more about B-cells than we know about T-cells.
When you think about learning, and this is This is again where I find these
models very helpful, the models
that create this mapping between neural network and the immune system.
If you look at those models, you see where is the learning residing,
what is the substrate for the learning? It is in the cytokine variable.
So if you think about what they do, the cytokine variables simply tell you exactly
which B clone talks to which T clone, and within these models,
it is also in the other direction.
And that is exactly what is being changed as a result of hypermutation and selection,
when we go through that process.
Because if we change the receptor properties of the Bs and the Ts,
then we also change who talks to whom.
So we're effectively changing the patterns that, in the language of neural networks,
would be the patterns stored in a Hopfian model. So we're rewiring the network?
We are rewiring the network. At the same time as we're changing the specificity of the cells? else.
So in a way, the network is a consequence of the information being stored.
And since we changed the information being stored, we're effectively rewiring the network.
So we could see the cytokine as a virtual synapse now that connects certain... No, not the synapse.
The synapse is expressed in terms of the cytokines.
So the cytokines would be... They're the connection.
They form the connection between the B's and the T's. They are the equivalent
thing of the bits in the patterns of the Hopfield model.
Exactly. And they build the synapse.
So what So what's now the capacity of then such a memory?
If the cytokines would then, in this, the analogy we're now pursuing,
the cytokines are in that your connectivity matrix, right?
So given the scale of the immune system and the number of elements,
what kind of capacity would we then look at in these cytokines?
And what kind of learning rates would you expect or the rate of change that you can support?
Yeah, this we can now answer as a result of the models. And the answer is, in fact, very trivial.
This is always when you have understood something you
know that you could have thought about it before but in standard
hopfield models what you know is that you can store if you
want to have perfect recall only relatively small number of patterns but each
of these patterns has an extensive number of bits here is the other way around
you you can store and recall an extensive number of patterns proportions to
a number of b clones but each of them will have only a finite that number of
bits of information in that.
So the capacity is in the order of n, and precise numbers you find by calculating
the phase diagrams, which has been possible now.
You can compute these things. Right. But then another kind of connectivity matrix
might be defined by the T helper cells, for instance, that would modulate the
interaction between the Ts and the Bs.
For the nervous.
Just focus the whole learning problem towards the synapses keeps things relatively
controllable, understandable.
But maybe in the immune system, there might be multiple levels at which this
plasticity now gets expressed or multiple substrates or not.
Are you happy with just focusing it fully for now at this cytokine modulation?
I think for now, this is what is being done simply because we are already very
happy with what we can achieve with this model.
But in the future we
will want to study in more detail the actual process
of what you would say learning in neural networks what we
would call hypermutation and thereby the changing
of all these cytokine variables in the immune system and
then we will undoubtedly find that things get more
complicated we will have to look at different types
of p cells different types of t cells and all those
things so you will inevitably get a
next layer of complexity simply because you move away
from what is currently one of the most primitive starting
points imaginable so that
will happen that will happen but um i
think on top of that we will find and that is going to be done by experimentalists
we will probably find new players that we haven't even been thinking of not
just b cells or t cells but other cells that do things we might find interactions
mechanisms Mechanisms that we don't know now.
But now think about the signaling. So in the nervous system,
in the end, it's electrical, as we discussed.
In the immune system, it could be through the metabolites, molecules that are
going through the circulatory systems.
What are the forms of signaling that now the immune system has available that
would make it different from the kind of signaling you would see in a nervous system?
Um let me think about that one so in the nervous system i think there's one,
complication that maybe is not so much a complication in the immune system and
that is in the nervous system we also have topology we have wiring and at least
in the immune system we don't have that because these cells just move around
in the blood the assumption is that
there are no barriers that single out certain B-cells as being having given
access to certain areas as opposed to others.
There are some aspects only which are that the hyposomatic mutation takes place
in certain parts of the body.
But apart from that, we have no topology and that makes your life a lot easier.
So in terms of the signaling, there is the extra constraint in neural network
systems that you have to look carefully at the wiring.
These things cannot float around, neurons cannot float around and establish
and interact with whoever they want.
And space is a tricky thing, and especially if you think about mathematical methods from physics.
In one area, and that is space, they are remarkably unsuccessful.
And that is when you look, for instance, at a very simple cubic lattice of molecules
that only talk to their neighbors, that is still an unsolved problem.
Even if these variables are just binary, they can be on or off,
and they talk to their neighbors in a three-dimensional arrangement,
it's an unsolved problem so space brings an
extra dimension that makes problems really
significantly harder and that is
absent in the immune system so that is a difference which actually works
to the advantage of modeling in the immune system and
in brains we can get away with it we have been able to
get away with it for a long time because at least neurons
talk to a great number of other neurons if neurons
had only been talking to neighboring neurons we would
be in real trouble and thank god we're
not right so then
but how can we now test these models because um also look at the case of neuroscience
right there was a lot of enthusiasm for the hopfield kind of networks because
people had this this hope like okay now it becomes mathematically tractable
neuroscience can become like physics we can have analytic solutions god knows what,
but that was 85 more or less, right? So we're talking about 30 years.
And has that impact been so overwhelming? Are we really now radically,
have we radically changed the way which we think about the brain?
Not so sure about that, right? So on the other hand, it has been very,
as a heuristic, it has helped people to pose certain questions.
So now in something you can argue for the immune system, you stand now at the
beginning of an equivalent process, right?
Where it is this fundamental insight using these techniques from theoretical
physics has helped you to get the handle on a system that you don't understand.
But what's the trajectory that you now envision? What are the next steps to
also make it empirically testable?
Yeah, so we are very aware of the danger of the same thing happening in immune
modeling and what happened in neural networks.
And we can come back to what I think are the reasons for that because some of
them are not scientific but psychological.
Now at the moment what we're trying to do is we're trying to,
in a particular multidisciplinary project that we've started in the UK involving
not just people like me who do models but also bioinformaticians and experimental immunologists,
we're trying to find indeed models in which we can test the predictions of the
model using experiments.
So we would do this in the form of having a model in which an antigen comes
in that has never been seen before, and then predicting the dynamical response, first and secondary.
And see to what extent this is similar to what we get in what is called unchallenged
individuals in an immunological response.
And then try to build the models from there, test every time that we can make a prediction.
At the end of the day, the only real test of whether you have extracted knowledge
in any area of modeling is whether you can predict unseen situations with the
model and and then test them. If you can't, you haven't learned anything.
If you can, you have. And what's the status of the measuring techniques that
the immunologists have to detect this sort of...
I mean, for your models, you're looking for quite a lot of detail in terms of
the temporal and spatial dynamics, or certainly temporal dynamics of what's unfolding.
So this is really difficult, in the sense that it is very hard,
and we do struggle with that, to find people with
the right type of data because you're looking for individuals
who are who've never
experienced challenged by a particular antigen before so
you have to go for really rare diseases you cannot go for
flus or things like that because there is almost no one who will have a primary
response to that so you have to go for very funny things like yellow fever and
And our collaborators are trying to get these people from populations of individuals
who are about to make a trip abroad to an area where they've never been,
and who are then being inoculated beforehand.
And these are the ones that we try to get the data from, because then they get
a response to this inoculation that should be very similar to primary response.
Response, because then at least we can do tests before having modeled the secondary
response, before having modeled the consequences of the hypermutation.
But why no animal models?
Well, why no animal models? It just happens to be that the collaborators we
have don't work with animals.
They work with people, because they come from a clinical environment where it's
human immunology they work on. And can you do some of this in a sort of petri
dish where you just put some antigen into a sample of blood?
I've been told that this is really hard. Right, okay.
That it is one of the tricky aspects of experimental immunology.
Look, here we have all these Brits, they're going on holidays to faraway exotic
places where they're going to run the risk to catch all sorts of strange diseases
and and they get inoculated, and imagine this whole thing now works,
what's the specific prediction you would like to see tested first?
We would like to see the evolution in time of the distribution of clone sizes.
So if immune system is not challenged, so let's say before these people get
the inoculation, the theory can do a prediction already on the distribution
of clone sizes in normal state.
Then comes a challenge. This challenge has a consequence, and you see this in
the distribution of clone sizes.
And the distribution of clone sizes is a typical thing that can easily be measured.
It but this is the first test it doesn't mean that this is the perfect test
just means this is a very clear transparent test and if it doesn't even pass
that test then you know you have missed something important in your model you
go back to the drawing board and you think what that could be,
but this is relatively accessible data and this is what we're focusing on so
it's a distribution of the sizes of the clones but it's like fine-tuning the
model is the model falsifiable
so what would be the observation in this experiment that
would completely kill the model that you know
proposed with your colleagues well i don't
know what the definition of completely kill is because you can always go back
to the drawing board well that's the question right yeah exactly so but it this
is a falsifiable model in the sense that you could find a clone size distribution
which is qualitatively very different from what the model predicts and it's
not something that you could fix with matching parameters and things like that because.
So it could work it could not work it is a falsifiable thing but but.
The tricky area is the area in the middle, where it works to some extent.
It is not complete nonsense that comes out. It is also not perfect.
This is the general situation we find ourselves in, in modeling biological systems.
So then you try to systematically think about the ingredients without just putting
your modeling hat on, but also in talking to your experimental colleagues,
because things that you do have to make sense, see what could be responsible
for the deviation between the immune experiment.
This is the tedious cycle that we have to go through, whether in physics,
biology, any discipline.
That's correct. But now, so here we have this model.
Right now, we're in a phase of the science that we actually try to understand
how this system works at all, right? This is what you were clearing in your talk.
There's a lot of aspects of this immune system and its responses that we just don't understand.
But the next step would be to control this immune system. So imagine we move
to this phase, now you have the model, we understand how it works,
understanding would imply we can control it.
So what kind of control would you envision we should pursue there?
Well, I have a very simple but clear vision on that.
I don't know whether it's realistic, but this is what I'm heading for.
And that is in the immune system there's
a very clear division of objects that
immune cells can sense being categorized either
as self or non-self as enemy or
friend and we know one thing and
that is the immune system is capable of changing
its mind from one day to the next and this happens certain diseases when all
of a sudden finds that the immune system starts attacking a particular type
of cell cell and this is not desirable but we know that the immune system can
do it what we do not know is what actually made that change how came that change about.
So we first need to understand this. We need to understand what is happening
when someone develops alopecia, where all of a sudden your hair fall out.
Because it's a mechanism that is possible, but it's a mechanism that we would
like to exploit and control.
Because if I can manipulate this, if I can tell the immune system which objects
are from today onwards to be seen as enemy,
which until now was seen as friend, then we can
redirect the immune system to attack anything we like
and this is what i see as as really the
future of medical applications of the immune system and
this can be um especially relevant in
cancer medicine where there
are different levels of ambition one can have so
one level of ambition i think is a bit too far for now and that is to
train the immune system to see the difference between a healthy
cell and a cancerous cell of a given organ
but we don't need to do that we can also lower
our ambitions and say look there are certain organs that a person can live without
thyroid is one example if we could at least instruct the immune system to regard
all thyroid cells as enemies then you could treat a patient first with surgery remove the thyroid,
and then you switch your immune system to attacking all remaining residual thyroid cells.
And the difference between that and the normal route, which is chemotherapy,
is that if you do it right, you will not have any off-target pathologies that
you create, any damage that you create.
And it is an active agent. It is not a passive chemical.
It is an active agent that will seek out anything that looks like a thyroid cell.
So I can see this as the future. But isn't it, there's some molecular stuff
going on there which is not captured in your model yet, presumably.
No, these models are far from that stage. But I'm now looking ahead.
This is one of the things that could become possible once we have an acceptable
level of description of the immune system.
Because I think we have to be extremely careful at the same time.
Simply because the immune system is so powerful, and we've seen that in some
trials that went dreadfully wrong. on, tinker with it the wrong way,
you can do tremendous damage too.
And it occurred to me that that's another use for your model because you can
test interventions in the modeling environment before you try them.
Because you explained how this particular disastrous trial started out with
the successful animal models.
So the animal models don't always predict what will happen in a human.
But maybe a computational model will show you a variety of possible results
and you you could then look out for bad outcomes that you might not have seen in the animal.
Yeah, a colleague of mine from Brazil, from Berlin, once described it in the
following way, which I think is a very good way of seeing it.
If we build a new car and we crash test it, we don't run Mercedes against walls.
We do it on a computer first. That's why we do the crash test.
In medicine, we're not often doing that, sometimes because we don't have the
computational ability or we don't have the knowledge to do it.
But I think that is the future.
Exactly as you say, you have a decent model and you have to be careful that
you test your model against experimental data, but once you get to the stage
where you trust your model to some extent, see if it exists.
Then you can crash test ideas. And in the model, you can make those changes
that you think on the basis of the theory would be the changes required,
and then test whether it actually works like that or does have perhaps unforeseen
consequences that are dangerous to the patient.
I think it's more than crash testing as well. I mean, the driverless car research
is now moving into simulation because they can't capture enough real-world data
of unlikely things that will happen when you have millions of driverless cars.
So you want to simulate those situations.
But there are already attempts underway, as you also mentioned,
also failed attempts, to try to really master and control the immune system
for more personalized healthcare. Yes.
So to what extent do you see these current steps as being consistent with the
concept of the immune system that you are advancing with this model um i think
it is going in the right direction in in early.
Immunological therapy ideas very often
what was done was non-cell specific interventions in the immune system which
you can sort of see as similar to just rebooting it giving it a big kick and
then hope that it settles into a new state which is doing what it was designed
to do and more recently differently,
especially with this example of these CAR T cells,
the research has moved away in the direction of making genetic modifications
that are not generic, but they are tailored to the specific needs of an individual patient.
And in the case of CAR T cells, they create these receptors to be specifically
tailored to recognizing and docking to the cancer cell.
Now that is moving already in the right direction. It's the only thing is it's
fairly limited because this only is done for cytotoxic T cells.
So this doesn't exploit at all the interaction, the network concept,
the interaction between the T clones and the B clones.
And in that sense, it is still very different. It's not this idea that I have
of controlling the self non-self boundary.
That's not what's happening, but at least it
is moving in the the right direction it is trying to make non-generic changes
but it's trying to make changes to components that are really tailored to the
specific needs of an individual and they are spectacular some of these results
it isn't doing everything we would want these things to do the CAR T cells don't
work for solid tumors at the moment.
There's a lot of research going on in seeing what is the
reason why it doesn't work there what can we change and there
have been already several generations compared to the first generation
of these methods they've come a long
way and that's why they've now been uh approved by the various authorities right
but why why would this work on by manipulating the t cells and not the b cells
because you could imagine that the b cells that was labeling um whatever object
you would like to classify as non-self.
So can you explain from the perspective of your model why it has been effective
on the T cells and not the B cells?
That's a very tricky question because I don't really know the answer to that.
And I can only speculate as to why that may be.
But given the model, you must have an intuition, no?
Well, let's approach it from the other side. Would it be a viable alternative to do that?
I think it would be. But I need to look very carefully at the differences,
is if any between the response of the immune system and the way it does its business in,
attacking foreign antigen compared to how it deals with deranged own cells and
i would not be surprised if let's say the players responsible for the immune
response would not be the same,
because b cells are being trained very specifically in the germinal center to
look away from the cell cells, to really look only at foreign antigen.
So it may well be that the way the immune system handles cancerous cells is
different, distinct from the way it is most effective in handling foreign antigen.
There must be studies looking at that, but I'm not aware of them at the moment.
But it is a very good point.
So you move from surgical physics to neural networks to the immune system.
Do you in any way miss the study of neural networks and the brain,
or not that much? Do you find this more challenging?
That's an interesting question again. and I've had
a really good time studying neural networks but I
also find very often that when I've worked
on something for 10 years or more I tend
to look for something different and it
can be that different in terms of the modeling
and the analysis can be found within
the same domain main or outside and
and what i've seen at the time is that i
found all kinds of new things to work on which happened not to be neural networks
but now just by accident i'm moving back and i'm touching that field again simply
because the technology leads me there um i've always enjoyed I thought it was
a fascinating area to work in,
but at some point I got the impression that in this community it became an inappropriate
norm that one would have to know everything about biology,
everything about the computer science aspects, everything about the theory,
in order to be effective. I believe that's a fallacy.
And I think it has done some damage to that field and it has sort of pushed communities apart.
I find these things very interesting because I've now, about 10 years ago,
moved into what I call quantitative medicine, did a lot of work on protein interaction
networks, gene regulation.
And I've asked myself, how can we avoid that in the future?
So what we've done is we've set up a mechanism where young people are given
a program, a training program, in which they are being taught by people from
different disciplines, at a level that they can understand.
And we try to educate them to work differently, to not believe that in working
in an interdisciplinary team, one has to know everything about all the disciplines
because one ends up being mediocre at everything.
But they have to identify what they are in terms of their primary discipline
and make sure that they are really on the ball in that.
But they need to speak the jargon of the people at the other side of the various
fences, experiences, and they need to know what keeps them awake at night.
And that's the only crucial thing. But they should not make the mistake to think
that they have to be experts at three different disciplines.
It's not realistic. It's counterproductive. I think at the time,
but maybe I'm wrong because you know that community better than I do,
there was a bit of that happening in between the 1990s and 2000s.
It's still like that. Yeah. Yeah, yeah. I mean, multidisciplinarity is clearly
still a challenge also in the field of neuroscience,
where there's from the side of the people doing the experimental work,
which is extremely intense.
Of course, they would feel like, well, these guys haven't made the miles to
really appreciate the complexities of what we're doing here.
They will not really appreciate the meaning of this kind of data.
And the modelers or the theoreticians would often have the feeling like,
well, we're not being listened to, right? So, there's this massive disconnect.
And that's also why I think in neuroscience, theory overall has made less of
an impact than you might have wished.
But now, so, okay, given this long trajectory you've made, what should be Ton's
law if we want to understand the immune system, the brain, living systems? So, what's Ton's law?
Well, I don't think I have one law. I've learned a couple of things,
and I can go through them if you want. The first is interdisciplinary interactions.
What i just explained to you and i think where we should move to is a is a way of working based on,
respect of expertise on the other side right if i work with an experimentalist
a bioinformatician i have to trust them to know what they do and i i leave them
to it what i do need to know is,
what is it they really find interesting but what i
should not try to do is to convince them what they should find interesting
is what i find interesting yeah so these
are the types of of psychological things i've learned that means empathy not
not that empathy i just need to know what what excites you
paul i need to know what what is it that makes you really
happy you need to know what makes me happy and we have to accept that these are
different things but they could all be found in the same project but they can
all be found in the same project and that's how we can work together best and
i don't and i should not try to do someone else's work or even attempt to know
what they know this is completely counterproductive so that's that's that's one of the first things.
Generally, I think we live in a bit of a strange time in terms of how science
is organized generally, and I think there will come a point where there will be major revisions.
One of the crucial lessons is never sacrifice quality. Everything else you can
sacrifice, but never sacrifice the quality of what you do.
Be your own worst referee yeah because fashions
will come and go reputations will come and go but you want your name to be a
sign of quality the only way to protect that is to never sacrifice quality and
if you then perhaps are six months later in submission of a paper and it doesn't
end up in nature who Who cares?
But maybe in 15 years from now, they will still read that paper.
And all the nature papers will have been seen and forgotten about in six months.
Quality is everything in science. That should never be sacrificed.
Okay. Another loss you want to add to that or two is enough for now?
No, I drink lots of coffee, Paul. Okay, good.
So to finish up, so we're going to send Tony from Sheffield to London five years from now.
Okay. Okay, given the current political situation, let's just assume it's London,
it might be somewhere else.
You never know, with Brexit hanging in the air.
But okay, there comes Tony, he visits you at King's College to check whether
a specific hypothesis was falsified or verified.
And that is the hypothesis that you're going to specify today.
So what's the one hypothesis that you really would like to see tested in this five-year frame?
That it is possible to instruct the immune system to change which object it
sees as self and it sees as non-self.
I think that would be a big, big step forward if we can achieve that and actually
have experimental validation for it.
Great. Tom Cullen, thank you very much for this conversation.
Thank you. Thank you very much.
The CSN Podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
For more interviews, recorded lectures or upcoming conferences in the field
of biometrics and biohybrid systems, go to csnnetwork.eu.
Music.