Okay, so I'm here with John Doyle. We're still both at the KITP for the workshop
on network architecture.
This is Paul Fouchure, the Convergence Science Network.
Work and john you you presented your work on on architectures as a control engineer,
you looked at at architectures in
particular focusing on single cellular organisms right
so so and that seems a bit of counterintuitive move
for a control engineer but why why did
you move to the to biology well i
got interested in biology you know around 10 years ago or
10 or 15 years ago um i'd been
working initially motivated i'm pretty much
a very much at the theoretical end my phd is in math but i
was early mostly motivated by aerospace control problems because they were doing
in you know the 80s the most cutting edge things but then in the 90s networks
started coming you know we we the internet really grew to prominence um didn't
really have a theory so i got interested more in networks and uh just kind of gravitated towards,
um bacteria because i had uh colleagues at caltech who worked on it it was kind
of a convenient um but also um i've found that they have in nature one of the most remarkable.
And evolvable architectures the architecture of
the bacterial biosphere um for one thing it evolved into us but it also continues
to have tremendous evolvability and robustness and so i want to understand what
gave that biosphere that robustness and i think we've now found.
Deep principles that are very consistent with what we're learning about how
to build technology networks so it was examples in addition to the internet
of really sophisticated
evolvable networks that we
could compare but now before you go to internet and talk about robustness,
What's what are these principles of robustness that?
That you see these biological systems. Well so one thing is they are.
They get robustness by having a plug-and-play, sort of what we in technology
call a plug-and-play architecture.
That is, on almost every timescale you look, they can quickly adjust to,
say, changing environments.
And so on very short timescales, they can very quickly rearrange their protein
networks to respond as they needed. But on longer time scales,
they can change their genomes very rapidly.
They can swap genes.
So the bacterial biosphere as a whole almost acts as one large gene pool.
So on long time scales, they can very quickly evolve their genome.
And on short time scales, they're very quick in responding to the environment.
And what's interesting is I think their architecture makes both of those better.
And that's one of the things that I think has surprised the biologists a bit
is that the proper architecture can facilitate both rapid robustness on short
time scales and rapid evolvability on long time scales.
Else but now in your so
this is a bit the phenomena right these are
the phenomena we want to understand but not in in terms of principles you you
talked about layering you talked about constraints um and you talk about let's
say feedback so so how does this now relate to these phenomena that we're trying
to look at right well the way we the way in engineering Engineering,
we think about, and mathematics,
the way we think about describing architecture is in terms of constraints.
And the IEEE Transactions paper on architecture takes that point of view,
as does the recent PNAS paper aimed at neuroscientists, and it tries to explain
how we think about constraints.
And one of the place constraints come from is the behavior of the system as
whole is constrained by the environment it must act in and if it's going to survive and persist.
And then there's usually constraints on the components which come up from the bottom.
What can you build? What are the resources? What can you build it out of?
And then additionally, there are constraints that come out of theory in some sense.
I mean, theory reflects reality. But the idea is that much of engineering theory
is about the additional hard limits that come either out of limitations on computing,
which is Turing, limitations on communication, which is Shannon.
And what I talked about in this KTP lectures is the Bode limits on robustness and feedback systems.
These aren't well known in the scientific community.
And they're a little bit fragmented even within engineering.
But we're trying to build unified theories now.
But the final constraint then is the design choices that are made,
either through evolution or through deliberate design.
And Gerhardt and Kirshner, who are biologists, have a very nice phrase for what
a good architecture is, is constraints that deconstrain.
And you choose a few constraints, and it might be, for example,
the use of ATP as an energy carrier, the use of NADH as a redox carrier, the codons.
In the internet, it would be the packet formats and the TCPIP protocol.
Those are chosen wisely. Then they free you up to use that as a platform.
For both very robust networks and very evolvable networks and that's
where layering comes in is because layering is seems to
be the strategy that both engineers and biology have adopted to make flexible
uh robust evolvable systems and what layering is is a particular way of structuring
the chosen constraints and so that's what we talked a lot about it's not a simple topic
but um it's familiar enough
to people from neuroscience and from cell biology and
from engineering that there is becoming
i'm hoping a common language which is what i'm trying to promote about
what is layering but then how do
layers map to modules right because also in
your analysis you you stress the notion of
modules quite a bit yeah so how should i see this
is related to layers and modules so layering is
i think the the highest level view of what modularity is and i think it's the
most uh important aspect of modularity um i think there's been a lot written
in the scientific community lately about modularity that is i think
naive and not wrong so much as not the most important aspects of Larry.
And so the thing that people are probably most familiar with modularity is the laptop that they have.
And they can, or they download this podcast and it just runs immediately.
And so the idea is that the kind of modularity it has is that you can download
new software, you can download new podcasts and they immediately work largely
independent of which computer you happen to have.
Um, and you can, you can go to the store, uh, order online, uh,
hardware that you just plug in.
And what makes that possible is what's hidden,
what you don't see, which is the operating systems and that,
that they sit between all these applications and the hardware and,
uh, and allow you the modularity.
And the idea is that the modularity looks so simple and so easy when it works.
It usually either works perfectly or not at all.
But it's the layered structure that allows that to happen.
And we believe that's the same thing in biology. It's what allows for horizontal
gene transfer as a mechanism for bacteria to evolve rapidly, for example.
And that so they have the same sort of
plug and play modularity that our technologies
have when they're good but now does modularity always
imply let's say a static organization of
of a structure or can you
have let's say virtual modules that change if
you want their boundaries and their interfaces to other modules so
how could i think about this well i think i think
the constraints a deconstraint way of looking at
this is that there's always different time
scales for different aspects of the architecture there's there's usually
things that are persistent on very long time scale so an example of something
that is persistent on very long time scales in biology would be the use of atp
as an energy carrier it's both universal across all of biology and probably
has been there for billions of years um the the uh the codon usage is nearly universal and has
probably been there for also similar long times.
So those are constraints that really persist for a long, long time.
What they facilitate, however, is tremendously dynamic responses.
And so by having a common energy carrier throughout the cell.
Individual modules don't have to worry about an energy source.
Core metabolism make sure that that
the atp charge is maintained as constant as
possible and an enormous amount
of sophisticated control goes on inside the cell or inside our bodies to maintain
that energy charge um and then what that allows you to do is on again on every
time scale adapt very dynamically to the challenges so what you see is the combination of a
permanent constraint that's fixed forever.
But that choice, if done wisely, enables very, very much dynamics.
Now what happens is on fast time scales, even what's a module is changing very rapidly.
So for example, just in say in metabolism, you'll the depending on what,
what biosynthetic pathways are needed, we may come and go on demand.
Um and and what
what and depending again on what's in the environment what
was a food source may go away
and now all of a sudden there that's a say an amino acid that
you have to start manufacturing again this is bacteria are the
most plastic in this regard we're not very flexible right but
at the same time we can eat all sorts of things um and so we're
omnivores and our ability to do that is
again because we have a
sort of front-end process that turns
whatever we eat into a few basic building blocks from
which we synthesize all of our tissues okay but now if you
if you describe it like this coming from engineering you
seem to be implying that actually engineering also
works according to these principles yeah is it actually true
for for sort of the larger systems
that we have put together as humans well one.
Of the problems is the larger systems we put together as humans
are not sustainable and so they
have design flaws and one of the ironies is uh
when i first started getting interested in biology the biologists
would tell me oh you know you're going to see biology's kludgy it's
an accident it's a tinker evolution's the tinkerer um that's
very true but in fact you don't
you don't tend to see uh the gross
design flaws in biology that are all over in engineering and the reason for
that is that that much of the core uh these core protocols in biology have had
millions to billions of years to evolve um and And sure,
we have some funny aspects of because we were built out of fish parts.
And so building a human out of fish parts is going to have some problems.
But the the biochemical layer.
It seems to be spectacularly uh well designed from an engineering point of view,
so i think and so what the what you what's hard to do with any of these systems
is to sort out what is a hard necessary constraint and trade-off and and what is a,
an accidental design flaw either of evolution or of engineering and then you
see both um so the problem with engineering though is in our at the big scale
we're profoundly unsustainable,
In our energy, transportation, waste, water, food, almost everything.
And the more you learn about it, the more disturbing it is.
The more I understand ecosystems and their interactions with our technologies,
the more frightened I am.
I think I'm, funny, I'm kind of a global warming skeptic. I think it's much
worse than the scientists are telling us.
I don't think they have the tools yet to do the sort of worst case scenario
analysis that we do in engineering.
And we take for granted that our planes rarely crash, and that's because engineers
do that kind of worst-case analysis.
So we do certain things very, very well in engineering. We've gotten to the
point where we can build platforms like airplanes and vehicles that are very
sophisticated and very robust, but we're not so good at the networks yet.
And the internet was a spectacular innovation, but it now has a lot of recognizable design flaws,
but we've built so much on top of it
that it's hard to now change
those if you make a if you make a mistake in
a core protocol and then you build a
lot on top of it it can be very hard to now change that right so if atp wasn't
a good energy carrier biology would have very little choice but to just stick
with it yeah exactly so but now so so what but this seems to be also a conundrum
a bit in your, well, these ladies are getting a bit noisy, right?
So maybe we should change location.
Because I have to filter all this stuff out later again. Keep on going.
Because what I want to bring up now is that in your, so the one that now we
see, if you want, a bit of a contrast here, right, between your analysis of
life, of biology, biological systems, Thank you.
And the success of this framework you see in nature, in actual development of
these large integrated systems like internet, right?
Because like you said earlier, if you look at how internet has been put together,
it's one big kludge, right? It's not really.
But it was based on a few very profound and visionary insights.
And it's not necessarily from from control engineering
in any way no no in fact it was operating systems
uh and engineers right and um and
they now they knew elementary
control theory and made quite a bit of use of
it at a time and they did it at a
time when uh communication theory was almost
entirely dominated by information theory and
so decades ago there was a major battle between sort of the information theorists
and the operating systems people over how to build these networks and and for
almost accidental reasons in the u.s.
The OS system people won and they built the internet and now what's happened
in the last 10 to 15 years as we really now have a much more integrated theory
that includes information and control theory and And greatly explains both the
successes and the flaws in the current Internet architecture.
And one of the big challenges now is to use those insights to build a next generation.
But the Internet's a good example of a situation where the really brilliant
engineers are always way out ahead of any of the theoretical understanding.
And so we're always playing catch up with the best.
It's the same way, I think, in medicine. And I think the best doctors are always
making leaps way out ahead of any scientific basis.
And then, and I think the scientific community in some ways quit doing this,
is trying to come back and help the doctors flesh out the details.
I think the doctors have, for a few decades, really been on their own as the
scientific community pursued genomics as the answer to everything.
Thing right but the only problem is of course that with respect
to internet we might be already so much committed into
one certain way of structuring the system that we have no chance ever to to
re-engineer or redeploy anything that's a big worry yeah that's a big worry
and so um i'm a little less concerned about that than i was from a technical
point of view in the sense that i think.
The architectures that I think are the most appealing that are being proposed
actually have the internet as kind of a lousy special case.
So I think it could be organically grown around a new architecture and then
it's a little bit like a city.
You could sort of leave the old parts as a kind of scruffy old neighborhood.
I think that's possible. I think the challenge is going to be probably in these
big systems, not technological, but the interplay between technology, markets, and policy.
And I think if there's a system that we understand most poorly,
it's the interplay between technology and policy and markets.
And right now, extremely ideologically driven, not the least bit rigorous.
And I would say of the fields that I've looked at, I think the area of economics,
and finance and markets in that world is among the most ideological driven of any.
And so the problems ultimately there may not be just purely technological. Right.
But now to get back to the first architecture, right?
So in some of those Also in the discussions here in this workshop.
We want to get to some notion of, let's say, brain architecture.
But then let's first try to understand what we mean with architecture.
And you have used this analogy with, let's say, a garment to illustrate how
you think about architecture.
Could you maybe explain? Yeah, I've been trying to look for an example of an
architecture that would be so simple you could explain it to a school child, right?
Because I think we need these concrete metaphors. And of course,
anything that simple is dangerous because it can't have many of the important
features that we care about. But the reason I picked on clothing is everybody knows about clothing.
And the thing about clothing is that it shows two very different kind of layers.
And the PNAS paper that's now online and soon to be in print discusses that.
And the idea is that the most familiar one is sort of inner to outer wear.
So you have an inner layer that's
close to the skin. And I'm imagining clothing in a cold environment.
We're naturally adapted to live in a very hot environment and don't really even
need clothing in a hot environment.
But where we really need clothing is harsh, cold environments.
And so in those environments, we often have an inner layer that's close to the skin. Right.
And then you have an outer layer that protects you against the weather from
the outside, wet, cold, wind.
And then a middle layer that provides warmth. And what's interesting about this
view of layering is it's so simple anybody can understand it.
What you have is three very different kinds of layers that combine synergistically
to create a whole that feels good to the skin, protective of the outside,
and as warm as necessary.
And tremendous plug
and play modularity you can swap in different parts
of the garments but it also shows another
point which is enormously confusing within
the scientific community not in engineering but most
combinations of garments do not produce functional outfits
they really while there's enormous variability
in what you can do there's still
pretty strict strict rules in what can go where or they don't
work at all so you can't you can't make underwear
outerwear you can't make socks into hats
you can't you know for the most part so that's one
way of looking at the layering and but another is the composition of fiber which
is then um spun into yarn which is then knit or woven into cloth which is then
sewn into garments A whole other dimension of layering.
Very different than the first, but again, illustrates properties both of biology and technology.
And so, in that PNS paper, I try to flesh out that example as one that I think would help.
Both engineers and neuroscientists talk to each other because it's a,
it's such a simple example.
Now the danger is it's too simple. It doesn't have dynamics.
The, the, these compositional forms are very, very simple.
The way clothing just sits on top of, you know, one layer of clothing sits on
top of another doesn't really, doesn't really lock in like you would in a,
in, in many other systems. There's a lot of things that are wrong with the thing,
but it gets a starting point.
Okay, that's good. So if we have layers, we also have different,
let's say, processes that give rise to the key properties of these layers.
But now, if I would like to have, let's say, a more parametric description of
such an architecture, what are the key, let's say, parameters that I should worry about?
Well, in this, I mean, again, using this, stretching this probably to the point where we'll break,
um there is there is the rules by which the constraints by which you can assemble
so like i said most most combinations of garments do not produce a functional outfit,
and there's a few rules that you can write down in principle anyway and we certainly
learn them And children learn them as to what make a functional garment.
But the interesting thing is, once the garments are selected...