Ricardo, one of the speakers at our summer school, and Ricardo,
you presented a view on machines, if you want, and robots,
control systems, in which you made the argument that actually they should naturally
include phenomena you could call consciousness.
So, what are your considerations there to come to this rather,
let's say, counterintuitive conclusion?
Well, I think it is not a counterintuitive conclusion.
I think it's just a need of how things are developing and all the needs of having
more robust systems today where we can rely our lives upon in this changing
environment where things are changing so fast.
So, the question for the new machines and the machines of the future is that
they must be much more adaptive in a sense.
And not just adaptive to changes in the environment, but adaptive to their own changes.
And that takes us to this question of consciousness.
It's a question of name, more than a question of fashion.
Thank you. Yeah, but in some sense, you make an argument based on the sense of complexity, right?
Where you would say, well, if we want to start to deal with the world,
the dynamical world, in a way that is acceptable, the controllers will get more complex.
And at some point, the controller will find it necessary to control itself.
And it's sort of somewhere at that point that you're saying consciousness comes
in. So how should I understand that? Where's that transition point exactly?
Well, the question is the moment where humans cannot really control or address
the complexity of the systems.
And the only possibility for that is to have a scalable approach.
And having a scalable approach, humans don't scale well. That's a problem because
humans typically think one by one.
And thinking by teams is not yet solved for humans.
But for machines, it could be. And the question is to transfer the competences
for managing systems, managing complex systems into systems because they can scale up.
And that's the question at the end. Because growing complexity can only be handled
by systems that can grow.
And humans cannot grow as they are today. And that's the problem.
When we are in the case of technical systems, we are touching the limits of understandability.
So in many cases failures cannot be understood by humans because of the complexity
and problems that are appearing for example in countrywide electrical networks
cannot be understood by humans and that's the problem,
we need to transfer into systems that scale more than humans and that are able
to handle these problems that seems counterintuitive, no?
Because in some sense you're saying we want to have scalable systems to come
to a scalable system they must be sentient they should have consciousness humans
are not scalable, they don't scale very well however.
You didn't say that, but that's what I was thinking. Humans are conscious. Yeah.
So that would suggest that consciousness is not a necessary ingredient of these
scalable systems. Or did I misunderstand the argument?
No, the question, we are real. The question of consciousness and self-awareness,
not trying to mimic humans at all.
We reached a point finding some basic design principles or structure principles.
And then we found this community dealing with self-awareness and consciousness
in discussing these topics.
But we came not from the side of mimicking humans, but from the side of finding
some structural competences into
the systems themselves, related to self-awareness and self-competence.
And then we found that the people doing research on machine consciousness,
in many of the cases, they are just trying to mimic human behavior.
Without this understanding of the basic principles regarding self-awareness
and self-management at the very end and deeply.
So we came from the purely technical side into the topics where those people are also discussing.
So in a sense, we are not really trying to build machines that mimic human consciousness at all.
We are trying to build machines that have the competences that are associated
to this self-awareness in humans, but perhaps not the same way.
Okay, but explain something to me that I can, I mean, in some sense,
you're coming from a control engineering perspective, right?
And you say, okay, in control engineering, we're hitting a boundary now.
But then what's this traditional control engineering view that we now have to change?
What's this traditional view and why does it break down exactly?
Well, the breakdown is that typically control engineers model,
have a model of the plant they are controlling.
But they don't have a model of the controller because
the complexity of the controller is so high that they really cannot have
a mathematical model the way they had in the past so you
can have a differential equation model of a car and then you build a controller
for it but then what happens when the controller is failing how can you control
the controller and this is missing and that's the problem where traditional
control engineering is not addressing at all because for example in other cars,
you have plenty of networking inside,
plenty of electronic control units and computing.
If something fails there, there is no control over that.
So if the failure is in the control system, current control technology is not
addressing it. And that's the problem.
Because they are only addressing faults at the level of the plant,
but not faults at the level of the controller.
So we need controllers that are able to control themselves to overcome these difficulties.
Because of the growing complexity of the control systems. In that case.
So that's clear to me. Following this argument, I could then say,
well, but how do I now prevent falling into an infinite regress of these controllers
and meta controllers and supervisory controllers and so on?
Yeah, the question is something that was mentioned during this course.
We bound the infinite regress if in every step the needs for representation are reduced.
So, in a sense, the system is collapsing into a single compact and simplified
representation of itself.
So, that's the question. The question is not just to have a whole representation
of the complete system, but a
compact, more synthetic and more simple representation at each meta level.
So at the end, the system will collapse into a single representation that is
addressing, so to say, in a pyramidal way, the understanding of the lower layers.
So is that a hope, a hypothesis, or is it a reality? No, it's just a hypothesis. Okay.
It does sound a little bit like the old intentional systems stance of Dan Dennett,
where he would say like, well, we can resolve the homunculus problem because
we can just split it up in simpler, simpler, simpler homunculi.
And in the end, it's just a binary decision that has to be made to its lowest level.
Yeah. Yeah, but this is, in some sense, reducing away the problem and not really solving it, right?
You explain it away because, actually, then you have to show that this is possible,
that you can actually, if you use the word simplicity for this,
that you can get to this kind of compression of complexity. Yeah.
Could you give an example of this kind of compression? How do you think about
this, even for a simple controller?
For a simple controller, you can find this kind of approach in very complex
plants where you can have thousands of control loops controlling basic properties
in plants. And then you have loops over loops.
And when you go up through the pyramid of control, at the very end,
what you have is just money.
And you have a single loop controlling the money, the money that the plant is producing.
And then beyond that, the safety and the maintainability and environmental impact.
And then you have a unit control and then you have, well, this is a scale down
and you can find this inside the plants.
But the problem is that in that system, this kind of hierarchy is not self-aware.
So it's not addressing the problems it has upon itself and humans are there
to handle And it's the same problems that you can find in information technology,
in networks and computing systems and infrastructures.
It's the same problem that the computers controlling things are not controlling
themselves. That's the big problem.
But this is an interesting example, definitely.
Also, as you say, right, it's always the humans enter the loop somewhere.
And maybe it's on the basis of that luxury that we have these general purpose
controllers running around, which are human beings.
That then these engineered systems can actually operate.
And this might give us then the false belief that the principles in which they
operate are valid principles.
Do you think that's a reasonable criticism? Yeah, it is.
The question about humans is an old question in automation.
Humans should be there or shouldn't be there. And the question is that today,
in many of the systems, humans cannot cope with the problem.
And that's the problem we are facing these days. Humans are not able to solve
problems at the level of electrical networks, countrywide electrical networks.
They are not able to solve problems at the level of flight control systems when
faults appear. Because of the requirements of speed, of information integration,
of focus of attention, humans cannot solve the problems.
And then what happens? When those faults appear, systems do crash.
And there is no solution today for this kind of problem. Then we have blackouts,
we have very big airplane accidents.
Because humans cannot cope, and we don't have solutions for handling those problems
at the level that we need.
Following your earlier argument, I could then say, well, humans cannot cope
because the engineered system has created a situation in which it cannot cope
and subsequently the human user cannot cope either.
Yeah. Is that fair? It's fair. Okay. And that counts because we are all the
time pushing forward the requirements for the technical systems.
And now we're in the frontier of pushing forward the requirements of technical
systems beyond human control competence.
And we are in such a frontier today. There are phenomena that are appearing
in technical systems that humans don't understand.
Take an example of that? An example, for example, there are waves of electrical
voltage crossing across Europe, north and south, and people don't understand what's going on.
Because there is no way of understanding the whole picture at all.
We need an information theory and a model of what's going on that goes beyond
the capabilities of humans.
That happens also with technical systems in information and infrastructures.
The understanding of how a complex server, a computing server that is composed
by thousands of computers at
the same time, how it behaves when something fails, when there is a virus.
Humans cannot cope with that. The only solution is to shut down the systems
and start from scratch because nobody is understanding what's going on and there
is no possibility of doing that.
But the only strategy that we can think of is making systems keep themselves working.
However, the way you phrase the problem now, which definitely is a serious problem,
seems to create almost a paradox because certainly where I'm coming from,
there's this belief like, okay, if we take a biomimetic approach,
an approach more based on understanding of the brain, we can solve these kinds of problems.
But in some sense, what you're saying for all practical purposes,
also the human brain is not able to solve these problems today.
So my belief is actually based on a false assumption.
Is that correct? Yeah, in a sense, it is correct. But the question is that we
are proposing is not bio-inspired things. What we are saying is that the things
that humans do are an instance of this kind of approach.
That is not enough. I mean, there is a kind of solutions that are,
so to say, the self-awareness solutions.
And humans are just an instance of that with their own limitations.
And making a copy of humans for solving those problems is not the solution.
The solution is going beyond particular human implementations and particular
human brains and understanding the basic principles that are there.
And that's the problem. We need to understand the very principles of self-awareness
to be able to create a new class of system that has much more competence regarding speed and.
Capability of integrating information that humans do have
but to do that we can get by inspiration
but in the sense of understanding how humans do and extracting
the basic principles of that not copying the brain but understanding how it
works and use that kind of architecture to scale it up that's very exciting
so you're saying actually the human brain is is like an imperfect approximation
of some ultimate brain i would say that we might that we might be able to figure
out i would say say that. That's a good word for it.
Okay, so how do I know that I found that perfect brain?
No, the question is, can we find those patterns in the brain?
Can we understand how humans think about themselves and get some basic principles about that?
Can we have a good theory of how humans think about humans?
And that's a big problem. Not just for extracting the basic principles,
that's a big problem also for interacting systems with humans.
The main problem, to my understanding about humans interacting with computers
is that computers don't have a good theory of the human.
It's not a problem of having good theories about computers.
It's the question of how humans are working and how is the perceptual systems of humans.
The question for me is, can we understand humans in a sense of understanding
how they work to make systems that are not human at all, but are exploiting
some basic competences that humans are able to do?
For example, a basic competence is the cognitive flexibility that humans do have.
Humans can learn how things are working. Can we translate that into a machine?
Not by building a human, but by building into the machine the capability of
understanding how things are working.
This is a basic competence that we need for technical systems.
And that does not mean making a human.
That's a big mistake. People try to think that to make intelligent machines,
we should copy humans. I don't think that's the problem.
Would you say then that the human brain is compromised because of,
let's say, the way it's embodied?
Of the evolutionary constraints imposed? Or why is our brain imperfect in that sense?
Well, the brains are imperfect because evolutionary change does not create optimal
solutions, just good enough solutions.
Solutions that are good enough for the ecological needs. They are working.
So humans' brains are good for controlling mammals in the context of 60,000
years ago or so. They are not optimal solutions.
They are scaled up to the level of solving the problems at that time.
And that means that if we want to translate that into a different body with
a different set of requirements, we cannot really copy it unless we are doing a humanoid robot.
Would have the same kind of problems and the same kind of interaction of humans.
If the context, if the ecological niche for the machine is very different,
copying the brain will not work.
We need something very different for that.
So there are plenty of constraints in the human brain that are related with
environments of the humans, with the body of the humans, with the evolutionary history of humans.
And we need to identify them to get rid of the constraints and the limitations
and go to the basic principles indeed. Right.
Now, there's some fear mongering going on certainly across the Atlantic Ocean
about what's called the singularity. In case we're going to build this perfect
machine, it might decide it doesn't need humans anymore.
We get in some sort of Skynet Terminator scenario.
Do you share these concerns? Not really. Not really for a single reason.
I don't think that we are able to create the level of self-awareness and selfiness,
I would say, into machines that humans do have.
We can build some basic properties of humans into the machines, but not create….
Today, these complex entities. I'm not afraid of that.
I'm pretty sure that when we reach the level of being able to create this kind of entity,
the technology will be so sophisticated that we can put any limitation that
we want in the sense of Asimov's laws of robotics.
We will be able to bound the behavior. That's a big problem because today the
problem against building complex systems is how to bound the wrong behaviors at the end.
And that's a very big problem, but I think that technologies today are very aware of that.
That before you press the start button of a machine, you need to really bound
it, bound the behavior, and know how they are bound. Right, absolutely.
So now, coming from this perspective of control engineering,
and also as we discussed in the summer school, there is often this idea like,
Like, well, the brain is a control system.
But now, as we discussed, the control systems that we know as engineers are incomplete.
So how far does it get us to just make that claim the brain is a control system?
I mean, is it a bit too easy to say?
At what point of definition does it actually give us insight?
The understanding of why I say that the brain is a control system is very simple.
Most of the inputs to the brain and the outputs of the brain are just feeding
the brain and getting some products, waste products on it.
And all the meaningful inputs are information.
And that means that what is happening in the brain is an informational process.
It's not a chemical process producing any kind of molecule that the body is using.
The molecules that appear there are molecules related to information.
So, what is happening in the brain is information coming in,
information coming out. That is
the function of the brain, and that is the function of the control system.
So when systems that do that are information processors, and when information
processors link sensors and actuators, they are controllers,
they are no other thing. So it's not a question of...
We can say that it's a question of observing the thing. So we get cables coming
from sensors into informational processor and then going out into actuators.
That's a control system as we build it today.
But that's in some sense a very broad framework in which we define control.
So can we be more specific? I mean, do you say like, well, we have engineered
specific kinds of control systems that we think are particularly relevant when
we look at the brain or when we look at a certain area in the brain?
Is that... No, in fact, this is the whole thing. This is all materials from the 50s and 60s.
This is the whole cybernetics movement in the... The problem that happened with
cybernetics is that it was split in different fields.
One was information technology. The other one was control technology that is
related to building technical systems at the end.
Some people kept it into the world of networks and brain understanding and biology.
But the field is the field of the 50s and 60s. So this is an old picture.
This is not something new. And in fact, if we read what people like McCulloch
and Wiener said, they were saying that controllers in technical systems and
brains are doing the same thing.
And the analysis they did in the past are the analysis that we need to foster today.
And now we have something that they didn't have, that these huge amounts of competing.
Fast enough as to control bodies of enormous complexity.
So we are now in a situation that we really can go back into this cybernetic
vision and really understand how brains work as controllers. I mean, this picture.
So in the domain of, let's say, the neuroscience of motor control,
there's quite some excitement nowadays on particular kinds of concepts coming
from control engineering and people believe, okay, this is now explaining what
the brain is doing, what structure X is doing in terms of forward modeling or inverse modeling, etc.
What do you think of these approaches?
Is this actually helpful or do you think it's sort of confusing?
No, it is very helpful. The problem that we have is that when we look at control
technology and control theory, the problem is that control theory, as we,
I would say, control engineers do, do is based on solid mathematical models
and sets of equations that we can solve.
And we can solve analytically, and that's the question.
When we have models that are so complex that we really cannot solve analytically,
then that's the field for other people trying to do simulations and trying to
do computer implementations and trying to do, well, environments for testing,
forward testing of models, in a sense.
And this is the field of people that are modeling brains, because the mathematical
description of a brain is so
complex that we really cannot solve the equations by analytical methods.
So the communities, in a sense, are split into two ways of doing things.
One way is the mathematical-analytical method, and the other way is the forward
model-based synthetic method of using computers for looking at what's going on.
Because really, we cannot invert the model, and that's the point.
When complexity grows, when non-linearity appears, we really cannot solve the
equations and invert the models to make the controllers.
Then we revert into this alternative way of exploring.
And by simulation what's going on and trying to understand that.
Because really we cannot analytically solve the problems.
But the community should get together. Again, there are problems and theoretical
concepts that are critical to be shared.
The concept of observability, for example, the concept of controllability.
Those are concepts that should be a common trait across all these technologies.
So how would you see a concept like controllability giving us leverage when we're look at the brain?
Well, the question of controllability is a question of how can we do the body
do whatever we want and what things we can do with this.
And the question is, can we solve analytically that problem?
Can we decide if a particular body, for example, of a humanoid robot is controllable?
And that is a theoretical problem
that can be solved. But many people in robotics is not trying to do that.
It's trying just to, for example, people that are trying to do flexible They
are trying to solve a problem by just experimentation and hacking,
I would say, instead of having the deep models that they need to address the problem.
But the problem here is that we have isolated communities with different ontologies,
some more rigorous and more mathematical, more physical, the other one more
related on empirical aspects and, so to say, philosophical even,
even metaphysical assumptions.
Assumptions, and we need to get rid of the separation and have a single community
across this, from the very simple controllers that we can find anywhere,
to the brain, and to the psychological,
more complex and more sophisticated psychological aspects, and have a single.
Ontology that can be shared across all this stuff.
Just take, for example, the concept of information.
Nobody agrees on that. And when we talk to different people,
they don't reach an agreement on that. And that's a problem.
The problem, I think, the big problem that we have is the community being split on those areas.
Now, to bring them together, if we, for instance, go back to this notion of
the brain as a controller, or more specifically, as was discussed in the school,
the brain is an adaptive filter.
And if you have this very broad definition of a controller, like,
okay, whenever I'm mapping some input states to output states, whatever
happens in between i consider a controller in some
sense then it's not very a very specific formulation right because in that sense
and indeed anything that is dealing with an input output mapping would fall
in that category and automatically also the brain so for adaptive filter it
would be sort of roughly the same thing because i have an input output mapping
that i'm just sort of adjusting to some to some criteria so is it.
So at what point does it actually give us leverage when we look at this?
When it really adds understanding and at what point is it sort of covering a
little bit actually the important details of the questions we should ask?
Yeah, this is related with what I was saying before about the ontology.
When people say an adaptive filter, when people say a filter,
when people say a controller, some people will say that they are different things.
But if you go at the basics of the thing, they are the same kind of thing.
You were saying input in, output out, and that's all.
So you have a system that has inputs and has outputs, and that's all.
If you connect that system a particular way, it will be a controller.
If you connect that system in another way, it will be a filter.
That will depend on how you use the inputs and outputs of the system itself.
And that's a very big problem that is related to something that has appeared
in the course, that is the idea of function.
What is the function of such a module? It is a controller or is a or is a filter.
And there are two interpretations of this function thing.
One is the relation input-output that should be described by the mathematics,
describing the thing. And the other one is what it is for.
What is the function? It is playing in the system and it can be a controller
or can be a filter, can be whatever.
The same kind of mathematical description input-output can be applied for different
functions in different systems.
And the clarification of the The terminology is one of the major tasks that we need to solve here.
Because we can have an endless discussion about is this a filter or is it a controller?
Or from a theoretical point of view, this discussion is no sense.
Because those two kinds of systems are the same kind of thing at the end.
The only difference is how you use them.
That will depend on the use, not on the thing.
The thing is not a filter, it's not an intruder, is a dynamical system. That's all.
And many of the discussions we can find in current designs are related to not
understanding those basic, so to say, theoretical underpinnings of the system.
But now you also have been in projects where you have been really trying to
link your understanding of control engineering to, let's say, the biology.
Yeah. So what are the lessons from that experience? How successful have these attempts been?
Well, I'm not very happy and I'm very happy. So how can that be? be.
I'm not very happy because it's difficult to break the frontiers between communities.
So having biologists talk with roboticists by providing a common language to them is not working.
It's not working because the languages of the communities are being kept as they were in the past.
It's not easy to move or to share concepts at all.
So it's a happy situation.
On the other side, it is good to to see that people is hearing and then trying
to collaborate and hearing the others.
So I think that the first thing to do is to get this common language at the
end, and then common ontology.
That is, to my understanding, the basic ontology of physics,
and then on that, going up.
And I think something will help is that, for example, in neuroscience,
most of people doing neuroscience today are physicists by training.
So they start to talk about the mathematical basic concepts or theoretical concepts that are there.
So I hope that things will change in the future. But I think that the main problem
is that of the communities.
In my experience, the problem is the breaking of the communities because the
problem is so big that nobody has the possibility of addressing the solution.
Because someone has the tools, the other one has the global understanding,
another one has the data, the ground truth, the experimental conditions,
and we need to put all there into a single picture, otherwise it will solve the problem.
To work our way towards the finish line.
You have been going around in these fields for quite a bit and you also have
this very unique experience of interfacing two of these other disciplines and
really to try to solve problems very concretely.
So if there is this one law, the one Ricardo Sanz law that we should all adhere
to and try to understand the brain and how to then build this ideal brain that
can be the ultimate controller, what's Ricardo Sanz law that we should adhere to?
The most important thing that we should do is to try to be rigorous and formal
about the terms that we are using.
If we are able to do that, then we will see that the concepts that we are using
in different fields are the same.
And that will help transition from a pre-scientific domain, that is what we
are now, into a newborn scientific community.
So I would say that the first step to do is to be rigorous about the terms we
are using that is the very first step very good five years from now I'm gonna go up to Madrid and.
I'm gonna ask you look Ricardo there was this prediction you gave me and now
I want to know whether it came out or not what's this one prediction you're
willing to stick your neck out for today well I hope that in some years we will have a,
real implementation of the fundamental concept.
Working in a wide scale of different systems, from simple machines and simple
robots into all networks and real large industrial plants.
Having a single set of concepts implemented in such a variety of systems will
give credit to the vision, in a sense.
So what's the most advanced concept I can hold you to five years from now?
The most advanced concept, I think, is a system that is able to control itself
based on the model that engineers used to build it.
Okay, cool. And that's our key cornerstone for the work.
Great. I'll be back and I'll ask you about it. Ricardo Sanz,
thank you very much for this conversation. Thank you.