This is the Convergent Science Network podcast. Leading researchers in the domain
of neuroscience, brain theory and technology are interviewed by Paul Verschoor and Tony Prescott.
This is Paul Verschoor with the Convergent Science Network podcast and I'm here with Daniel Polanyi.
Daniel spoke about his work here at BCBT Summer School 2016.
And Daniel, you very much emphasized the more, let's say, information-oriented
perspective on cognition, and in particular embodied cognition.
So, how did you end up taking that specific perspective and trying to understand
this complex phenomenon? phenomenon.
One key issue that bothered me for a long time is the question,
how is evolution directed in a way that moves it forward sufficiently fast?
So one question I was interested in many years ago was evolution of sensors.
How does an organism or a species learn evolutionarily that a particular sensoric
channel has information that's relevant for its survival, or could be relevant for its survival.
Which means that because we can assume that evolution is very local in terms
of exploring the solution space,
that there are indicators which are of local nature that nevertheless,
give a drift to the evolutionary process, which advance it towards exploration
of further and deeper informational sources.
And the way to quantify that, to characterize that, for that you wanted to be
able to say how much or what this information is about.
And for this we had to understand how to reinterpret information theory as basically handed down by Shannon.
In a way that allows us to incorporate the concept of relevance.
So relevant information.
And a very important article came out in 1999.
It was the information bottleneck article, which basically made the argument
that you can actually color, you can actually tag information.
In other words, you can make a distinction between information that's relevant
and information that you may have,
but you don't want but now and
you you take a very specific view on
on the evolution in that sense of both uh sensing and
and cognition because you consider sensors as being highly optimized systems
that that work close to let's say their physical limits um is that is that critical
to this approach that you take that sensors are really that highly optimized or that's arbitrary,
In principle, it's arbitrary, but there are indicators that this gives us a
hint on the fact that information really is a major driver for evolution.
So I'd rather would say that this is originally it was a motivator to say,
yes, information is important for our nature.
But as we started looking at the information perspective itself we suddenly
saw that informational perspectives if you assume that evolution uses it as
a proxy for directing, for genesis.
Processing information processing or behaviors etc.
That it may actually desire or not desire but drive, push towards a increasingly
warm, high resolution sensorics.
So in other words, it's basically a mutual.
Feedback loop mutual positive feedback but now
it's okay if we consider that that sensors are
highly optimized they're highly optimized relative to what excellent
question the original idea was that they are highly optimized relevant to some
hypothesized goals that was what you originally looked at so the original relevant
information was just about that some indicators came up that that indicates no, it's not enough,
you get actually a high level of other possible goals that become accessible.
And of course, if your niche changes, or your agent gets a bigger brain,
slowly, or you have additional goals in your life, it turns out that suddenly,
with the same senses, you can solve other problems.
So in other words, you get the opportunity to drift from goal to goal.
So several questions emerge, where do the goals come from?
Second question is how does this drift possible at all?
Because often, if you're highly specialized, you can't change anything without
losing performance or else.
The argument that we suppose now, or that information theoretical view has given
us is sometimes you get more information than you bargained for.
This extra information gives you what people would call some kind of.
Openness for evolution or permits you more
adaptiveness in evolution that then you
would naively expect but now we
can distinguish different sensor systems right so for
instance we because the oldest sensory systems are probably mechanical systems
um so they just detect forces then we would have sensor systems that deal with
molecules molecular structure which is chemical sensing then you have your sensory
systems with sound pressure waves which which might build again on mechanical
sensing, that's your auditory system,
or you might look at a lateral line of fish that would pick up turbulence in flow.
And then lastly, you would have sensor systems that like photons,
and there you have vision, right?
So in that sense, these biological systems you look at, and that you try to
understand at the first step,
the sensors from which interface to these different environments in which they
can exist, you could argue, well,
but maybe these different environments have also different informational requirements
and a sense of a single notion of optimization that will sort of fall flat rather
quickly because there's diversity of subdomains in which sensors have to adapt.
So do you believe we can have like a generic informational optimization criterion
to look at these sensor systems or we should already link it to the specific
subdomain in which they act if we go from mechanical sensing to vision?
I should expect that, of course, the ecological niche plays a role.
And you will basically select for various, there will be different attractives.
If you look at how a sensor and its environment interact, you have different
attractives, different solutions for the same problem.
Classic example is of course, bats.
Bats use essentially the auditory sense in a vision form, in a vision modality.
This is a very interesting development because it's vision, but it's also active
sensing, very, very active sensing without sun generation doesn't work.
But on the other hand, you have, for example, owls, and they just improved their night vision.
So it's not a unique solution, and maybe there is a historical accident that
gets you in a particular direction.
One interesting case is, of course, snakes can detect infrared,
and they use actually skin sense,
which has been basically anatomically formed in such a way that it operates
as a camera obscura for infrared.
Now, the funny thing about that is everything about this hardware that they
use essentially is equivalent to what we have.
So in other words, the main difference between the snake skin sense is,
apart from the wiring the brain essentially the
anatomy not the actual nerve uh
quality or skin property or so
it's really the anatomy most but now
if you if you say that that sensors tend to
operate at some physical limit right they're optimized and indeed we look at
these strange kind of sensor systems like infrared and the snake that sort of
is is capitalizing on different sensory systems in addition of course such like
an infrared detector only can.
Is effective as a sensor if there's actual signal transduction related to it.
So can we really talk about the evolution of a sensor in isolation without taking
into account the morphology in which it's embedded and also the signal transduction
mechanisms that it has to deploy or it has to be interfaced to?
Of course, I completely agree. I mean, you have multiple systems interacting.
And there may be in the evolutionary history, you had different balancers,
different drives in different directions, and there may be not a single attractor
into which you converge, starting in from the same original species.
So I don't expect that. But again, this becomes then a process of actually modeling
the particular evolutionary pathway.
What the informational view gives you is not saying how this pathway will look
like, because that requires a lot of assumptions.
What it says is, however, what possible niches could look like.
So what are places where information is hidden that could be discovered?
And it gives us a hint why you actually can find this niche.
The big question in evolution is not that there are solutions which work.
The big question in evolution is, how do you get there? How does evolution actually discover the niche?
And the argument is, if you have some information that indicates something interesting
is there, there is, that would be the hypothesis,
an innate drive, or an innate, relatively generic mechanism that always assumes
that when there's better information available,
then it's worth refining it, wherever it comes from.
And then you have basically something that accelerates adaptation by pure.
If you like, experience that this actually works in our world.
In other words, information is a local, gives you a local gradient or a local
indicator where good solutions may be found.
Right. But this is an important issue, right? Because it's also sometimes because
of the processing that goes on behind the sensor that you actually can reach
the actual hyperacuity of the sensor itself.
So that means the optimization of that sensor sheet, In some sense,
it's not informing you about the kind of information processing that is possible.
You start, let's say, to integrate across multiple senses. A typical example is chemical sensing,
right, where the sensitivity of single chemoreceptors on the male moth antenna
are orders of magnitude weaker than the detectability,
the ability that the animal has with respect to heart rate responses to pheromones, right?
So, you will see heart rate changes to homeopathic levels of hormones in the
air that you will never be able to pick up with a single sensor that they have on their antenna.
So, that means, in some sense, it would argue that to really think about sensor
systems in isolation could also then be misleading you in terms of understanding
what the informational capabilities are of the system that is integrating that sensor.
So, isn't there a risk in the analysis you presented that you say,
well, if we have a sensor and the sensor gives you the upper limit of what can be achieved.
And we know for many biological systems, this is maybe not really the case.
They can go beyond what the sensor as such in isolation will be able to deliver.
Well, if you look at that, you'd have to look at integration over time,
for example, and take memory into account.
Right. So, of course….
When we say a sensor limits the information in the sense of data processing
inequality, that you can't get more information through a sensor than the sensor permits.
But of course, if you can integrate over time, then the effective bandwidth is much higher.
You can accumulate evidence, there's no question about that.
And if you do it for long enough, then you at some point have enough information
to make your decision for itself.
Or you can integrate information which otherwise is worthless and suddenly other
information comes in and suddenly this information becomes valuable.
In fact, in the case of the snakes, the infrared and optical information is
integrated in the optic tactile and it's quite intricate how they interact and
how the snake decides there's an object of interest there.
So this now becomes relevant for the second point that you made,
also your talk, where you say well so sensors are optimized so that means also
from an energetic perspective,
they they are optimized to give you certain kinds of
information efficiently while the process and the cognition that runs behind
it is expensive right because metabolically the brain takes a disproportionate
amount of your energy budget as as an organism and um so so you see that as the
main from an informational perspective this this
this energetic optimization of then the processing
the brains perform is the main challenge for evolution
is that is that the consequence of what you're saying it's a bit more complicated
than that um there are different time scales at play here because if i have
a brain that's big it will eat that energy and i can't just say okay as a as
an adult i will just shrink my brain by 50 percent who need it actually does happen in some animals.
There's a it's called a fish that eats its own brain.
And that animal basically finds a rock.
So it's wrong until it finds a rock, and when it finds it, it will never leave the rock again.
And when that happens, it actually basically consumes its own brain.
So it does happen, but it's not typical.
So the way to look at it, in my opinion, is from a point of view,
can you profit from a bigger brain, say, on a longer timescale,
longer means over several generations?
And if you can, then yes, you maintain
that, otherwise you just reduce the investment into bigger brains.
You should also not forget that there's of course Darwin's brilliant idea of sexual selection.
So maybe the highly social animals will be more selective towards more intelligent
sexual partners, and thereby the brains will be driven to be bigger.
On the other hand, that has to be sustainable,
and that only works if the brain actually that's something sensible right
now would you but now so now you see the
what i'm driving at right because i was
making the point that the sensory the
the perceptual or the sensory capabilities of the
organism can go beyond what
the sensor can deliver by virtue of doing the processing right so now i could
argue well maybe this solution was identified or was converged on during evolution
Because to really optimize the sensor would actually metabolically be way more
expensive than to put that effort in the processing.
Like integration and time, use memory.
And it's in that combination that they can actually get a virtual sensor,
if you want, or an effective sensor that gives me the information I need.
I basically would be loathe to separate them so strictly, because sensing and
processing are, I can't imagine situations where it's almost impossible to distinguish.
So in some cases, relatively simple to distinguish, in some cases not.
We have an example, I didn't show that in the talk because of lack of time,
but we have an example where we can choose whether we prefer to use memory or
sense the sensor as to achieve a certain utility value.
And you can shift that around and say if the sensing is cheaper,
then you shift it towards sensing and you use less memory, and vice versa.
And you can look at it yourself. If you look at your Google Maps,
when you find a road, look at the map all the time, then essentially you say,
sensing is cheaper than remembering the path.
But of course, this is very inconvenient when you do the path a lot of times,
and it may be actually cheaper for you to keep it in mind, not having to look
and stop at every corner to watch the map externally.
Externally so i think i think i do agree
with you that um processing is not maybe
viewed as an extension of sensing perhaps that's
the way it's it emerged that basically um
the system discovered that while treating your own brain as a meta sensor is
is a good thing uh on the other hand it's i'm not clear the way but whether
we can really separate that yeah but still for the for your framework, this is,
let's call it the challenge or a constraint we have to again look at,
maybe after we went through the framework.
Because it seems to say, well I must be optimal in informational terms because
that will help me to reduce the cost of the processing.
I would say that.
That the cost of the processing must be sustainable.
It's like having a company that permits itself to have a certain amount of administration level.
And for example, if I have a big administration, you have to make enough money
to keep these people working.
Doesn't mean that you immediately reduce the administration when you don't have
enough money, or that you will just not survive on the long term if you have
a big administration that you can handle or that can handle your stuff efficiently.
So the parsimony principle is, of course, that if I have something to process,
then I don't want to waste too much effort because I may have other things to process.
If I have an emergency service, of course, that emergency service has a certain
bandwidth, public, which is required, and it has to be activated at any time.
And if the emergency service requires 10 bits per second for some reason,
these 10 bits must be available to me as an organism, but of course,
I may do different things and of boring things I'm doing kind of grazing or just walking around.
I don't want to lose a lot of processing power because I may need it for other
things. So the argument is a mixture of energy processing, a mixture of other
resources that may be required for other tasks.
It is, however, still a parsimony.
Right. Okay. But then, so that then brings us to sort of the linking of the
sensing and the action and decisions, right?
And this is where I think you really put the brunt of your effort to try to
understand what should be the properties of what's called a decision-making component. So...
So how do you see then the core on the one hand constraints that this decision-making system is facing?
And how do you see what are the main principles that allow a decision-making
system to satisfy those constraints?
Well, the core constraint is, of course, the way the agent is embodied in the
world. This imprints a signature on the information flow. of.
The way what you do in the world impinges in the world and you perceive the
impingement again determines what you can possibly do.
Because that's not something that it's in your choice. It's given by physics,
by your physicality, by your body.
So this is the main constraint that determines what happens.
The other constraint is, and that's much freer, and that's of course an apostolate,
which may be wrong, of course, that the brain essentially Essentially,
at least in our models, it's essentially free to organize this information.
This is, of course, not real.
In reality, there are other constraints. But what we would like to know is what
other constraints are natural.
So one example is, for example, this goal-relevant information where we specifically
split the simple decision-making and the long-term goal, say,
to study the emergence of sub-goals.
Of course we put in an assumption here that there's a long term memory that
stores the goal we want to go to ultimately this is an assumption as Sander
van Dyck one of my former PhD students said it's embranement it's,
not the body that we are fixing here, but actually how the brain is constrained internally.
Ideally, in our studies, we want to limit the assumptions about embrayment as
far as possible, because we would like to have an answer is what type of brain
structures do you want to process this information efficiently?
So in other words, you ask, I want to process information, okay, Okay, a certain amount.
And the question is, are there special sub-manifolds of solutions which prefer
certain brain organization for processing the search adaptation faster or efficient?
This is a question that is ongoing research.
We don't have a clear answer. But clearly, when you make assumptions about a
brain, if I may use this word.
Then we get things like, for example, versions of sub-goals,
just by virtue of saying, okay, there's somewhere where I'm storing slowly changing long-term goals.
And then this concept of some goal emerges naturally from the interaction in the world.
So, making judicious assumptions about how the brain is structured,
how the world is structured, can give you very natural hypotheses about emergence of natural phenomena.
Again, yeah. But wouldn't it be fair to say that it's more like enmindment?
Because you cannot really make normative statements on structure at best you
can make normative statements on function on information flows just for clarity I'm not very,
ideological on that it is not a statement about how the brain actually looks
it's a statement about how a possible organization of information processing
may look for certain purposes.
We have assumptions about how information is being processed,
but these assumptions are.
At this stage not really well founded and
they are based on possibility and whether
the phenomena that emerge are something you
actually see yeah but in that sense you really want to
get to a normative formal framework right
because that was also one of the points you made that it says
the biology is ambiguous we have no clear understanding
how this works as a structure robotics is
arbitrary people have many different solutions for the same
thing we don't know whether there's any common principle so
what we really need is is let's say
a normative framework that says okay these are the
decisive criteria that all these systems have to follow in order
to be now informationally optimal yes okay
and and that that's why i was hassling you earlier on
this on this notion of informational optimality because that's of course a very
important guiding principle and assumption of this whole framework yes right
yeah um so now but then you you linked your framework to this informational
framework that you're advancing to the Carnot cycle,
which essentially describes energy.
It's sort of the expansion of a chamber that allows you to do work.
So why do you think that's a good metaphor to look at decision-making and information
processing in biological systems?
I was very kind of going quite bravely into an aggressive metaphor here.
But of course, there are actual attempts to link information processing and physics.
And we have seen quite a bit of progress in recent years, actually.
For example, by David Volpert, who has generalized Landauer's principle,
and there are a couple of other really interesting pieces of work in this area.
And the question is, on a very low level of physics, there seems to be an intricate
relation between energy processing processing, energy consumption,
energy production, entropy production.
Of course, these levels are very, very far away from where an organism sits
in his information processing level. However, in between...
There are many levels, and we have to take into account that in every level,
there is information processing happening.
When a cell organizes its organelles, the organelles organize the ATP consumption.
This is organization. This is information processing.
And I would now go on an extreme speculation.
I made it entirely wrong, so don't take my word for it.
But what I would say is that as you progress to arrange your information to
higher and higher hierarchies,
you have a kind of loss function or loss component to basically every hierarchy
level loses your factor 100 of your free energy that you have.
And what remains is kind of your investment for the next level.
And as you go up and up and up, only very little remains for you to actually
operate on freely and free in the sense that you can do new discoveries that
accumulate this information. Most of the stuff is administration.
So administering ATP, where ATP goes, where your cellular motors are driving
stuff out, getting trash out, getting nutrients in.
This is information processing except that you don't know what's happening.
It just happens under the boot.
But I do think that in principle a complete theory would encompass the lowest
level unbreakable barriers of Landauer essentially and friends.
To the highest level where essentially information is almost detached from the physicality,
in a way um and find out no no
of course it's not detached there is a link but i think this link becomes more
and more tenuous with every level so it's very at this stage we're very far
from actually seeing how the high level information processing constraints are
linked with a low level physical so i two parts with two answers one answer
is an aggressive method metaphor nothing
else the other part is no it's actually not just a metaphor it's real but um
that kind of cycle is really far down the scale and cognition is very far right but that's an important,
point about this right because it also means that you are willing to commit
the notion of information processing to actually physical properties of the
system because in the end what we're talking about is quantifying the entropy
in the system where the entropy is increasing or decreasing.
If you say, look, this is information processing, it means there's a change in entropy.
Essentially, this is really important that we don't get confused what we mean with information then.
We need to be careful. You don't necessarily reduce entropy in the system itself
by doing information processing.
You have to take into account the environment.
So in the environment, you can choose sub-environments where you reduce entropy,
for example, by information processing. sensing.
You definitely increase other type of entropy because you simply generate trash, if you want to.
And even your system remains unchanged.
You can have a completely reactive robot that essentially pushes all the boxes
in your room to the walls.
But that system reduces the entropy of the boxes distributed over the room.
Itself, it's completely reactive, so there's no internal change of entropy.
And of course, entropy in total of the universe increases, because there's heat
and the atoms, the gas of the air moves faster, whatever. So in other words.
Information processing does not mean for the agent itself reduction of entropy.
No, it relates to the observer describing the agent in its environment.
Yes, if you include the environment, then yes. And also it depends on what you
consider the environment.
So it really depends what you look at. And on top of that, even it would depend
on which state variables of the agent environment you consider.
Absolutely, yes. So this is really important to understand. It's a relative perspective.
It's observer-dependent. It's always observer-dependent, except if you go to
the full wave function, if you like, the full state,
in which case the problem is, in my opinion, not yet satisfactorily addressed,
but perhaps it will happen.
Right. So then to illustrate, to introduce your framework,
you start to make a distinction between open and closed-loop systems And to
try to build a more formal perspective on how they would be differentiated better or worse, right?
So why do you think now open and closed loop, that comparison is helpful to
introduce this information theoretic framework that you're advancing?
We could have looked at, let's say, sensory processing as we discussed earlier, right?
Okay, I mean, essentially, this idea of to Shetton Lloyd of considering these
two cases there in the ways of the extreme cases, an agent that is basically
nothing else but the blind process doesn't take in any information.
It's basically a modulation of physics.
Consider a modulation of physics, which has a particular property that itself,
it does not take in any information.
Closed loop means it does take in this information and gives it extra power.
It makes it a more complicated process.
But it turns out that you can, and that's where it gets interesting.
Bound the extra entropic influence of this closed loop agent by how much it takes in.
And this is, in my opinion, very cool, because you see for the first time,
in a way, well, not for the first time, I should be solving it before,
but in a precise sense for the first time,
that cognition or cognitive performance underlies information processing principles.
You can't just make decisions of a certain quality without having a certain
informational invariance or minimal balance respected.
And that is cool because you essentially say cognition is not some kind of abstract
platonic thing that just happens somewhere and you can make anything happen.
No, you can't make anything happen. happen, there are certain constraints on
what you can make happen under certain circumstances,
which I think is a really important step because it says, unlike what AI usually
does, which reads AI kind of intelligent decision making in an abstract world
that's devoid of any constraint,
you can essentially think of anything, you are constrained by very, very tangible.
Aspects of the world.
But now, so what you're saying is, look, if you consider the open loop case
plus the information added by being able to sense, you give an upper bound of
what the closed loop controller can do. Yes. Right?
But I could, that's a toy example because within the context of the niche and
that you earlier emphasized as being also relevant, if I have an open loop control
in an environment with predators, I'm dead in no time.
Right? So then you can go, okay, but maybe the upper bound would require the
open loop plus the information coming from your sensors plus some minimum memory system.
Sure. Right? To satisfy, let's say, some lower bound of survivability. Of course.
Does that matter to you or that doesn't matter? No, no. Memory is the next step.
Okay. We did not consider memory for a very simple reason because,
well, to shed a light, did not consider memory. But of course,
memory is the next natural step.
In fact, there are some attempts to consider memory as a kind of constructed
reality that is constructed in such a way that it keeps the things active and alive, which are...
Relevant and which require history to accumulate, for example. Yeah.
So if you want to have a thresholding process, you have a memory that is able
to count or to measure, oh, yes, I have seen enough, and please,
now we can make a decision.
So yes, of course, memory is the natural next step.
But memory is a strange thing, in my opinion, because it's half environment
and half agent in a way. It's a quite hybrid thing.
I agree no you're right so then
okay so so here we have this this example but
now we can start to define let's say constraints onto this agent environment
interaction but now in open closed-loop case you discuss basically we have a
world state uh which are considered discrete states if i got it right and this
is something else we can worry about and then we have sensory states and we have actions, okay?
So we have a three-state system. And so world, sensor, action,
and they're coupled to each other.
What in that makes the agent then? That is a very sharp question.
So if you remember my diagram about world, sensors, memory, actors,
memory, and so on, I emphasized the symmetry between world and memory.
And when you ask what makes the
agent there the question is very subtle if you look just
at the graph you don't see it the graph
itself does not make a distinction there's no way of distinguishing world
memory my personal opinion is and that's completely speculative i can't prove
it at this stage the main difference between world memory is the fact that the
world arrows are highly constrained there is very little that can happen there
and the information density is low the world is simple.
Memory is where you can rewire in principle, at least arbitrarily.
So you could have a maximally compressed information processing.
Which is essentially when we talk free will, I think that's what hides behind it.
The fact that in principle you can have via evolution or adaptation or whatever,
a very, very complicated rewiring of the memory, which is virtually arbitrary.
Think of a computer, a keyboard, you can choose any keyboard you want,
can rewind it as you want.
But what you can't choose is how to organize the pixels on the screen so that
your eyes will recognize it.
So in other words, there you have very strict constraint about geometry,
but on the way your actuators will operate with that, and basically your memory
can operate with it, and you have many choices.
And I believe that the agent, If I give you a system which contains a world
and an agent, I think the agent will be that part of the system.
Where the constraints are basically unconstrainedly complicated.
So the world would be the part which has lots of compressible structure.
It's a very vague statement, I know. But let's resonate with it anyway.
So already the diagram that you sketch is in some sense a tiny fraction of the
total set of possible states, because in some sense in your diagram, you go back in time.
You say, okay, I'm at T0, and now I can show you back into the past where I came from.
And that's this trajectory of world states, sensor states, action, et cetera.
Because at any point in time, there's a plurality of world states, right?
Sure, yeah. There's a plurality of sensor states.
There's a huge action potential. That's true. Right?
And then it all collapses into one world state, one sensory state,
one action. And then we have our next world state, right?
So that means we have to constrain that highly variable set of states of different kinds.
So my question is, aren't we lacking the key state that makes the agent an agent,
and that is an internal drive state,
that the agent is, I'm ready to explore to serve my survival of my informational
needs, or I'm ready to consume a resource because I have to take care of my energetic needs.
So, isn't that the internal state defined by the survivability of the agent,
not a key constraint on this plurality of world sensor and action states?
Well, the model is quite general. So, in fact, what M is…,
i didn't say so it's not a problem to plug into
m or to internally consider m as
consign contains such a let's call it pseudo goal
or pseudo teleological um parameter that's
not a problem um so we could split it so the assumption that m is one coherent
blob is the most generic assumption when you have just one age um so you could
put it in wait why would you put it in m It should actually already mediate
between the sensory state and the action state,
right? In the earlier example.
Yes. In the earlier example, we had just a reaction. But essentially,
in the later example, S doesn't talk to A without M.
Yeah, exactly. So you collapse it all in M, essentially. Everything is in M, yes. Okay.
So then, okay, so we have a scheme now. We can think about how behavior comes
about and how behavior in turn depends on sensory states in the more advanced
version, how it also depends on memory.
Okay, good. So we got that. But now what you really want to understand is,
okay, what should my actions be?
And then you say, well, my normative perspective on that is that your actions
should serve your informational needs because the controller wants to optimize
its information processing because that's the most expensive thing it's facing. Correct?
It's a mixture. I mean, we looked at Lagrangians. So we looked at a mixture
of goals or goal rewards versus informational needs.
It can go just for informational needs, but in that case, you can just do nothing, for example.
In the case of empowerment, it doesn't care about informational needs.
In fact, it's completely orthogonal to that. It says, this is my goal.
It produces a goal from this prediction. direction.
I have not made a statement on how to balance informational needs and empowerment.
It's possible to do that. We have some work in that direction.
Then you get some salient strategies emerging. This is work by Tom Anthony.
But the interesting point is really at this stage that we want first to understand
the ingredients before we try to build a synthesis of things which we individually
don't understand. Right. Okay.
Fair enough. So basically what you're saying is, look.
Within the informational perspective, perspective right i
can make a normative statement of how i should bring these
things together and how i can use my memory to do that yeah it's
just one one one view yes on
this system right and there um you
you made the point that there might not be a free lunch but
sometimes there's free beer so so what
does that mean relative to optimization of information that
was essentially an allusion to the embodiment example
so the twisted world example where you have basically in
the agent that if the world is
simply organized so there is a labeling of conditions which is consistent over
the world then the agent can solve certain natural problems very very easily
with little information processing when you do the relabeling which in terms
of what we would call traditional,
AI is completely equivalent people would say it doesn't
make a difference it turns out for an embodied agent if
you take the embodiment seriously so actually know of an ease are essentially
a local coordinate system of the agent which agent takes it with it if that's
completely skewed up says and twisted around the world the agent will.
Have a much harder time performing the same task in other words,
your world if it's well designed if your embodiment is
well designed or your world is nice to you that's the
way i like to say it then your cognitive cost
is so low that you can easily solve a task that actually looks quite difficult
and that is something that of course a group of pfeiffer and many others have
made for a long time but i believe information theory gives us a window and
to why it's such an advantage.
It really reduces the cognitive load we need to solve the task.
Yeah, but I could argue that isn't that almost a trivial statement because if
you get information for free then it's easier.
Well, you don't get this explicitly for free. The information is implicit and
the physicists would call it a gauge symmetry.
So an agent that has its actions Basically, when you move the agent around the
world, these actions keep a certain type of meaning.
If that meaning completely is perturbed or shuffled around from the movement
of the agent, then this meaning doesn't help you.
This is what we have in the twisted world. In that case, the agent basically
is moved around, but the actions north, east, south, west lose their meaning in any other location.
They mean different things. On the other hand, if you have basically anything
that takes its action with it, it's like a local gauge symmetry,
basically saying, I'm taking this property of north means roughly the same thing in this world.
Now, in which sense does it mean the same thing?
We haven't properly defined that. But what we did say, we measured the information impact.
That is very visible. So it's not saying, this is the number of bits that the
world actually gives you.
We say, if the world has this kind of symmetry or pseudo-symmetry.
Then you gain cognitive load. Your cognitive burden is reduced.
Right. My hope would be that there would be at some point a way of actually
writing down, saying this is how much your world actually tells you up front. Right, exactly.
So if we forget about the embodiment for a second, what is the specific informational
quantity that's being now optimized?
Is it like the description length of the information I deal with?
Is it, let's say, the information gain I have per step size?
In our examples, it was what we call relevant information, the information you
need to take an action at a certain average utility level.
So that's basically a pull quantity. It tells you how much information do I
need from the environment to perform actions at that level.
Description length is something more complicated than that we don't look at that at all,
this will be also probably more related to learning itself,
rather than to actually we would like to call this metabolic information it's
information that you just process,
basically on every step it's like you bring out your trash you repost,
you process it and this is what you do every day it's basically how much time
do I have to allocate or how much resources, information resources resource
to allocate for just maintaining status quo. Okay.
So then the criterion is really utility, right? I have to sort of optimize it
to reflect some utility.
But then how well does this scale if the number of possible goal states increases
and also when the potential goal states can be contradictory?
Excellent question. The scaling is something we start to address.
It's a problem in empowerment. The component has some aspects,
gets some aspects of that.
We have various tricks, I've released an approximate of algorithms which we are using.
It's probably of the method to be doing the most well-developed one,
because it turned out to be.
Now, you used to talk about contradictory goals. It's an excellent question.
And I'll give you an example. If you are in Barcelona and you say you want to
go to, have to decide you go to Lisbon or you go to Granada.
Then I would say that, first of all, you can go to Madrid and then decide.
So in other words, it's part of the rule, which you can take without committing to a particular goal.
Then at some point you have to commit and then you have to split let them make the decision.
And I do think that organism, if we believe our goal-relevant information formalism,
would profit from doing so for various reasons.
Number one, you don't have to know so much.
You can concentrate on moving on highways only. For example, the road to Madrid.
You don't have to learn all the side roads that you would have to take to Granada or to Lisbon.
First you go to Madrid, and only then you worry about the next step.
Second, organisms, I think, profit from not committing to a decision too early.
So if you can avoid to not commit, it's actually an advantage because it means
that you can still reorganize, re-decide if another goal pops up.
So my argument would be contradictory goals, as long as you don't have to make
the decision now, don't hurt you necessarily. You go just to the intermediary
goal, sub-goal that supports the goal.
Okay. But now there might be other constraints on that, right? Like cost.
It's not only that goals can be contradictory, like I want to go east or west.
It can also be I can get a higher reward, 10% more than baseline,
for 20% more metabolic cost.
Now that might be a bad deal. But if I just go for my utility,
it might be an acceptable increase, or it might be I have to take a certain risk of damage.
In order to obtain a certain reward. So it's not only that they are opposing
within the task domain, if you want,
but they can also be opposing on different dimensions of survivability,
to call it that. This is an excellent question.
I don't think there's one answer to that.
If risk-taking is an interesting point, I would say,
if I had to make a blunt statement, that you take only risks if you really think
that your chances in the future to get that goal are not that high.
So risk-taking will be higher if you are in a bad position. It will be lower if you are not.
That's very natural. I think that would also, if you write it down properly,
emerge from a mixed evolutionary reinforcement framework. work.
I would expect an agent that is very confident of continuous and steady growth of power, not risk it.
In fact, utility, if you look at utility of course, they look typically risk
averse when you are in positive and high positive values, but they are risk
friendly when you're not.
The other point, and this is very interesting, is what is more important,
the goal or information saving?
Now, I would answer it this way.
If you are somebody who is trying to.
Run through a door because he wants to get catch the train
um so he runs to
the door and just you know tries to get in the middle of the
door so they have enough space and doesn't bother getting stuck a little or
something like that and tries to get get the train that's it so he probably
will go for information saving yes he wants to be fast but he doesn't have the
tire and hasn't practiced it but imagine a sports an olympic sport which is
running through doors in the shortest time
possible reaching a train that leaves in exactly 15
seconds and people train for 10 years to put
the olympics off running through catching the train
catching the train i bet with you
these guys will not care about the information cost they will
basically run through the door in a way that will optimize their
throughput so they will run through the door or whatever gripping the
handle with a hand so they can swing
around in a very specific way and train probably to
leave exactly at the angle of 63.5 degrees so that
will be exactly propagated into the just
closing door of the train in other words yes if
i do better infinity then i don't care about saving information right but if
i'm in my default behavior that's one of many possible actions i do and that's
the typical behavior for lending organizers they're not playing olympics usually
then i take the one that's cheaper.
Right. Okay, so we have constraints now on how to optimize information processing.
And already you indicated that embodiment itself can be not a source of constraints
that help you to optimize information processing within the decision-making system.
But now there's sort of a hidden assumption there that the world is actually
Markovian so far. Thank you.
Right? Not exactly, no, no it's not.
So we have done that for systems where in principle we're at a Markovian.
If you do a relative information according to the formulas we've shown,
what we do is the sensor is actually sub-Markovian.
So you choose a sensor that just picks out the information it wants and that
creates a non-Markovian world.
However it doesn't care. So the information in the models we have looked at,
the sensor essentially will be less information-carrying than the world,
but it has the freedom of choosing information it wants.
But you know the upper bound for your information stays constant,
right? The upper bound is a full… Because the world is predictable.
In principle, yes. However, you can do the same thing with a limited sensor.
And what you get there is typically that your relevant information goes up, goes up, not down.
Surprising result, Christoph, Christoph Salge, you get more relevant information
if your sensor is incomplete, because you essentially quality of information is worse.
So you can't choose the information you want to get a worse set of information.
So you have to look at the bigger part of it to get a same quality of information.
But that would mean that, let's say, locally you have less information,
but let's say collectively over all your sensors, you gain information.
Not necessarily. You gain what we call piggyback information.
Piggyback information is information that's not useful for the original goal.
But you have to collect it to be able to reach your original goal at the desired level.
It's kind of what, whatever you order, you order a laptop from a company,
you don't get just a big box with lots of styrofoam, which you don't want, but you get it.
And it's a bit like that. So this extra information comes with it.
But the extra information is correlated with the goal relevant information.
It's not uncorrelated. It is correlated, but you don't actually,
you could, you could throw it away, could throw it away and just keep the core valuable information.
But the problem is throwing it away is, is, is a waste. You process it already. It's already there.
So can you do something with it? And we claim that it gives you an opportunity for acceptation.
So I'm using it for other purposes than original, the original goal.
So open-ended evolution and that it may be a driver for pushing sensors to the
maximum refinement without requiring this to be an explicit.
Evolutionary pressure which would answer why you may have very good sensors
although there's not an obvious reason why you'd have to have a maximum resolution,
right so I get that but why I brought up this idea of the predictable world
or this Markovian assumption is that,
in terms of a normative framework where you want to dictate in some sense the
principles along which the system has to optimize itself,
maybe in a Markovian world those principles are collectively different than
in a non-Markovian world because in a non-Markovian world I am forced to explore.
Okay, yeah. To a larger extent and following maybe different procedures than
I can do in a predictable world, right?
Now, if I'm forced to explore, this might compromise my optimization norms for
my information processing system.
Absolutely. So how do you see that trade-off between then the ability to explore
in an unknown world or partially unknown world while optimizing my information processing?
I would like to make a comparison with business.
In business, you have the components. You have basically the null money flow.
And you have the investment. And investment is what we would call exploration.
If you have lots of extra resources, you can invest a lot in trying to basically
reach out to new markets.
But sometimes you don't have the reserves, and then you just live on your metabolism.
You don't invest anything into exploring.
So I don't think there's a single answer to that, because exploration is not
per se a value. It's a value because of two reasons, the world does change,
and we may want to be ready for this change.
Because things may also, we just have the, we can.
We have the resources, the extra resources that we can use.
Like Trisco of Columbus used when Spain finished the war.
Or the Reconquista, they had extra ambition, extra money. Portugal was already
starting to explore navigational routes.
And Spanish thought, okay, we have the extra money, let's invest it.
But I think if you operate on the verge of survival, you don't explore. You just try to survive.
Or if you're at a very, very, very well exploiting niche, there are some examples of that.
One issue I see there is that your exploration norm might actually work orthogonally
to your exploitation norm.
And this is, of course, a conflict that you have to resolve in some way if you
talk about the controller, right?
So, of course, you can then say, well, under survival pressure,
exploration will be minimized.
But at baseline, let's say, you might want to explore so you actually know how
to escape in the future more efficiently or what have you, right?
So, in some sense, it also means you might want to break stable states in your
optimized information processing in order to, let's say,
identify new models by which you can describe your environment.
Which in some sense relates to the question I had in this morning about three-year-olds
having absolutely no memory about the period before that time.
Because there's some state transition there in human memory,
right? so the information processing optimization might face a similar catastrophic
forgetting phase that is absolutely required to abduct into a new level of operation.
I do think that this is an interesting question. I view the process of doing
stuff as you have a point, this is where you are, say, and the question is,
do you have knowledge about the environment of where you could move to?
This lateral or virtual
knowledge of what could happen if I
would move there I think it's very important of whether you want to explore
or not so if the other options are pretty good there's no reason why not send
out some species or spend some
time in nearby solutions if you know the solution is just the best then,
exploration becomes a problem it's like you have a company which is incredibly successful product.
And it's very clear that any modification of the product will,
not be good and that does happen and this company has a big problem actually finding then a new niche,
they have this problem um so why i'm taking the company because in the company
there are sentient beings who control it and they can actually do a forward
model evolution as far as we know is limited before the bonds perhaps there's
some local forward volume can do in and sexual selection, but it's very limited.
So it would really depend if the optimum is sharply defined.
I think you have a problem with exploration.
Exploration can only happen in big steps, and I don't see, for example,
a local algorithm like evolution doing that.
Humans do something else.
When humans explore, they, because of the superior wiring or the more intricate
wiring of what we call M, the memory,
they can modify the topology of a search space and suddenly things that are
far away become close so I can, when the moment I have Newtonian dynamics suddenly the concept of,
ballistic flight is something that's close to what else would be just a local
trying to get something somehow right now suddenly I can make a prediction,
yes I can throw a stone to the moon in principle if I do it with the right type of energy,
In other words, this is a qualitative transition in this concept space,
which we cannot see in evolution.
Evolution doesn't have it. So in evolution, everything must be somehow locally visible.
There must be local hints that an exploration.
Can be successful if there is no local hints then poor
species will be going extinct if something
changes we can see that in cycles of of
say um parasite host cycles
that are very very strongly linked or
orchid ears and and hummingbirds which
are very very tightly linked and they can't actually
separate to take out one member of the
echolo ecological web that will never disappear is
no no very unlikely they
will end up right so then um okay
so now we have this is the informational perspective on if
you want optimal decision making but now you also brought in this notion of
empowerment which sort of is a complement to this informational perspective
so what is what is unique about this empowerment notion and where did it come
from the original idea came it's it's very funny to say is it came originally
from this robot football project.
We had the wish to model agents that go to the ball and kick it without having to tell them so.
So we wanted not to avoid an external reward function.
So the idea became more formal when we introduced the perception action loop
and the information of U.
It was clear that this model would be very naturally presented or represented
this idea by having a potential set of potential.
Actions in the future how much change they could possibly invoke in the environment,
in other words how much can you influence the environment and it was very clear
also that you need to see the influence if you can't see it doesn't count and
that was very natural to say,
basically from social sciences the concept of empowerment Empowerment means
that people, disenfranchised people, for example, realize they can change the situation they are in,
and they can also perceive that change.
The perception is also important. It's a subjective measure.
And it turned out that this measure turned out to be surprisingly effective.
We tried it in various scenarios, and something like 11 or 12 different scenarios,
is what it seems to really do produce behavior, motivated, self-motivated behavior,
which does not require an external reward function. It basically produces goals.
You give it a dynamics, and it gives you goals, more or less.
The idea behind it is, if you have an organism, how does this organism choose its goals?
Of course, there are some fundamental goals, like finding enough food and meat
and so on. These are fundamental goals. And it comes from that what you do.
And it seems to be that empowerment. So maximize your options because that maximizes it.
The states you can reach in the next step if some states go away or niche go
get smaller it increases your chance of getting out of this situation and that
seems to work surprisingly well so the motivation was can we understand from
an evolutionary part of you with very,
limited assumptions how organisms can generically create their own goals when
there is not a very clearly define goal-like eating or fleeing a predator or something like that.
So that means if we take as an example tool use, right?
So I encounter an object and empowerment would then tell me,
well, you can learn that this object will have a certain affordance.
That means relative to your morphology and your goals, you can achieve a certain
objective with that, right?
So that's what you do. You don't necessarily learn just the local properties of that object.
You learn how to embed it within your own affordance repertoire yes but now so empowerment then.
Allows you to incorporate objects but how well does that indeed again scale
to same issues also within informational sense because depending on how you process this,
how you represent this how you segment across different objects you might have capacity
limitations yes so how does it scale well
empowerment itself as it's defined it doesn't
care about cost so it's really as you said more formal to the other view you
can combine it you can put a cost or a kind of cost limitation on how many action
sequences or action potential actions you want to consider and when you do that
we get interesting results namely dominant strategies strategies,
which are particularly effective in changing the world.
So you won't probably remember some kind of weird wobbly movement that happens to move somewhere.
You will remember really a clear, well-directed, well-established movement that
clearly changes the world to one state rather than to another.
So that's something that actually this limitation gives you.
We have also various tricks and algorithms and approximations how to calculate
empowerment, also the continuum.
And this is being developed because now that it's established that empowerment
really does a lot of cool behaviors, it's worth investing and actually scaling it up.
And some tricks allow us, for example, to push empowerment forward many hundred
steps or something. There are quite a few tricks, it's not, it's really,
really drastic approximations, but they give you qualitatively.
Uh again sensible results and this
is what an organism needs an organism will not optimize this function
to the very best it wants some kind of thing that
works that's good enough it's good enough yes but now so for empowerment also
the way you conceptualize this it's like injecting information into the world
and recovering the result right and then in something you can frame it again
in a compatible framework of information processing.
But now the exploration that I have to engage in to understand what this object
might contribute to my goals will take a certain amount of time.
So how rapidly does such a process converge and how does it also depend on the
degrees of freedom that the object would afford?
Learning is not yet part of the model except for this one example that I showed
you with a pendulum, where it actually learns how to model the forward model.
When you say a concrete goal, we have not yet linked concrete goals to a goal,
which is something that needs to be done, of course.
But in real life it's also similar. Imagine you play some new game that you
just learned the rules of.
I won't mention goal, you probably know how to play, I'll mention focus,
which is a really nice game.
We once did it many years ago as an exercise for our students.
And the point about the game is that none of the students knew the game.
We didn't know the game. There are no libraries, opening libraries.
So we really had to learn this game from scratch.
And it was very interesting. In the beginning, it just looked like random walks.
You do something, something happens.
And after four or five games, you as a human player start to see structures.
You start to see, oh, this does this, this does that.
This does that and you start to pick up the
salient points and this is where empowerment would come in it would basically
say these are the salient points of the world of course it doesn't solve the
problem of actually beating your opponent but it structures it it tells you
okay these are the points from where i can try to see whether i can beat my
opponent so the argument would be it creates broad.
Road map or milestones which tell me this is where I want to be and if I want
to have control over these and these states of the game.
And then you can ask, can I get to the goal there? So the argument is always, you are local.
Your understanding is always local. Do these.
Landmarks in my mental map give me a hint how I'm getting closer to whatever
goal I may want to achieve?
And this is, in my opinion, how we are able to learn very abstract games or
math or things like that, that we get these landmarks where we go to,
and then we start mapping out where from these landmarks are sub-landmarks,
and which sub-landmarks are conceptually closer to where we want to go to.
It's purely hypothetical, but I think that's the way to probably look at empowerment.
Empowerment itself finds the landmarks, the main ones, doesn't find the goals,
or it creates the landmarks as a goal.
But of course, when you have a specific goal, it just may give you a way of getting there.
So but now so if we look at these two frameworks right for this
information theoretical framework and talk about optimizing information processing linking
sensory states to actions um and then
we have this more embodied action-oriented empowerment
view right and they're orthogonal but empowerment is something that also challenges
the informational view because empowerment is also telling well there's a lot
of information really in the embodiment in the action in the world that is offloading
the informational processing that is going on.
So maybe this whole emphasis on this very centralized view on behavior,
where it all has to happen in this cognitive engine that is optimizing informational,
maybe this is so far at an extreme of our search space, right?
If we talk about embodied action, that maybe your empowerment notion will sort of invalidate it.
How do you see that? Well, it's an interesting question. Why do we have agents at all?
Why is there a concept of an agent in physics?
Why did they emerge? In my opinion, it's because in a way, some type of information
likes to be accumulated.
So essentially, like wants to like, if you like.
An organism, why does it want to procreate?
Frankly, I think the reason is because there are some processes that basically
parasite, or parasites in a way, on the physical world.
And these parasites, they like to continue doing so.
Because that's what a parasite does. It wants to propagate its unique way of life.
Even if you, the physics doesn't care. So it's not antagonistic either.
If a parasite parasites another organism, that other organism may not like it.
And so that will try to get rid of it, of course.
So then it comes out of realistic and it's meta level.
But I don't think that you can make a unique statement on, oh,
empowerment is one that's giving you the one perspective, the other one is getting the other.
They operate and they may have different parameters, different time scales.
For example, the increase of your bandwidth of processing is slow.
You can't just make your brain twice as big.
Unlikely, perhaps with CRISPR we can at some point try that.
I think it would be ethically unquestionable, but in principle not to try.
What's fast? Empowerment is relatively fast. You need a forward model.
If you look at information preservation and information saving, that's something we do.
Subconsciously, when we learn something, we use up a lot of bandwidth. with.
Once we know how to do it, we use very little bandwidth because it's probably
rewired, reorganized in such a way that it will eat up less information.
So I would say learning to grab a complicated object or handle a complicated
object will be very bandwidth intensive, takes a long time.
And it's translated somehow, it's rewired in a
way that internally will use less information
so in other words this this process happens
all the time and what the time constants are
and what the weights are it's not something i
would be able to make a speculation right but now if if we try to map these
these this concept um to physical systems like the brain would it imply that
the brain tries to optimize its mean activity level is it really that isomorphic,
or not. No, no, no, no.
I think the way to think of it is slightly different.
If I have two brains, one uses a lot of information processing to do something,
the other one loses very.
The other one has an optimized way of doing it. It's clear that the other one has an advantage. Why?
Because it can handle other tasks too. It can learn additional tasks.
It can concentrate on other tasks. It can react to danger faster.
So it has lots of advantages indirectly. directly. So, both brains may be, for example, two twins.
One twin has learned how to play tennis many years ago. The second twin just
learning it. They play against each other.
Well, who's going to win? Of course, the one which spends less time thinking about his moves.
Very simply so. That's because he has had the opportunity to impress and squeeze,
and optimize its information flow.
And he can also perhaps even talking on the phone and upsetting his twin brother
this way, when the twin brother is sweating and trying to catch Justin to ball.
So in other words, the advantage is not necessarily energetic.
It's advantage in many dimensions.
Information theory is basically saying just, with this resources,
if you have them, that's how much you can process, and that's how good you can process. Right.
So Daniel, you also, in parallel to your theoretical work, you're now also the
president-elect of the RoboCup organization,
which of course is an understandable concern because this is also very much
about testing a lot of these ideas in the real world.
But why do you invest effort and time in sort of advancing this notion of RoboCup?
Why is that so important to you?
I do believe that we have several advantages by having that.
First of all, we have a direct comparison.
You can essentially come up with all kinds of algorithms which work in a lab
under bearing control conditions.
But at the end of the day, they will have to work in the field.
And you can compare, does it actually work? I can't predict,
say that it works if I have these and these and these and these constraints.
But on the field, there's always an excuse. Either it works or it doesn't.
And you see it immediately.
You see, oh, this guy has, this group has an excellent vision system or this
group has a very good walking system and then you can learn.
And even if you don't copy their system exactly, you can pick up.
Ideas at various levels of abstraction, either concrete code,
if that's being published, for example, which happens in some groups and leagues,
or else by seeing, oh, this is a concept that essentially everybody else is now using.
Let's take a very simple example. Omnidirectional drive used to be a concept like that.
At the beginning, it was not obvious for the midsize, so you can later everybody introduced it.
It's a very simple example, but there are more intricate ones.
The second thing is, I do believe that a lot of interesting questions emerge.
You have a relatively clearly defined task, a relatively clearly defined world,
and yet it's very complicated.
You have to get several things to work at once.
And it's very motivating to think about, okay, what do I need in principle if
I want to have such a machine to learn something like that from scratch?
Of course, many teams write code to win the competition, so they have to be
very specific about how to do that.
On the other hand, I think it's a great motivator to think, if I have a robot
that is not just running forward, I mean, if we look at walking robots, that's what they do.
A football robot cannot just walk forward. It has to understand what a sidekick
is and when to do it, when to use it.
Okay, this is all done by hand, but in principle, the challenge is, what at all?
When you're a footballer you do
that by instinct you train also a
lot but you have many things that you just do on the moment
at the opportunity that you do and I do think that this context switching that
happens all the time is one of the major challenges of AI so I think if we can
address that in a proper way so we can move away from hand crafted behavioral
rules to a more automatic.
Autonomous autonomous decision of how to switch contexts from,
say, walking to snopping to kicking to whatever,
we will have made a big step ahead in AI.
And finally, my personal view, that's a very, very personal view,
it's not official or anything, I believe that we need new materials, new algorithms.
And the new types of embodiment for robots to be actually able to achieve such
a level of competence where the big goal is 2050 to actually play and possibly
win against world champion,
that it will push this uh envelope much more strongly than if we say yes we
need soft materials but yeah at some point when when it's ready rather having
this perspective gives you an incentive to actually try these materials a bit earlier, of course.
But when will the first robot team win the Champions League?
Well, the Champions League is beyond 2050.
In 2050, the goal has been declared, playing against the world champion,
the human world champion, and win.
It's a very ambitious goal, but let's put it this way, when it was declared
in 1997, people really didn't believe that's even possible.
There were hardly any humanoid robots in labs. There were probably a handful
of labs in the world that could afford a humanoid robot. And today,
humanoid robots are everywhere.
So, even that already was a huge push ahead in terms of making robotic science more democratic,
more popular, and actually sell people that, yes, it's possible to make a humanoid
robot that walks and doesn't fall down all the time.
But do you think the main challenge is in the biomechanics or in the cognition and the motor control?
Everywhere. I think biomechanics is a big issue. Energy is a big issue.
I think having an energy so that the role can run for 45 minutes in undrafted is massive.
So biomechanics, energy is a major issue. But I think the cognition is also
a major issue because you will have to contend with players like say Messi or
that really are flexible in their thinking.
You can optimize in a particular situation something that you can shoot a penalty
shot without fail, essentially, assuming the hardware doesn't break.
So you could be better than humans on a penalty, for example.
It might be possible that we would beat humans on very specific swap tasks.
But in a generic game situation, to make the right decision,
hopping up, risking life and then to kick a ball above your head into the goal.
That's something that humans do.
And good let's call it chilean here
right yeah yeah yeah exactly and then essentially doing
that as a human player uh requires a
lot of guts and and nerve and instinct and
not every player can do that so it's very clear that it's a very special skill
but now can a robot full player in the current competitions get a red card there
are ways of getting fouls but right now the fouls are relatively mild.
It's basically blocking the goal and things like that.
In the future, and for example in the simulation league, they introduce already
fouls. So if a player I'm not exactly sure how they implement it, but it's automatic.
If a player kicks another player without going for the ball, I think then it's a foul.
There are certain rules that Roger implemented.
And the RoboCop is a bit like the Robot Olympics if you want, right?
Even though there's a separate competition also with that name.
And the Olympics also continuously change the disciplines that are participating.
So do you see also a RoboCup that might be changing that will further expand
into other domains? Maybe soon we have robot basketball or robot tennis?
There are changes. First of all, in the main leagues they are actually,
they become harder and harder.
That's why if you watch the games, sometimes the games look less interesting
as time goes by because they become much more hard to follow.
So football ball for example they took
away the colors of the goals they took away um lots of
structure from the field the ball has no color
anymore so it's really very challenging problem um
other leagues come and go so there are leagues
who emerge it's not the robot legged league
that was the robot sony ibo robots um that league was um introduced um i think
the first demo games were with 98 and it disappeared later when it was superseded
by a standard platform leak,
which is basically wrong. That's when you see the malware works.
So in this case, you have a development of the leaks or a disappearance of leaks
or emerge of new leaks. Like we have a logistics leak.
We have an at home leak, which is concerned with making robots more flexible,
so flexible they can deal with problems of home robotics, which is,
of course, a huge challenge.
It's much easier to develop a robot for industrial sets than for homes,
but those homes are notoriously unpredictable.
Would you feel that something like a robot war leak would be fitting?
Well, first of all, I must say that war is not a very nice term.
Set of tests. And the second thing is, first of all, it's destructive.
I find that a little bit unsettling.
And also it's an issue of ethics in general. I don't think that we want autonomous
robots to know how to destroy other entities.
I think that's where it gets problematic.
But the other thing is that Robot Wars exists, and it's a remote-controlled
league, so it's not autonomous. So in other words, RoboCop is fully about autonomy. You want autonomy.
Why I bring it up is that maybe the real challenge here is about building moral robots.
That's our real challenge. Because I think on the midterm already,
we really have to master how to build robots that are autonomously and truly,
let's say, assistive and moral in their behavior.
Because to build a destructive robot is easy.
But our challenge is how do we control this and how
do we make it transparent and what i'm worried about is that
right now the robot league the robot war competitions are
a bit sort of in the public media of military organizations of course looking
into these things it's completely outside the realms of transparent analysis
and debate and i think that's even a bigger risk so i was wondering
whether it would not be making sense to maybe have a league where robots can do damage,
but they manage to not do it autonomously.
Because this is what we have to master, and we have to drive this debate as researchers.
We cannot leave it to non-academic institutions to do this behind closed doors.
No one knows where it's going. And once it hits the streets,
we have no frameworks to deal with it.
So that's why I was wondering whether it would not be an idea to at
least start to think about how to also incorporate this even though it is a
painful issue it is an insulting issue sometimes but we cannot close our eyes
for it i would say and uh it's really nice that you bring it up in one of the
statements i made before i was um basically selected as the coming.
President is that i think that road ethics is a major point that should be discussed
in the I think it's an excellent opportunity because even in the football game,
how much damage I'm ready to do to my fellow player if I want to win.
So it's already appearing there. It's very clear that the concept of fair game is already there.
I think that robot ethics has the same problems as human ethics.
How do you prevent them? This is an example I wrote many years ago and repeated,
and unfortunately, reality has caught up with me.
With my example, I was saying, how do you ensure that a passenger airline pilot
does not take the plane and crash it somewhere?
This was an example I actually brought up. And you don't.
You don't know. You can't see into the head. You believe that socialization
helps, that you know the person, that you trust the person, and you believe
that they have a self-preservation need and so on.
So there's a whole set of safeguards which we assume.
But when you take them away, when
they disappear, when people don't pay attention, bad things can happen.
So I don't think that robots will be exempt from that however I think I see
a way forward for making robots more ethical and that's very simple it's actually
the same thing that humans need to do to be more ethical namely,
basically a generalization of the concept of empathy and there's just for an
example for how that could look like and again there's just pure motivation.
There's nothing well developed it's just the first glimpse Christian Guglisberger
from Goldsmith and Christoph Sager from Hertz,
they have developed a model of basically NPCs,
so players you have a video game and they are accompanied by a pet or a companion
and the problem is these companions are usually quite stupid they act really
in a stupid way and one thing that they did was using empowerment cross-empowerment,
as a value function for the pet, for the companion.
So the companion tries to maximize the empowerment of its master and its own
too. So it tries not to die, of course.
And when you do that, it's very interesting. For example, it will shoot an enemy
that endangers the master.
It will behave in various ways in a sensible way.
One thing, for example, that it shows how almost human-like it reacts,
empowerment looks at and takes into account all possible actions and so the pet,
has by default assumes that the human could be a psycho and kill the pet so
the pet has also a sense of self-preservation so it will accompany him and help
the human but it will stay out of his shooting direction unless you turn on
a trust flag in which case.
Basically the pet will trust human right and will enter his shooting shooting direction,
So here we have empathy, of course, because the pet has a model of the human
and what the human could be doing.
And if that model is wrong, then yes, I have a problem.
But at least it's very clear where empathy and trustworthiness and probably ethics come from.
It's basically knowing what the other one needs and not acting too much against. Right.
My prediction there is that what we're going to see is to build machines that
know how to handle trust will be orders of making it more difficult and to get the biomechanics right,
i'm not sure about that um i think trust is a state of mind so it's a belief
state it's basically saying what do i believe will the other one do uh it's
related to game theory um common knowledge advantage.
For example, if I am a player and I have an advantage having a pet,
my pet knows that I have an advantage having it and so on, then the trust can emerge on its own.
If I'm a new player, basically a newbie, perhaps the pet will say,
no, this guy doesn't know that yet, so I'll wait and see how he behaves before I trust him.
And that's in real life it happens to you have a team of people and
one person is new to the to the business a new
boss you don't trust him immediately you see how he behaves and turns out okay
he knows what he's doing he has a good model of the future and then you start
trusting so in other words i don't think that this is i mean the practice will
be very difficult of course as all learning is but i think conceptually,
I don't think we are that far away from that, conceptually. The practice is a dimmer store.
Biomechanics is, we are really far away. So, I don't think I agree with it. That's fine.
But I think the other issue that's really important here, why I think this should
be included in RoboCop, is that if we look at the history of our science,
we know there's no value-free science.
And the mistake science has made over the ages is to develop technologies and
knowledge and then leave it to others to figure out what the ethics of it is.
This is not working. So if we, as the researchers behind these kinds of machines,
are not actively engaged in that debate, we will not have normative frameworks when we need them.
Because, for instance, we go to bioethics or general ethics of human behavior,
we have no normative frameworks. We're stuck.
And I feel that just for that reason and also given this historical consideration
Consideration of science is not value free.
So it's also us, the scientists, who now have to engage with that.
So for that, I think it's important that gets included.
But now, so you have a broad set of interests.
You're driving this whole RoboCup community now forward into the future.
2050, you're going to beat the world champion. It's great to have predictions
that only need to be tested by the time we are in a retirement home somewhere.
So no one can blame us for making predictions that fail. But now,
if we would like to follow in a tradition that you represent,
what would be the Polanyi law that we have to adhere to?
Well, I wouldn't formulate that as Polanyi law. I think it's a very simple law,
that it's much older. That's the golden rule.
Don't do to others what you want be done to you.
But actually, the rule needs to be generalized. Because a machine that can offload
its memory onto a big computer has a completely different view on survival than a human.
A human turned off will not be turned on again.
A machine turned off can very well be turned on again.
And so I would say the goal rule needs to be generalized.
Don't do to somebody else what they themselves, according to the best model
that you can have from them, would not like to be done to them. Okay.
So it's a law of empathy, essentially. Yeah.
And then three years from now, we're going to go visit you with Anna,
who's waving at us there behind the glass.
And we're gonna we're gonna check
whether the prediction you're gonna make today was confirmed or
not so what's the the one non-trivial prediction you you would like to see tested
three years from now that you're gonna see confirmed three years is a short
time can we increase the period four that's too small ten no come on let's do
it like this compromise three and 10.
Oh, that's not a compromise. I've made it hard. Well, I think I'm happy to make predictions.
I'm not so happy making putting times on it. I'll tell you. Okay.
Because I think that discoveries are power law distributed.
So it's like avalanches. You know, it will happen or earthquakes that will happen.
You don't know when and how big big thing.
So the prediction that I think is that we will have to completely.
Not completely, but significantly expand our understanding of how to create
contexts or switch contexts if we want proper AI to merge, rather than highly specialized,
highly optimized, one domain optimized.
That is something that I'm sure. I would not put a number on it, probably say,
no i'd rather i'd rather not give a number to
that because this this is not that may happen so it's more
an aspiration of let's say uh if you
want the general intelligence of this context independent robocop they can play
football and basketball uh for example where even in robocop that can't play
football but where i don't have to encode um how to switch context between the
from a stance to a kick to a defense, et cetera.
If I don't have to do that myself anymore, I would say we made a huge step ahead.
Not sure whether it's enough for what we would call AI to be complete,
but I would say without this, it will not happen.
Okay, great. Daniel Polanyi, thank you very much for this conversation. Thank you.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Program.
For more interviews, recorded lectures, or upcoming conferences in the field
of biometrics and biohybrid systems, go to csnnetwork.eu.
Music.
And thank you for listening.