This is the Convergent Science Network podcast.
Leading researchers in the domain of neuroscience, brain theory and technology
are interviewed by Paul Verschoor and Tony Prescott.
This is Paul Verschoor with the Convergent Science Network podcast.
And today I'm talking with Peter Gardefors, who's one of our speakers in our summer school.
And Peter, you presented your work in generalizing a theoretical framework that's
called conceptual spaces towards language.
So what's this notion of a conceptual space exactly?
It's a model of how you represent knowledge. I mean, I'm a cognitive scientist
and I want to understand in some sense how our minds represent knowledge.
And we have different traditions.
I mean, you have the symbolic approach of the early AI and then you have the neural networks.
My conceptual space is something in between, because I work with geometric structures,
with metrics, distances, vectors, and that kind of notions to model different kinds of knowledge.
So, can you give me an example of that? Well, the normal example I use is the color space.
We perceive colors, but we organize colors along three dimensions.
I mean, there is the hue, the color circle going red, blue, green, yellow, and so on.
And then there is the dark and light.
And then there is the intensity going from gray. So there's the three-dimensional space.
And that's how we perceive colors. And it's fairly well established in psychophysics
that we have this spatial arrangement of colors.
So, but now in some sense, I could say, well, if I talk about an object having
a certain color, like an apple being red.
Then this apple might have a number of properties. So it will be round,
it will have a certain taste, a smell, a color, and so on.
So how do I bring that together in a conceptual space?
Well, one of my key notions is that of a domain. I mean, we have several domains
that we organize the knowledge.
Color is one, size is another, shape is a third, temperature, and so on.
We have a lot of, some of them are based on perception, some of them are based
on action, some are based on our social grounding in the society.
So, there is this general notion of domain that is used to sort up the information.
So, you would not so much see it as, let's say, dimensions spanning this space,
but a number of domains, or is it equivalent?
Each domain could consist of a number of dimensions, and color domain is three-dimensional,
and taste domain is four- or five-dimensional.
So, are these domains like a Kantian prior? You're just born with it?
Partly I mean it's my theory is neo-Kantian if
you really want to bring in the philosophy Kant said
we are born with space and time I say that we are born with
we are disposed to have representations of space we learn time much later but
then of course there are many domains that are dependent on us being in a culture
and learning to discriminate new things and so on so our set of domains are
expanding as we grow up in a society and they can
change over time so it's a mixture if you want it's a
mixture so there's a so do you is there really a
a defined set of prior domains no no no no i mean we are born with certain sensory
organs that that determine some of the basic domains like color perception like
perception of heat and the tastes and and and so on but on top of this we build
a number of domains i mean I mean,
we extract other types of information.
For instance, I mean, I have been emphasizing our perception of forces.
We are quite good at seeing forces in people's emotions and so on.
Right. Okay. So now we have our domains.
And then you say, look, if I now see an apple, then I have a number of domains.
I might have a visual domain and a haptic domain and olfactory domain and a taste domain.
And now across these domains, apples are a set of data points.
In that multidimensional space.
So I have a cluster of data points and I say all these data points together,
this cloud of data points, this is now Apple. Yep.
It's a point in a very high dimensional space, but the space is organized along
a number of domains. Right, exactly.
But now, would there be a way to, let's say, reorganize that space?
Like, for instance, you might at time, you might sometimes discover that,
let's say, Santa Claus doesn't exist.
So suddenly one organizational element of my reality is sort of,
has disappeared, and now I
have to reorganize all my data. So how does it work in a conceptual space?
Well, first of all, you can add new domains. You can learn about apples having
nutritional value or something like that.
What happens quite often in learning new things is that you change the importance
of domains, which are the most important in making the classifications.
So in biology, maybe the behavior of an animal was more important when you classified
something as a fish, but then came the biologists and said, no,
it's the skeletal structure or how you feed your kids or whatever.
Other variables that are more important in zoological classification.
Classification yeah but then i was more worried about
the case if i lose a domain not about adding domains but if
i lose one that's why i thought about santa claus yeah
but santa claus is not a domain i mean it's a fictional fictional object
yeah i don't know any good examples of losing domain so yeah well imagine that
i believe that uh let's say objects move physical objects move in the world
because they have intentional states yeah and then i discovered that actually
stones don't have intentional states.
Now suddenly, the way I frame the world is changing. Okay, I would describe
that as the domain that maybe does not disappear, but you assign it a very low value.
It becomes a zero in your concept classification.
To take an example from chemistry, you had some idea of the caloric dimension
that was used to classify chemical stuff.
And then suddenly, there was
a revolution in chemistry and the caloric dimension totally disappeared.
I mean, you can find examples of that in the history of science.
In the history of human perception or in human classification,
it's maybe more difficult to find such changes.
So now I have this cloud of points in apple space.
If you understand your proposal, and if you want the center of gravity of that
cloud of data points is now my prototype of an apple, right?
Yes, and that cloud is, I make the assumption that it's a convex region.
I mean, a concept, Apple, corresponds to a convex region, meaning that for any
two points in the region, all points in between are also there.
And that is very useful in helping us understand how we can learn concepts to
have this notion of convexity.
But it does mean that you clearly have a very empiricist view on prototypes
and concepts that are really derived from, if you want, data points obtained
from the world. Yes and no. It's empiricist.
And once you have the domains, it's data-driven how you divide the spaces into regions.
But the domains are, well, the neocantrian position that they are partly given, partly learned.
Okay. But now, how should I think about the development of these?
Imagine I want to add a new domain.
So the discovery of a new domain could be the result of just adding data points
to my conceptual space and discover that I cannot organize these data points anymore.
And I have to say, aha, I should now assume there's another domain.
So how does that work? How do you go from single observations and their accumulation
to this decision? Okay, new domain.
That's normally tougher in individual. I mean, here in our society,
we have scientists who say that we need this dimension, this domain,
in order to understand the difference between this and that. I mean, so I...
You can do it as an individual, but it's very tough.
I mean, you're living in a culture where there are new ideas about what it means
you need to understand something.
Right. But do you see this in the same in relation to, let's say,
the standard notions of assimilation and accommodation that Jean Piaget would
talk about in development?
You can interpret it in that way. I mean, he didn't never say anything about
how these processes work. He didn't have any model of assimilation and accommodation.
Assimilation would have been just collecting data points, and accommodation
would be changing the space, I mean, the structure of the underlying space.
So I can model these processes that way here. Okay, but now at the core of your,
and this is also what you said earlier, at the core of your approach is really
a notion of a metric, a topology,
and also a quantification of similarity, right? Right.
So can that really generalize to, let's say, all different levels of, let's say, concepts?
No, that's a good question, because our concept classification is very much
dependent on our perceptual mechanism. That's the basic classification.
But then we can introduce more abstract concepts, different means,
and in particular via language. I mean, you have mathematics and in law you
introduce new concepts by definitions.
And we use language to construct new concepts. And there you may get out of
the geometric constructions.
I mean, I don't know if I can apply my methods there. But they are dependent
on being grounded in these more low-level concepts.
You can't start from the bottom by defining concepts. You have to ground them
in some kind of perceptual domain.
Okay, so you're saying the conceptual space is exposed if you want to both feed
forward or bottom up influences perception, sensation, and top-down influences
like high-level symbolic systems related to language.
And in some sense, what you're saying is that what is most clear for you at
this point in time is how this feed forward component actually operates.
And then the top-down element is sort of for the future. Yeah.
Well, I have one area that I think is very useful to create new concepts is by using metaphors.
So if I mean, to take an example from science, when you introduce the notion
of electricity, you have these notions of current and voltage and so on new
dimensions that you don't perceive.
And in order to understand how they work, you can then compare electricity with
water running or whatever.
And you get some kind of grounding in the perceptual domain in that way.
But the electrical dimensions, they live on their own in one sense.
You can formulate them mathematically.
If you go to domains like, let's say, machine vision.
There are lots of methods around for data association. So these are basic different
kinds of methods, adaptive filtering methods and what have you,
that try to explain how you can bring data points together in high dimensional space.
Is that a rather direct expression of your conceptual space notion or is there a difference?
Well, you must distinguish between technical solutions of dimension reductions.
I mean, you can have lots of data sets and you can use multidimensional scaling
or principal component analysis and so on to reduce and pick out the most important dimensions.
But that's a technical solution. Then you have the more biological solution
where you really try to connect the dimensions to some kind of perceptual mechanism
or to your motor systems or whatever.
I mean, and that's two different methods.
Okay. All right. But they're not necessarily... You could imagine that any of
these... Some of these methods would be rather directly implementing these notions of a conceptual space.
Yeah, yeah. Okay, so there's no fundamental problem there.
No, no, it's just a question of how biologically realistic you want to be in
your implementations. Right, exactly.
So now we have this idea of a conceptual space.
And you've been working on that for quite a while. Yeah, yeah.
And today in your presentation, you were emphasizing two, let's say,
extensions of this framework.
So on the one hand, you wanted to capture a notion of action,
because so far you had looked more or less at static concepts,
like apples, and on the other hand you want to go to language.
So let's start with action. Well, I want to go to language as kind of application of these two.
I mean, I want to use conceptual spaces to model the cognitive representation
of actions, and I want to use them to model the cognitive representation of events.
How do we think of events? What is the structure? Just like we have discovered
the three-dimensional color space.
I want to understand what is the action space in our minds. Right. Okay.
So in a sort of boneheaded way, which I usually tend to follow,
I could say, well, no big deal.
I mean, also action I might be able to classify in some high-dimensional space.
Let's say different limbs involved or whether it is
just between agents or
it's a single agent or an action on an object or whatever right
so so why is that straightforward generalization
not enough to classify action what's missing well
what kind of data do you start from i mean you you can get a lot of data points
about the human human body moving but that's a very high dimensional set you
have to reduce the data somehow and you can do use as i said just technical
ways of doing it But you could also use data from psychology on how we perceive actions.
And then there are experiments showing that we have a kind of hierarchical representation of our bodies.
I mean, there is the main body, then we have arms and legs, and we have forearms
and lower arms and upper arms, and we have hands and fingers.
I mean, you have a hierarchical representation.
And an action is a movement in this hierarchical structure. So it's a fairly
high-dimensional vector, dynamic vector that represents.
And you need a more condensed way of understanding what an action is.
I think that our brains need that kind of reduction.
So it's interesting, right? Because that would mean that you're saying,
well, the notion of conceptual space is like a core representational engine,
which is modality independent.
And then dependent on the perceptual filtering, I can now exploit this mechanism
in different ways. So, this would be for action that you're saying,
well, to map action into a conceptual space, it's basically how I filter action.
So, I should maybe not look at action in static terms, but I should look at
more at its dynamical property. So, how am I going to do that?
I mean, if you see a person walking, take walking as a typical example here,
you don't care about the clouds of the person, you don't care about whether
it's hot or cold, you care about the movements of the person.
So you abstract away a lot of the domains.
And what remains in my mind is that you look at what are the forces that the
person exerts on his or her body parts.
So I see an action as a pattern of forces.
That's my reduction of the domain for actions, to focus on the forces.
Okay, so now, so also you showed in your talk that, for instance,
humans actually need very little, few cues to extract motion,
right? Like biological movement.
And it's also been shown by others like Martin Gies and other people.
So how does that help me to get to a notion of force?
Well, I mean, first of all, these examples, I mean, it was Gunnar Johansson
who started with this patch-like technique.
You only put small lights on the joints of your body and that's enough,
the information you get to identify actions.
And you identify them extremely quickly.
I mean, it takes 200 milliseconds to see what kind of action it is.
It helps because it shows you that all the other features are not necessary
you don't have to see the surface of the person moving,
you don't have to see the color all the other features are gone so you see the
kinematics and then of course you could say okay this kinematics that is,
what you perceive in action but I go one step further and say that no it's the
changes of the kinematics I mean if you talk mathematics there's a second derivative
that bring out the forces forces.
I would say that when we identify an action, it's more using the forces give
us a more coherent and simpler representation of actions than looking at kinematics.
Okay. This is interesting. I want to say, look, we have to have a direct perception
of change in the kinematics, that's one thing. That's the second derivative.
The direct perception of the change you call a force.
But if I would look at that from, let's say, an engineering perspective,
I could say, hey, wait one moment, but that would not map necessarily onto the
forces I'm really exerting on the degrees of freedom of that walking human, right?
So, how do I relate these two now?
Okay, that's a good question because you have the perceptual inputs when seeing
somebody walking and then you can exert the forces on the body limbs.
On the other hand, you have the kinesthetic experiences of your muscle control,
and you know that you have to stretch your arm in a certain way to push a door or whatever.
And I'm making a tacit assumption here that these two systems map onto one another.
This is basically like when I understand when you're talking,
I'm somehow representing how you produce the sounds, but I also have to map
them on how I produce the sound.
It's the same kind of perception and motor action mapping. Right. Okay.
But that would mean I would have great difficulties to understand the walking pattern of a bird.
Because the bird has a rather different kind of body than I do,
at least according to me.
So now the interpretation, the grounding of the forces that you call them,
I observe, will be difficult.
So how do I circumvent that problem?
Well, we understand the movements of other animals. We do understand ourselves.
So there is this generalization problem. On the other hand, once your brain
has learned to extract the second derivatives of movement, you can then see
the patterns in movement.
You can see that the patterns of sparrows flying are similar.
They have much quicker wing flapping than if you look at an albatross flying.
It's very slow and much more forceful wing flapping than in a sparrow.
There are differences in the patterns, even in birds, and we can learn to identify them.
But now, wouldn't there be an other way to interpret this where we say,
well, what we really learned to classify are these derivatives.
And we can call them, let's say, kinematic dynamics, something like this, or kinematic change.
And these changes will, in the
end, be brought about causally by forces operating on degrees of freedom.
But from a perceptual perspective, I don't care about these forces.
I care about kinematic change. Okay. So what would be wrong with just saying,
look, why don't we call this kinematic change for now?
It's a bit of a tough question, because now we've been talking about biological
movements as the only type of actions.
And we humans are not very good. Well, we are good Newtonians in one way that
we can do these second derivatives.
But our minds are also full of other types of forces, like in social interaction.
I mean, I know that somebody is my superior. He or she has a force, can control my actions.
I know that I'm attracted to a certain woman. and we describe attraction as a force as well.
So in our understanding of what causes actions, there are other types of forces
than the traditional Newtonian forces.
That's why I want to use forces rather than just the movements,
the kinematics or the dynamics involved in biological motion.
But that's interesting, right? Because then what a prediction could be,
You say, look, our perceptual systems, as applied to, let's say,
many different phenomena, different levels of complexity in the world,
are, if you want, assigning a notion of force.
They are just inventing, let's say, a pseudo-causal relationship behind the
change in the world we observe, which can be completely beside the real causes
of what we observe. Sure, sure.
That's a good interpretation because our brain is adding these,
let's call them theoretical variables or hidden variables.
But this is just like in visual perception. I mean, our eyes are adding the contours of objects.
I mean, on the retina, there are no contours, but our brain is adding that in
the interpretation of the world.
So when we are looking at an action, we are adding the forces as a kind of contour,
if you like, of the perception you have.
It would be like a dynamic contour. It's a dynamic contour, exactly.
So then that means we should not get too confused about this notion of force.
No, no, you shouldn't restrict it to the physical force.
But there's an interesting prediction there, right? Because it really means
that this might be a way in which the brain is also imposing an interpretation of the world.
And by virtue of that, that we are so good at recognizing biological motion with minimal cues.
Yeah. Okay. No, I mean, since you mentioned Kant earlier, I mean,
he was saying that we can't help by seeing causes and effects.
And I say we can't help by seeing forces. I mean, that's part of doing the causal effect relation.
Exactly. Yeah. No, that's very good. So now we have an idea about how to deal
with motion perception of action.
But in some sense, it's more like a classification, right?
Because now we have these changes
in my posture, let's say. I change my posture. I call this walking.
And it has a certain gait. and then you say, well, and I can detect and classify
it because I know how this gate was changed over time, right?
I lean on one leg first and the other and so on.
But then I could say, yeah, but that's not action because to talk really about
action, there must be an intentional component in this.
This is just movement. So how do we get from movement to action?
No, I don't. Okay, that's a philosophical point. I mean, some philosophers would
say that an action involves an intention. I don't use that.
I don't restrict actions, the notion of an action to intentional actions.
I mean, no, I don't. I mean, so I have a more general, maybe the term is not
appropriate, but that's how I use it.
Okay. Now, look, I'm not actually going to bicker over it. I just want to understand
whether you saw a division there between, let's say, movement pattern action,
but basically you say, look, as long as we're changing the body,
that's what I want to recognize.
Yeah. Okay. So now we have a way to directly perceive features of movement so
that we can map them in the conceptual space.
Now, what are the domains of that conceptual space to organize representations of action?
So what are the domains? The domains are the force patterns.
I mean, these are the forces.
They can be complicated because you have a body moving as a complicated structure
of forces. So it's a pattern.
But the domain is still this more general notion of force.
And my claim is that when we classify actions, I shouldn't say it's only that
because there could be constraints coming from other things.
If I hammer something, I use an object. I can't be hammering with my nose.
I can be hammering with my hand, but that's a metaphorical use of hammer.
Hammer means that that kind of action is involving an instrument.
It's constrained by having an object that functions as a hammer.
Maybe not the best example, but something in that direction.
But still, what are the domains?
What's the dimensionality of this space? Is it the limbs I'm using?
Is it the direction of movement in some Cartesian coordinate system?
What are these core domains? Is it body types?
Okay. We've been talking mainly about biological motion.
A car has a sort of pattern for you to accelerate and decelerate the car and so on.
Of course, in physics, there are generators of forces, motors or muscles or whatever.
But I think that when we perceive actions, when we categorize action, we extract from that.
A robot walking has a totally different system of generating the forces than
a human walking, but we would still classify it as the same action.
My claim is that we can abstract distract away from the mechanism behind the
forces and just focus on the forces.
Okay. All right. So, but it still remains to be seen how that force space is exactly structured.
Yeah. Yeah. Okay. So now we've actually… Well, it depends on the parts of the
acting individual, the acting object.
I mean, how many force generators there are and how they are related.
Like our muscles in our body, for instance, which is complicated.
But it's interesting, right? But if you take the car example and let's say ourselves
motoring around, if you want, or navigating the world, indeed,
at an abstract, more Cartesian level description, you say, okay,
I'm changing my position in space.
And that's the level where these forces act. And I don't really care whether
the wheels are turning or the legs are moving.
So this would make that point about abstraction. Yeah. Okay.
So now we have an idea of conceptual space, how we can use this to...
To understand action or describe, represent action, heavily relying on,
let's say, a transformation of action in the real world to an internal sense of force, if you want.
But now you also elaborated the same framework towards the notion of an event,
which is rather critical in how we deal with the world because we don't only
have dynamics, we also have an event.
I have not been able to find very many cognitive theories of events.
There are lots of philosophical theories, but if we think about the cognitive,
I mean, The most naive description of an event is something happens to something.
And then this something I call a patient. This is what is in the focus of the event.
And then there is something that causes a change. And there is a result of this.
I mean, well, in most cases, something happens.
So I divide an event into two vectors.
One describing the forces that apply to the patient. and the other vector describing
the change in the patient.
Sometimes the change is null, but it can be a state.
So my basic model of an event are two vectors, one force vector,
one result vector, acting on a patient.
Very often there is an agent generating the force vector involved in the event, but that need not be.
It can be just gravitation or some other non-object generating force. Right.
But now, so this is nice, right? So we have the notion of event and we have
decomposed it in, let's say, an agent and a patient,
which let's say are some core entities who are sort of physically present, defining the event.
And then we have exchanges between them, which are the forces.
And these are the two vectors, right? So we have a cause and effect. fact yeah um
so but are these is that really the
minimal description of an event we might
have events where there's no agent that is causing anything
yeah the minimal is is a patient and
and and and a force and a result vector that's a this is my hypothesis i mean
this is a theory and there has some some consequences uh in how we perceive
actions for instance i mean we would We'd be very surprised if something happens without the cause.
I mean, that's the Kantian notion again.
I mean, if we see something happen, we presume that there is some kind of action
pattern going on behind the screens. Right. But if nothing happens?
We can still describe an event in which, in some sense, nothing happens.
Well, there are two kinds of nothing happening. One is just that there is no
force, and consequently there is no change either, no result vector.
So it's just a state. And that's a special case of an event,
a fairly boring case of an event.
But then there are also events where there is a force and a counterforce that
balance each other, so still nothing happens.
So when I'm pushing a door and the door doesn't open, I mean,
I'm exerting a force and the door exerts a counterforce.
Nothing happens, but there is still an interplay of forces and there is still
an action from my side of pushing the door.
But the counterforce brings out no effect.
But if the event is, let's say, more abstract, where we say,
in 2012, the Summer Olympics happened in London, took place in London, right?
So does that mean that I would have to decompose that in all sorts of microscopic
elements where I can now again recover these forest relationships?
Relationships in this case it's a lot of intentionality going
on i mean lots of people involved in this event and their
joint intentionality constitutes the creation of this summer olympics it's a
very complex event involving i mean a complex factor of of mental causes not
physical causes generating this event i don't know how to analyze this in detail i mean i stick to
the more concrete actions.
Right, okay. No, but this is interesting, right? Because this issue of generalization
is a challenge right now for the framework.
Yeah, of course it is. I mean, going up to these more abstract types of events
that we talk about that are generated by our societal structure,
by our interactions with other people. I mean, I don't really have a good analysis.
I mean, this is the kind of program I have. I hope to be able to extend it.
But I start with the more basic concrete actions which involve physical forces and so on.
No, but moreover, of course, it's sort of an easy game to sort of throw these
pot shots at you and say, oh, here, I have an exception.
So that's, I think, not really the point, right? It's really the point to try
to understand where would be, let's say, principled transitions of the approach.
That's why I thought Olympics might be sort of abstract enough that that could be challenging.
It is. So we can now decompose, let's say, events in a patient.
Why did you use the word patient for, let's say, the core object in the event?
Well, that's something that undergoes the change, so to speak.
I mean, sometimes the agent is identical with the patient. When I'm walking,
I'm exerting a force on myself and I'm changing my own position.
So there are cases where the agent and patient are identical.
But in many cases, we can separate them, or in most cases.
Okay, so is there any psychological or neuroscientific grounding for this decomposition of an event?
That's a very good question. I don't know a very good answer to it yet.
I mean, there are some people who are speculating about, I mean,
we have two visual pathways in the brain.
I mean, the more dorsal is going for the motion patterns, and you can think
of that as picking out the kinematics.
And maybe, I mean, maybe, I don't know if you can find some correlates of picking
out the forces, but I have no idea.
And then you have the more eventual pathway going to object identification,
which is more involved in the static properties of all the objects.
So you have a little bit of division of the forces and the objects.
But I mean, that's on a very rough and general scale. I don't know if there
is more detailed knowledge here.
But now, so in the face of that challenge, in some sense you're using very physical
metaphors, right? With force, cause, effect. And is that not limiting the framework?
Because as soon as we talk about these more, let's say, metaphorical causes or about intentions.
It might become confusing if we want to think about this as a cause.
If I ask you, please give me this cup of coffee, Peter, to really start to think
about that in terms of causes can also get very confusing.
My larynx is doing stuff, sound pressure waves, hit your cochlea, et cetera.
And then so, oh, but Peter grew up in this culture where these words mean certain
things. But it sounds like a very, very unfruitful way to pursue the question.
So wouldn't it be more useful to replace the notion cause and effect to something a bit more neutral?
Well, that's what I'm trying to do. I mean, by having these force vectors or
result vectors, that's, in my opinion, more.
It's a way of reducing causes and effects to something that I can describe in
semi-mathematical models.
Okay. But then force would then be broadened to something like a metaphorical force.
Yeah, well, as I say, social forces, emotional forces would be included in this
notion. But are there other concepts we could consider for this?
Maybe it's a bit of a boring question, but it can lead to so much confusion,
right? If we use these physical metaphors.
Yeah. So I'm just wondering whether there are other candidate concepts for this
we could consider or not really.
I don't know. No, I mean, some people talk about powers rather than forces,
social powers and so on. I don't know whether that's better or not.
I mean, I don't care very much about the terminology here. I mean,
I care about what kind of models I'm using.
And as we talked about earlier, I mean, my use of action may not fit with the
philosophers' uses of action.
But that's not your main concern. That's not my main concern.
Okay, very good. So, okay, so we got events sorted. it.
And also, you would make the point that this decomposition of events would hold,
in principle, at any level of description. So whether I talk about the cup falling
from the table or me writing a paper,
the basic decomposition of agent, force, patient, and then a resultant vector
or force would hold. Yeah, that's a minimal description.
Then many events involve more components.
I mean, if I hit something with a hammer and I have the instrument that's
between my force exertion and the force that happens to
the object I'm hitting uh and i can have
uh when i give if i give something to
you i mean there is a physical movement of the cup if
i give you my coffee but there is also the the transfer of possession i mean
which is a much more advanced and intentional part of of the of the action so
there are there are different details so different components you can add in
in a in an event description but the The basic ones are the two vectors and the patient.
I mean, and I would say that this is a cognitive theory, so that these parts
appear in all our representations of theories.
Then you can go down on finer details and add more components,
depending on what level of description you're aiming for.
But now, if we decompose events in these terms, you could imagine that we could
expose humans to, let's say, event descriptions. We put them in a scanner,
fMRI, and we see which areas of the brain light up.
And then hopefully you see four different areas where you can say,
okay, agent and two forces. And is there anything like that?
I think so. I mean, people have been looking at how verbs are represented in
the brain and we are getting into verbs now.
I would look for differentiations between these force vectors and the result
vectors. If the brain lights up in different areas, depending on whether you
talk about the causes or whether you talk about the effects.
I don't know if we can find any clear results. I've taken part in a small experiment
on this where there are some indication that we can make such a division.
But there is a lot of research to do on this. I mean, what happens and does
the brain distinguish between causes and effects in its analysis?
That's, for me, a very interesting question, but I don't know anything about
the answer. Well, actually, very basic learning mechanisms like classical conditioning
have been interpreted in terms of the brain extracting cause-effect relationships from the world.
Because you get exposed to, let's say, the tone and there comes the foot shock.
And in some sense, you might interpret that as a causal relationship. That's true.
So there have been relatively interesting theories about this.
And it would be very consistent with what you're proposing.
Okay, but that's an area I don't know very much about. Okay,
yeah. So, indeed, but now you mentioned language because this,
I think, is sort of if you want.
The long-term ambition of this program is also to account for language.
Yeah, I mean, that's, I shouldn't say the main application, but the application
I'm working on right now. But why do you call it an application?
Because I use this model. I mean, it's a model. I can use it to analyze causes
and effects, how we perceive causes and effects.
I can use it to analyze how we use verbs. I can maybe use it to analyze social interaction.
I don't know what. But at the moment, I'm focusing on how this maps onto our
understanding of language.
Okay. Okay, but before we go to language, there's an interesting implication of that, no?
Because then you're saying, well, from a cognitive perspective.
There's a core meaning system organized in conceptual spaces.
And this meaning system is sort of modality independent, and now it can be expressed in language.
It can be used in how we interpret the world through haptics,
vision, blah, blah, blah.
But this is like an independent meaning module.
Yes, well, independent meaning, but it is a kind of core module.
I mean, the event representation, I think, would be at the core of our understanding of the world.
And it depends on, I mean, you can have it to analyze perception.
You perceive an event, but you can also use it as a basis for formulating linguistic expression.
You can use it for different things. And as you say, I mean,
it's modality independent.
I mean, I don't assume a separate module for language semantics and another
module for representing perceptual concepts. I mean, for me,
they are on the same system. But that's interesting.
The really important consequence is I can use this meaning system to also make predictions.
Oh, yes. And I can, because it's modality independent, I can make cross-modal predictions. Yes.
Right? So I think that… And these cross-modal predictions show up in language
in terms of metaphors. Right, exactly.
And they show up in body movement, in how we gesture or how we mimic things or stuff like that.
Okay, so given that we're both very enthusiastic about this program….
How far did you get in accounting for language with it? Well,
not to the details. I mean, I've got into some of the rough stuff.
And if I start at the very basic level, I mean, all languages,
I mean, the syntactic structures of languages are quite different,
but all languages seem to have noun phrases and verb phrases.
I mean, that's the basic thing you can have. And that division,
I mean, linguists never explain why noun phrases and verb phrases exist.
I mean, why are these the basic building components of language?
They are taken for granted.
Now, given the model of events that I have presented, known phrases map onto
agents and patients primarily.
They might map onto instruments and other stuff. But basic uses of known phrases
is to denote agents and patients.
And basic uses of verb phrases is to denote force vectors and result vectors.
So this division between the two objects and the two vectors maps onto the division
between noun phrases and verb phrases.
Okay, but how clean is that mapping?
I don't know how clean it is, but if it serves as a grounding for learning language.
I don't think I can explain all features of the semantics of a human language
because that's very rich.
But if I can say something about the general principles of how the semantics
is structured, I can use that to explain how children can learn a language.
Because the data you get as a child is not very rich. You need some kind of constraints.
And these event structures, if my model of the cognitive representations of
events are correct, they give you a structure that constrains what words can refer to.
I mean, I make this basic distinction between noun and verb.
Phrases and verb phrases.
But now, of a verb, you said, look, a verb is essentially a convex region in a single domain.
Yeah. Right? So that seems a rather strong statement.
Yeah. I mean, before we get to that, I mean, let me start by saying that a verb means either.
It either refers to the force vector or it refers to the result vector.
That's right. You don't have a single verb. I mean, you can have compositions
with prepositions and stuff like that where you get. But the root of a verb,
is my hypothesis, refers to either the force vectors.
And these verbs are what we call manner verbs, how you do things.
You hit things, you pull and you push.
And then you have verbs that refer to the result vector.
And these are, well, results like painting or heating or whatever, opening.
But wait, if it's a verb, it always requires an agent. Right.
Um but if it's an agent it
requires that um but look
if we have to the patient now and the patient
undergoes a change let's say uh you
you hit my finger with a hammer my finger turns purple
the result vector is
not necessarily an action as i can
capture in a verb no no right well you say
turns purple i mean that's uh you have turning here means
and then you have the color term so what happens is
that turn it means a change or a movement no but
you agree i could describe it in a way that there's no verb involved i
said now yeah my finger is blue
well the property change yeah there's a property change there's a property change
so how so how do i well on it this is not in itself an action of the patient
right it's a property change yeah it's a property but it's that's the result
vector and it's described i mean you can say turns red turns purple,
you can say becomes purple.
We have those verbs that help us in saying that there is a change in a property,
and purple is a property word.
But you're saying it's going from some other color into the region of purple.
So there is a change in the result. The result vector is going from some region
into the purple region. That's the result vector.
So what I was after, I was just trying to understand whether you're saying verbs
describe change or verbs describe action.
No, no. No, there are two kinds. There are verbs that describe actions.
These are the manner verbs. And then there are verbs that describe changes.
And I say there is a fairly tight division between them. There are some verbs
that can be used for both topics, but in a particular sentence,
they're used either to describe the manner, the force vector.
Right, but I was trying to understand, let's say, the overlap between these two domains.
Because you could say, look, you lift the hammer and now the patient ran away.
So now I'm acting so I'm not turning blue anymore so how unique is that mapping
of these two kinds of verbs to either the cause or the effect,
I would say it's fairly of course there is a particular action can have very
many different outcomes as you say depending on the situation on the context
but we don't have verbs that summarize causes and effects I mean this is This
is my constraint on how we learn these verbs.
They either express the cause, I mean, the force vector, or the effect of an event.
So that's a kind of cognitive prediction here.
For instance, you could be running towards me, and I'm running away. So we're both running.
The agent and the patient are both running.
So the verb we use to now describe the cause and effect is the same.
Is that a problem? No, because you use the preposition away,
which introduces a result.
Run away is, you modify the manner by running away that introduces a change
in space, which is a result vector.
By having this preposition added to running, you're changing it from a manner
vector to a result vector.
Okay, but that could also say Peter's running and Paul is running.
Yeah, but then you don't have a cause and effect.
Okay but then
then it's two events but you did bring in the proposition now
so then it becomes let's say a conglomerate yeah it's
not only the verb anymore no no no it's it's i mean there
has to be at least one vector and one one uh editor patient i mean so that's
the noun phrase and the verb phrase are the minimal elements of of an event
description okay okay so it's a phrase it's not just a single word no no no
it can it can i mean then you add grammar but i mean you're and all kinds of
compositions of descriptions.
But what is denoted by these composite expressions are agents and patients.
Okay, so now we have these two kinds of verbs, and then from that you came with
the prediction, if you want, because, okay, if I look at that from the perspective
of the conceptual space.
It means that if every verb is indeed in the convex region, they cannot sort
of be in two disjoint regions at the same time.
And therefore, let's say cause and effect verbs, there are no occurrences of
verbs that are both cause and effect.
No. And as a matter of fact, you can make it even stronger because among the
result verbs, they only refer to a single domain.
So when you're heating something, you're changing the temperature.
When you're moving something, you're changing the spatial location.
And when you're painting something, you're changing the color domain.
I mean, I had previously a theory about but adjectives that say that adjectives
refer to regions in single domains.
So you have the color words, color adjectives that refer to the color space.
You have the temperature, hot and cold, I refer to temperature, and so on.
Now, I'm generalizing this single domain hypothesis also to verbs.
And there are lots of potential counterexamples.
I don't know how well this hypothesis will hold up. But if it does,
it shows a very nice parallel between the structure of adjectives and the structure of verbs.
Right. Which would be nice from a cognitive point of view.
From the point of view of cognitive economy, I should say.
On the other hand, you have some wiggle space, right? Because you have a free
parameter in your model, which is the domain itself. Okay, so that...
So the point is, even if you find counterexamples, they raise interesting questions
about really the structuring of these domains.
Well, I'm saying that at least that the same domains apply to adjectives that apply to verbs.
And we find these nice mappings between movement verbs and position adjectives and stuff like that.
But now one problem I had with that prediction is that I could also argue,
well, look, yeah, sure thing.
This is not a big surprise because, let's say, in general, words used in a language
that are ambiguous and confusing will just die out because they don't help for pragmatical terms.
So in that sense, could I say, well, this is like, this is the truism.
Because, of course, words in our language are only there because they're not ambiguous. Yeah.
I would say it's, well, you can think of it as a truce, but the reason is that
if you have two complicated words, I mean, words with two complicated semantics,
they would be very difficult to learn.
And having them sorted up into referring to a single domain makes them easier to learn.
I think that this is a good constraint that explains how children can pick up
language so quickly as they do.
That's one of the constraints, and it's not the only one. But your prediction
is that language have evolved the way they did, also to facilitate learnability.
Oh, yes. Oh, yes. And, I mean, there are these famous Chomsky arguments of the
positive of the data. We don't get enough data.
We have to have innate structures.
I don't think that's a very strong argument because he only considers syntax in his studies.
If you look at the interplay between perception and language,
as I've been doing in my studies of actions and events, I mean,
you can generate a lot more constraints on learning.
And that's what one of my aims is, to identify these constraints on how the
semantics of words are structured.
Yeah, but it's interesting about this argument about the poverty of the stimulus
originating in the 1950s.
That in some sense the data is still out there.
I mean, the data has not been conclusively summarized in some way that indeed
the sensory stimuli the growing child is exposed to is that restricted or not
structured enough to allow learnability.
But what kind of data do you have to support your position here?
No, I mean, the data I have comes from, to the extent I can confirm my models here,
that I have to look at, for instance, the verbs, to see whether this classification
between manner verbs and result verbs hold water, and that's a hot topic in
linguistics at the moment.
And if it does, then we have some
data showing that there is this kind of division of meanings in verbs.
And that division of meanings would help me, would generate a constraint that
can help me in explaining meaning why kids can learn language so easily.
Okay, but that would mean that the structuring of the conceptual space of language,
should then coincide with the progression of language learning.
Yeah, I mean, that's a give and take variation, yeah.
So how, but how do you really see that feedback mechanism? Because there's some,
there must be some bootstrapping in this, right?
Because the argument is always, look, if it's just feed forward and you grab
it from the world, it's not enough.
No. So what are these key rules that would help the bootstrapping in this?
Do you have an idea about that?
I have thought a little bit about the ordering in which you learn domains as a child.
I mean, you learn the shape of objects quite early. You interact with things
with your hands and your mouth and so on.
Color comes perhaps later. You learn about spatial relations.
But then there are lots of spaces where you only learn things later.
And I looked a little bit at child data on when certain domains are learned.
So color and shape comes very early. But for instance, all the...
All the domains related to knowledge. I mean, knowing, believing, and lying, and so on.
Children normally don't learn them until they're about three or four years old.
So that's a more abstract domain that comes later.
Or take economic terms. I mean, like a loan or inflation, to take a tough word.
I mean, you can't teach a child that. Bail out.
Because for kids, money are coins and bills. And they don't know about this
abstract space of economic relations.
That's something that they learn much later. So my idea of some kind of progression
of domains would be another constraint on language learning.
Some domains come earlier in their development, and then via learning a language,
via being part of a culture, you learn further, more abstract domains.
But then your prediction is that there must be a hierarchy of conceptual spaces.
A hierarchy of domains, yes. Of domains. Yeah. Okay. And would that hierarchy,
let's say, also be an explosion in dimensionality or not necessarily?
Well, each new domain adds dimensions, so it's bringing out more dimensions.
Yeah, but it could also collapse dimensions of preceding domains, right? It could, yeah.
No, I think in general, our minds are growing in the number of dimensions we
are using. Okay, all right.
So then you had a number of predictions, if you want, about verbs, right?
For instance, that similarities in verb meaning, or the subcategories of verbs,
or the subregions in the notion of walking.
So what are these predictions exactly? Well, we categorize.
I mean, one idea behind the concept of spaces is that similarity plays a great
role in categorization.
So we categorize colors as yellow because they are similar in our perception.
And we categorize actions because they are similar in our perception.
So I would say that running is very much similar to walking,
or more similar to walking than running is to flying, for instance.
Yeah, that's a good example.
And we use these similarities. I mean, I don't have a measure of distance between
force patterns, but somehow the force pattern of running is more similar,
closer to that of walking than it is to flying.
And we use these similarities when we judge the similarities between verb meanings.
And also, if you can generalize, I mean, take walk again.
There are a number of subdivisions of walking. You can be limping,
you can be marching, you can be strutting.
I mean, there are lots of kinds of walking.
But they all fall under the general notion of the walking domain of possible
walking patterns, force patterns.
But the subdivisions then are sub-regions of this action space.
Right, and that would then coincide with, say, our ability to distinguish these
movement patterns. Yeah, this is just like we have subdivisions of nuns.
You can have an animal, a dog, a terrier.
You have this hierarchy. We have the same similar hierarchy of action representations.
But you also had, let's say, a more detailed decomposition of,
let's say, how verbs translate to action in the world.
Because if we take the example of push that you talked about,
to push an object in itself is not necessarily the complete story, right?
Because in the result vector, different things can happen, right?
So does that mean I also have to think about then subcategories or let's say
decomposition of push into different kinds of resultant vectors or not?
You must make a distinction between what the verb represents and the event you're
describing because the verb represents a force vector.
Pushing is a force vector normally horizontally towards an object.
But then the object may be stuck or may be heavy or so nothing happens.
But that's part of the event I mean that's the result vector but
the push it as the verb
only concerns the force vector and the
result may be a complicated story depending on how the world looks like but
how do I segment this now because imagine so you push me and I'm pushing you
back so I'm your patient and you're my patient yeah then there are two events
okay exactly right so the segmentation of let's say um.
Real-world events, this segmentation would fall along the lines of the forces
and how they're exchanged.
But would that decomposition be unambiguous? No, not necessarily.
Does it need to be unambiguous?
You can have different levels of description of events.
I mean, if you take wrestling, there is a continuous exchange of forces,
but we still generalize this
general pattern of two people pushing against or pulling against each other
as a general type of interaction.
So that's a high-level description. Then, of course, I can have a more detailed
description of a wrestling match.
I can have, I'm bending your knee or bending your arm or pushing there or lifting there or whatever.
I mean, there are lots of sub-events in this global event.
But is the prediction of this possibly that if the forces cancel each other
out, or if the roles of patients and agents cancel each other out,
that that is a necessary reason to step to a next level of abstraction? No, no, no.
I mean, it's just as when you're describing a scene around you,
you can say, well, I'm in a room.
But I can also say I'm in a room with this and that pieces of furniture.
I mean, I can go down and talk about the details.
I mean, it's a level of attention we have on a particular situation.
Situation okay so so now you you came
a long way with the conceptual spaces approach to understand
language right so in some sense you're saying nouns are categories
yeah right adjectives are properties um prepositions are are force and spatial
relations sort of relationships yeah verbs now are forces and results causes
and effects vectors yeah right and then adverbs verbs are like modifying vectors.
Yeah. What would that exactly mean? Well, if I push something,
I mean, I'm exerting a vector, but I'm not saying very much about how strongly I push.
I mean, the strength. If I say I push strongly, I'm saying that the vector I'm
exerting belongs to the, well, with stronger forces, I exert more force.
So it's like a multiplier or a modifier
of a vector. Or I can change the direction. I say I push downwards.
That means I'm changing the direction of the pushing.
Okay. So that's a very complete, if you want, decomposition of this very basic
notion of an event in the atoms of language.
And then your proposal is that sentences as such, as a unit, capture the event.
Yep. But now to go from these linguistic atoms to a sentence, we need grammar.
Not necessarily. I mean, we can communicate without grammar quite a lot.
I mean, there is this thing that's called proto-language, that is just meaning
components without any grammar.
I think grammar is needed to disambiguate a lot of things here.
I mean, in English, for instance, there are a lot of words that can mean either a verb or an object.
Like hammer. I mean, hammer is a verb or it can be an object.
And the word in isolation doesn't tell you what it is.
But if you put it in a context and add syntactic markers, tenses or third-person
markers or whatever, then you can see whether it's a noun or a verb.
So syntax for me is a tool to disambiguate the uses of words.
And then syntax is another constraint on how to learn a language because syntactic
markers will help a child to see whether we're talking about an action or we're
talking about an object.
Yeah, but what I find surprising about this, and that's probably because you're
more aware about the intricacies of language, But I would expect that you would
have said something like, well, I have my basic decomposition of an event. I have an agent.
I have my two vectors and I have a patient. It's a cause and effect.
And it's that order of an agent cause and effect on the patient.
It's that order that translates to syntax.
No, not necessarily. Why not? Because there is an event or an element in between.
There is an event. But there is also you looking at the event and you focusing on something.
I mean, you're looking at what the agent is doing or you're looking at what the patient is doing.
That's two different foci on the event.
And that attention focusing shows up in language.
So in a large class of languages, the subject turns out to be the thing you're focusing on.
So if I say that I'm hitting you, I'm focusing on me doing the action,
but if I'm saying you are being hit...
I'm focusing on what's happening to you. I'm making you the subject.
And in both cases, I'm the patient. I'm the agent. You're the patient.
So what becomes the subject of a sentence is determined by what is the focus
of attention on the event.
Yeah, but exactly this example, I think, is a good one. I mean,
you might have reached a stage in this discussion that you want to hit me.
But still we say Peter is hitting Paul. all. So first we have the agent,
then we have the cause, and then we have the patient.
And if it's the other way around, we say Paul is hitting Peter.
So it's that ordering in the language that is rather directly correlated with
the ordering of your event decomposition.
No, not necessarily. First of all, there are all kinds of orderings between
subject, object, and verb in languages, so that doesn't.
But I think the most important thing is just what is your focus of attention?
That's what determines the ordering, because the thing most focused on comes first.
Yeah, but isn't that now a tricky direction to take? Because now you bring in
for the first time, let's say, a point of view.
And you say, depending on the point of view, you reorganize the conceptual space,
or you reorganize the event structure.
But it shouldn't really matter, because independent of the point of view,
the event structure always has the agent, the cause, the patient, and the event.
No, you don't reorganize it. You focus on it, And then you turn this focusing
into a linguistic expression.
So this is an intermediary step between your event perception and how you describe it.
And between your perception, there is the focusing of attention.
And that determines partly how you assign syntactic roles to the parts of the
event. Okay, so this is interesting, right?
So I find this interesting. interesting, because on the one hand,
you're saying, look, from, if you want a cognitive representational perspective,
I can tell you how an event is represented, is decomposed.
So there we go. We have the agent and our cause, the patient effect.
Then you say those components, I can map on the components of language.
Yep. Okay. Yep. So we're still doing fine. So this is like we have our conceptual
space, the domains, and there we go.
And we map this to, let's say, components of language.
And then now we want to make a sentence. And you're saying the whole sentence
describes the whole event. No, I'm not saying that.
I'm saying the sentence describes an aspect of an event. You have to pick out
a certain aspect of an event.
And you can start very basically with a subject and a verb.
But then you can add prepositional phrases. You can add modifiers.
You can add, I mean, I can say I hit you, but I can say I hit you strongly with a hammer.
I hit you strongly with a hammer so your eyes became blue and so on.
I can expand the description of the event by adding more and more parts of the event.
So that means that I have now a transformation. I have to look at some transformation
that goes from, let's say, a neutral mapping into a conceptual space of an event,
as it is in the outside world.
Then I want to translate this to language, and for that, I have to make a transformation.
And this transformation depends on the point of view.
Well, yeah, you make a selection, basically. But in making this selection,
I still map it into your basic event structure. Yep.
No? Yes. And to map that basic event structure in language, I still face the
same problem, which means there's an agent to cause a patient and an effect.
I mean, the event structure will constrain language.
It will not determine what you say, but it will constrain the structure of your language.
But then where's the magic of coming in for the syntax?
Because still, this is how I have to map it to my sentence. Yeah.
But, I mean, of course, as I said, that you have to have some way of disambiguating the role.
So you have to decide, know whether something is a verb or a noun,
for instance, or whether something is talking about spatial relations or whatever.
You can do a lot with proto-language without syntax. And when children are learning
language, they start with the proto-language.
The grammar comes, most of the grammar comes much later.
Right. And if you start communicating in another foreign language,
you start without the syntax. I mean, you have a vocabulary and you try your best.
So, I mean, this is the proto-language structure of communication.
For me, syntax is just a tool for helping us in getting more better understanding
how you made the transformation for the event description to the linguistic expression.
So, Peter, to wrap up, I have two questions.
So, I mean, you've gone around in this sort of very cognitive view on the mind,
if you want, and its different capabilities,
proposing this notion of conceptual spaces for a while now, looking at its consequences,
and that seems to be panning out rather well.
So what would be Peter's law that we have to adhere to in trying to understand the mind?
Well, one important thing is that our mind divides information into domains.
I don't have a crisp definition of what is a domain, but I can give you examples.
And this domain structure shows up in a lot of circumstances.
So that's one of the big messages I have. Okay.
And then the second one is five years from now I'm going to go visit you in
Lund and say, okay, you gave me this prediction five years ago and now I want
to know whether it really happened. So what's the one prediction you feel most
passionate about today?
A prediction, testable prediction that you're going to see tested and validated
five years from now. I have a lot of smaller predictions.
I mean, I have predictions about how the semantics of children develop.
I mean, I can tell you about the order of objectives that we learn, for instance.
I have predictions about what will be easy to teach to a robot and more difficult
to teach in terms of verbs.
Give me the most exciting one.
With the highest chance of failure. And I haven't thought about formulating.
Okay. I mean, yeah. Okay. The highest rate of failure is that nouns in general
involve several domains.
And when we talk about dogs, they have sounds and smells and sizes and shapes and whatnot.
Other words, adjectives, prepositions, verbs normally refer to single domains.
So, if I can validate this hypothesis, I will have a fairly strong toolbox to
generate further, more detailed predictions about how language works.
I'm going to have this kind of program of extending this single domain hypothesis
to different word classes.
That's a fairly general hypothesis. Exactly.
That's a good one. And if that works out, I… You'll be very pleased.
I will be very pleased, yes.
Okay. Peter Gardefors, thank you very much for this conversation. Thank you very much.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
Music.