Right. Are we on? We're not on. We're right. I guess he's not part of the interview.
This is the Convergent Science Network podcast. Okay.
Leading researchers in the domain of neuroscience, brain theory,
and technology are interviewed by Paul Verschure and Tony Prescott. Seats, are we running?
It's Paul Verschure with the Convergent Science Network podcast,
together with my colleague Tony Prescott, and we're here in the room with David
Reddish. And David, you presented us your research on what you call the cognitive rat.
So what were you having in mind there with the cognitive rat?
The idea is that if we can actually define cognitive functions carefully,
then we can actually go in and we can look at whether animals also perform those cognitive functions.
Particularly neurophysiologically.
So if we look at the information processing that you need within cognition,
cognition then you can actually find those and
we found i think several examples of that do you
mean that rats have knowledge in some tangible describable
way yes rats definitely have knowledge about tasks that they have to do about
the lives that they have can we can we define cognition here and a bit more
specifically so what how would you define cognition um i the definition that i've been using,
and I like to run with this and kind of see how far it will go,
is the idea that there is covert knowledge,
knowledge about things that are not immediately available to the animal, to you.
So if you, for example, think about something, you will mentally time travel,
you will imagine yourself into a future.
So one of the things we've been able to show, for example, is that rats can
mentally time travel. They can imagine other places and other times.
For example. But now you emphasized in the introduction to your talk quite a
bit on the one that it's historical antecedents, like in Tolman and then Hull.
And also you spent quite some time on really explaining, let's say,
the decoding methods that you have developed to actually make sense of the neural response.
So what is this historical context in which you're doing this?
Well, historically, of course, there's been a big question of whether or not
animals can think, right?
And whether they are more than just stimulus response creatures.
And that, of course, was a big fight throughout much of psychology for many, many years.
And um at this point i think of course lots of people not just us have kind
of come to the conclusion that there are in fact cognitive information within
that and we've come to that in large part because there are these information
consequences that we can see.
Historically you know i mean that's where i my understanding of the history
comes okay but then And so the decoding methods you use to actually interpret
these cellular responses that you find,
and particularly you have looked at the hippocampus, but you also talked about
other structures in the brain of the rat.
Why do you put so much emphasis on the methods you use to decode these responses?
Well, I think the key is to really understand. So to me, the key to the decoding
story is that we are in some sense reading the mind.
And I think that's the key here is that,
If we really believe, as I think the evidence is now very, very solid,
that the mind is the brain, and that in fact the brain is a physical instantiation
which generates this psychological construct we call the mind,
then we should be able to access the mind by looking at the physical properties of brain.
And particularly in this case, that neurons communicate by sending these action
potentials, these spikes, and we can listen to those spikes,
and then we could ask, can we in fact find what those spikes represent.
So the idea of the decoding is that we really have to, at some level,
once we understand that information, we understand how it's encoded, then we can decode it.
And at some level, the argument is we have to believe that decoding,
even when it tells us that the rat is actually thinking about some other place.
So you put quite a lot of emphasis on representations or knowledge in the rat
mind of things which are not present at the current moment.
Clearly because you want to distinguish between thinking about things and experiencing
them in a direct way. Right.
Is that...
Is that where you were going out in terms of cognition? Do you link it to memory
or do you go more towards reasoning, being able to reason and think about these
things that are in your memory? Well, I think it's both.
So one thing, I don't want to dismiss perception.
Perception is an extremely complex question, and there's a lot of very,
very interesting questions about how perception works, which ends up being much more complicated.
Like, as I like to say, we don't perceive colors, we perceive objects, right?
So how do you construct those objects? It's a whole interesting question there.
But most of the cognitive question has been, I mean, everybody understands that
animals can perceive objects. We know because they can manipulate them.
The question is, does the animal actually use information beyond what's immediately available to it?
And yes, the evidence is very solid now that it does. But to me,
that's the cognitive question.
So it's both a memory and a decision. In some sense, memory is decision,
right? The only reason we remember things is to make better decisions in the future.
I mean, otherwise, what's the evolutionary point of memory?
Well, that's, of course, only when you want to use the notion of decision in
a very broad sense. Yes. So I want to be very careful about the term decision.
I'm going to define the decision as any time the animal takes an action.
Then that's going to be a decision.
So also this would be, let's say, a reflex-driven action? A reflex is a decision.
And I emphasize that because the question then is not.
How do you make a deliberative choice? That becomes a special kind of decision.
But it becomes a what is the information processing happening when an animal does a reflex.
A reflex actually is a decision because if you're measuring,
for example, how hot your hand is relative to a stove, you put your hand on
a cold stove, you don't have a reflex.
Put your hand on a very hot stove, you have a reflex of pulling your hand away.
At some point, there's a threshold. This reflex system has made a decision.
Now, the question becomes, what is the mechanism by which the reflex has learned
that decision? It's learned it over evolutionary time.
And what is the mechanism by which the reflex has made that decision?
And it's much simpler than deliberating between which job you're going to take, right?
But they're both fundamentally, in the end, taking an action.
But then how many levels of decisions do you distinguish in the rat?
Well, I think that the way to think about it is not so much in terms of levels as processes.
And as I see it, there are four information processing sequences that are very
identifiably different in the mammalian brain.
And those four systems tend to track. There's reflexes, there's deliberation,
which is actually an imagination of the future.
There's procedural learning, which is basically learning to do a procedure.
I like to use the example of hitting a baseball or catching a football or something like that.
I guess in Europe, it has to be not catching it, but hitting it with your foot.
That's right, you got it.
And then there's this fourth system, which ends up being, I call Pavlovian.
I'm not sure that's the right word for it, but it's a species-specific behavior you learn to release.
And I call it Pavlovian because this is what Pavlov's dogs were doing.
Right they there was you salivate in
response to food and they learn that when the bell comes there's food
coming so they salivate so that's turns out those
four systems end up being all of the fundamental information processing to take
an action okay so you're saying then they're there and each of these systems
will have then their own quality of decision making yes but now for their own
neural structure sorry and their own neural structures you overlapping or uniquely Uniquely defined.
Mostly separate. Okay. Not 100% separate, definitely.
But they each have primary systems that are clearly quite different from each other.
Right. But then to push you into a decision-making definition,
also because you like to be clear about the definitions, right?
If I take something like a scratch reflex, right? Or if I take like an eye blink reflex. Mm-hmm.
A decision would imply that there is a sense of having options and that you select among options.
But now, if you talk about the scratch reflex on the frog, which has been shown
to be implemented in the spinal cord, once you generate that stimulus on the
skin of the frog, there's just nothing else it can do but just scratch on that spot.
So, there are no options involved.
Well, but in fact, there are options involved because the amount of stimulus
changes. So if you have very little stimulus, you basically don't actually touch
the frog. The frog won't scratch.
You poke it very sharp, the frog will scratch. There's some level in there where it will sweat.
Now, I agree, reflexes, there's not a lot of these variability.
It's not like deliberation where you're making from many, many choices.
One of the examples I like to talk about is the famous scene in Lawrence of Arabia where T.E.
Lawrence is showing off how cool he is and how tough he is by holding a match
and letting it burn to his fingers and not executing the reflex.
And so I like to give this example because what this actually is,
in my view, is a conflict between two decision-making systems.
A reflex system that wants to shake the match out before it burns to his fingers,
and a deliberative system that wants to stop and say, no, don't do that because
I want to show how cool I am.
So you have this, I mean, a lot of people talk about it as kind of a top-down
control, and I think it makes more sense to think of it as conflict.
But still a conflict that must be resolved one way or the other.
That's right. In fact, one of the very interesting open questions is how are
those conflicts resolved?
Right, exactly. And that's something that is not actually really well known right now.
In the history of psychology, this idea that the rat is capable of cognition,
of course, goes back a long way.
And the work you referenced, you talked about Tolman, but there's better known
experiments on, for instance, the cognitive map, the ability to take shortcuts, and so on.
But you're wanting to go beyond that ability to say that the rat has more cognitive
powers that are perhaps closer to things which you think of or have thought of as uniquely human.
Actually, one of the really interesting things about Tolman's original cognitive
map is that it's much more cognitive than map.
It actually got translated and really thought of in terms of space,
because of course, they were running rats on mazes.
And of course, once they had the discovery of place cells in the 1970s,
that kind of suggested, and John O'Keefe and Lynn Nadel's suggestion that the
hippocampus was the seat of this cognitive map, really started to talk about
it as a spatial paradigm.
But the original Tolman concept was much more a structure of the world,
and that the cognitive map was an understanding of the structure of the world
with which you could essentially imagine yourself.
And so it actually ended up being, as I like to say, much more cognitive than map.
But what has tended to happen in the comparative literature as I know it,
as people have said, okay, rats are great at spatial cognition,
but that's where it stops. Would you like to go beyond that? Definitely.
One of the things is that I think that the reason that rats are great at spatial
cognition is it's much easier to construct a spatial task that rats understand.
And I think it's very hard to construct non-spatial tasks that rats actually understand.
Primates really like pushing levers for buttons, right? That's kind of the ultimate
primate machine, right, is a soda machine.
If you do something, you put some money and food comes out.
Rats don't track that as well. But if you take these very cognitive things and
translate them into a spatial place, not so much because of the space,
but even just moving them to different locations, the rats can do very cognitive events.
So we've, for example, done a task in which animal rats have to balance delay
against food reward. How much food are you going to get?
And what we did is we actually made the animals run to two different locations.
And then we changed the delays in this complex contingency based on their choices.
And they can do that. They understand that contingency and their behavior proves
that they understand that contingency.
And it was very hard to do until we moved them to a spatial location.
It's not that they're doing the cognition in space, it's that the space just
becomes easier to train them to do that task.
But you might argue that spatial cognition is where the system evolved,
and then it became more flexible, perhaps, in the evolutionary path leading
to humans, so that we can now use these skills, but in a much more domain-independent way.
Possibly. I'm not sure. The data as I see it doesn't seem to suggest that.
But I'm trying to think if I can think of any good data offhand that's going
to do that. Well, we know, for
example, that rats see these same kind of cognitive effects across time.
So there are these cells in the hippocampus, which you both know about,
which are called place cells, which represent the location of an animal and
can be used to both imagine positions in an environment and to navigate within an environment.
But these same cells actually break up delays across time.
And basically, it's almost as if the animal has a spatial map across the delay,
which is a very cognitive event and is not spatial. It's actually temporal.
So there we see a very similar use of the neural system, but in a temporal aspect
instead of in a spatial aspect.
But you could argue that you'd get that for free because movement in space implies
that you have a temporal component, right? I agree. I agree.
And I'm not – one of the dangers, of course, is how much if you say,
well, it's for free, does that make it less cognitive?
Ah, sure. Right. But I think that's an important point, right?
That a lot of these things, these processes are evolved to work in such a way
that they can solve these cognitive problems by building on other components.
Right. But then it seems you would agree with Tony's contention that it might be co-opted.
For other kinds of functions and cognitive operations. Well,
so the big example, for example,
that we've been looking at has been this thing called mental time travel,
which is the ability to look to the future and to imagine yourself.
We know that when humans do this, they actually construct a complete,
essentially an episodic event of that future.
What am I going to be wearing? Where am I going to be?
Who am I going to be with? What's the world going to look like?
And we know that the hippocampus is very involved in that future construction in humans.
We know that the rats can construct spatial components in that future.
What we don't know, and I emphasize the word don't know as compared to no, it's not true, right?
We don't know whether rats, when they imagine that future, are simply imagining
the location or imagining the entire episode of all of the flavors and other components.
I suspect, given some of the data we've seen, that they are in fact constructing that complete future.
But are they doing it in a fairly context-bound way in that of the options I
have now, I could go that way and I can imagine mental time travel if I went
this way, but I can't imagine what I would do tomorrow or next week.
Right. So the short answer is, I'm not sure how we'd measure it, right?
And that's, to me, the real problem is that until I know how to measure it,
I don't want to say it doesn't exist.
That's been, you know, for me over the last few years, this idea of,
well, I didn't think rats could even look at the future until we figured out
how to measure it, which is why I brought up this whole decoding stuff at the beginning,
because it's that mathematical technology of decoding that allows us to measure
these non-local representations of this future space.
So the question is, how do we measure that episodic future event beyond space? And.
I mean, yes, I'm pretty sure that rats are not writing Harry Potter, right?
They're not, you know, actually constructing big fantasy novels that they can,
you know, about all possibilities. That's pretty clear.
But another, and so it's something you go back to the old question posed by
Tolman about these cognitive maps, whether they're local or global, right?
He was talking about whether they have sort of very delineated strips of information
in there or whether it provides more global information.
You're saying right now, given the data we have, we actually don't know whether
these episodic memories that the hippocampus forms in the red are local or more
integrated and contextual.
But is that really fair to say?
I would not have separated the local-global separation that way.
I don't think that's what Tolman's local-global distinction was, as I understood it.
I understood it being more, can you construct novel sequences,
novel connections within your map?
If you have a very thin strip of information and you're looking,
I know what this street looks like, and I know what stores are on this street,
and I have another street a block away that I know the stores,
but I don't know how to cross those.
Whereas if you have the global connection, you can do all those crosses,
which is the shortcut story.
Right. Right? One of Tolman's points was if you have a cognitive map, you can do shortcuts.
That's right. it's very, very clear that the rats have a broad map on which
they can do shortcuts and that they can actually connect up information in novel ways. That's solid.
Whether the information is more than context within a specific location,
that is, could they imagine a completely new non-spatial context connection,
such as, you know, well, I can imagine elves and dwarves and Tolkien and all that stuff, right?
I don't know whether rats can do that. But again, I don't know how I'd measure it.
And if I don't know how to measure it, I don't know how to disprove it. Okay.
But look, even though we might not be that clear about the boundaries of these
episodic memories rats can form, what you have shown us a lot of data on today
is this time travel component.
Yes. Right? So what are the key observations there that you think are providing
us insight on the ability of time travel in rats?
Well, the key is Mental time travel, I should say.
Right. The key is that the representations are self-consistent representations.
That is, the neural signature is not noise. It's actually a specific neural
signature of that other location. So the neurons at the current lo- that represent the current.
Good representation of that other place right so that's one
piece but practically that means you first
run the rat you identify place cell responses
you know okay here i have a set of cells that respond to location a
i have another set of cells responding location b and what you now observe is
that while in the future trial the animal is in location a the cells that correspond
to location b start to fire correct right this is the signature that's the signature
right so there's a few processes that connect that up to cognition and i think that it's important
that when we talk about it, this is really coming primarily in our data in terms
of this deliberation story.
And so what you really need to do in order to get there is ask,
what are the features that you should see in a deliberation event?
And those features are that, for example, that the signal should be ahead of
the animal, not behind the animal, because he's presumably deliberating about
what he's going to do, for example.
Well, that is an interesting assumption, right because.
There you're just saying, look, the spatial trajectory the animal follows will
also correspond with its future.
Correct. But that's more like a constraint. You impose the task.
But there's nothing that prevents, for example, the mental time travel from
going backwards, right?
Theoretically, the animal could just as easily be representing past events,
events behind him, right?
But practically, we see them being in front.
Okay. The second piece, which I think is really critical, is their cereal.
That is, it is representation of one side or the other, not both together.
And so it's not just that it's spreading activation into the future.
It's actually spreading down a very specific path.
And I think that's another key factor that suggests, actually suggests what
Tony's asking about, that it's much more of an actual episodic event, right?
It's this, what if I go right? What happens?
And then what if I go left? what happens.
So but then you do impose certain constraints with the task.
That means it's an alley that an animal runs through or an elevated platform.
So it's not that the animal can move freely in an open space.
Correct. Right. So do you see that as a limitation to the decoding?
I see it as a practical step to being able to see the data we've seen so far.
And I would very much like to do in open space.
There are physical problems in terms of running on a physical maze.
Because if you're going to say, have the animal come back, if you go to left,
you go the same distance left and right, you can come back.
But if you have a kind of 45 degree angle, the animal can't,
the path back will be too long.
So there's physical realities that we have to put the animal in VR to break.
We haven't done that yet, but it's certainly on the table.
Okay, very good. Well, we built technology for that, so we can talk about it.
But now you also mentioned that you might see these forward sweeps, but at decision points.
However, at the feeder sites, which in some sense is a termination point of
a behavioral sequence, you get a very different kind of dynamic.
Right, you have these sort of these high frequency events.
So are they significant for this idea of mental time travel?
Well, they are also good representations in the sense that they are consistent
representations of other components of the task.
So for example, again, the animals at location A and now the cells at A do not
fire and the cells at B do fire.
So again, we have this consistent jump.
What's interesting about those data is that they're actually sequences on the
track, and the sequences are both,
they'll go in directions the animal has never traveled, they'll go across paths
the animal has never actually traveled sequentially.
And so it might be, I know how to go from, you know, the house to the library
to the stadium, or I go from the house to the library and I know how to go from
my house to the stadium, but now I actually know how to go from the library to the stadium.
I just go library to house to stadium, right?
That's a new path that I've never actually run.
And we know that rats can, we will see representations of that during these
high-frequency events.
So it tells us that, you know, coming back to this cognitive map point,
it suggests that animals have that more global representation of the cognitive
map where they are able to physically,
or sorry, mentally, not physically, but mentally actually connect up things
that have never been physically connected. Right.
But now, so at these termination points, these high-frequency events,
you might see forward and backward sequences.
Yes. Okay, so now the situation is changing. Correct.
But on top of that, if you start to see what you call shortcuts,
actually you could argue that it's neither forward nor backward.
That's right. It's more like an implication of the information you have sampled.
That's right. Would you agree with that? Yes. So what aspect of cognition would that reflect?
I think it's about imagination. I think it's actually daydreaming.
And I think it's like, you know, when somebody's sitting in the talk and they're
not paying attention and they start thinking about other things, right?
I think that's very much what we're seeing, you know? You know,
I mean, I can't say the animal's dreaming, right?
But we know that when humans, for example, dream that, or when they imagine,
when a human imagines things, like imagines a face, right?
The same part of the brain that is active when they perceive a face becomes active.
So should we be surprised that when a rat is imagining other places,
the areas representing those other places become active, right?
So I think you're seeing kind of imagination and thinking about how the world is structured.
But is it also, would you call it insight?
Yeah, I'd be happy to call it insight. But then, can you say something about the dynamics of this?
Because in some sense, if I'm exploring my maze, here I am, I'm the rat exploring
the maze, I'm driving my hippocampal cells that you are decoding.
Sort of time lock to my action in space yes right
but now in these sweeps i'm also
time morphing if time warping yes because i'm not replaying that at the same
real time as correct so so what's the relationship there between the dynamics
and of this kind of well cognition happens faster than behavior okay and i mean
i agree with you and so there's definitely we know that it's faster faster.
It's about 40 times faster than behavior.
Just that's the data. Why it's 40 times faster, I have no idea.
What is the mechanism that makes it 40 times faster? Again, I have no idea.
It's a very interesting open question. But if, for example, you can use this
decoding mathematics to ask what speed of sequence is the best description of your data?
Is it the speed of the animal is it seven times faster
is it 15 is it 40 so we did this and
at every moment we said which is the most active which is the best model right
so most of the time it'd be about one to seven times because of this this thing
called phase precession where the play cells do at these moments of sweeps it
seems to be 15 times faster it looks like it's kind of a long a faster event,
and then at these sharp waves at these replay events these events you're talking
about at the feeders, they were 40 times faster.
What was interesting to us is we could actually go in with the look at the speed
of decoding and then ask, can we find the events?
And we actually could construct and find the events from those speeds.
So I think it does happen faster.
Why it happens faster is a really interesting question. But is this constrained
by the basic rhythmicity of the structure you look at?
Like hippocampus, you know, you have a very definite theta cycle.
You have a gamma range of oscillations within the theta.
Isn't that already constraining, let's say, the rhythmicity of these replay events?
I would assume so. Okay. I mean, again, now we're talking the fact that this
is a physical brain, right?
And the fact that it's a physical brain means that you have connections,
your circuits are actually driving your system.
And so the question is, what is the mechanism within those circuits that enables
this rhythmicity and this speed and this learning and all of this?
That's a fascinating question. There's lots and lots of labs working on that.
I don't think the answer is known at this point. Okay.
So when people have talked about these sequences in the past,
and it's been known for some time, they've often done so in the context of memory consolidation.
And they've argued that if these processes are interrupted, so you don't get
enough sleep, then you don't consolidate memories.
So is that the same thing? Or is that a different explanation?
So it turns out there's been fascinating data over the last five years or so,
which has suggested that these events, these called replay events,
these sharp wave events, during waking states are different than during sleep states.
Now, there's nothing we've been able to directly observe within them in terms
of the frequencies or other parameters that we've been able to figure out that's different.
But if you interrupt them during sleep, you interrupt consolidation.
You interrupt the transfer of memory from the hippocampus, the semanticization
of memory from hippocampus to other structures.
However, Lauren Frank's lab, Zhadov is the first author of the paper,
showed that if you disrupt these events during waking states,
you actually disrupt decision-making.
And you actually disrupt working memory within the task, not the consolidation afterwards.
We actually looked at the content of these events during waking states and sleep states.
And during the waking states, you see a lot of things that relate to insight.
Shortcuts, forward, backward. It seems it's almost kind of covering the space
as if it's just exploring the space mentally.
But during sleep, they're almost always forward. They're almost always replaying the actual events.
And of course, that's what you want. If you're consolidating memory,
you don't really want to explore the space.
You want to consolidate what actually happened.
And so it suggests that what may well be happening, and I emphasize the word
may because this is exactly where the field is right now,
is that these events during waking states are more involved in some sort of insight, imagination,
processing, trying to figure out what's going on.
And during sleep, it's more of a consolidation event.
Not dreaming. The rats aren't dreaming. I think the rat's dreaming.
I don't have proof it is, but of course there is this new data from,
I should know the authors, where they did actually an fMRI study.
I don't know how they got the people to sleep in the fMRI machine, but they did.
And they were able to show that after dreaming, these are humans,
so they could wake them up and actually say, what were you dreaming about?
That they were actually able to use similar decoding methods to show that if
the dream contained people, the face areas were active.
And if the dream contained landscapes, the landscape areas were active.
And so just as lots of data from Nancy Kanwisher's lab and many other labs have
shown that if you imagine a face, you're using the face part of cortex.
This paper, again, I apologize, I should know the authors offhand,
but it was in science about a year ago. This paper...
Showed that during during dreams and we know their dreams because we asked the
people or they asked the people and the people said They were dreaming these
same structures were active now our rats dreaming.
I don't know well, I guess you would look for correlated activity or certainly.
Temporary related activity in the sensory areas.
Yes, and those exist right that has been shown to exist during these Sleep events
which during the sleep events you definitely get correlated activity in cortical areas.
And you also get that actually after these events, the activities in the cortical
areas actually tie together better, suggesting that there is some sort of transfer
of information going into the cortex.
So, but you would want to allow some things that humans can do that rats can't do.
Well, rats don't talk to us. Okay, but beyond language, which most people will
agree, but for instance, an ability maybe to frame what you're thinking about
or imagining about in the future is maybe something that we're particularly good at.
So I can imagine some arbitrary event, or I can ask you to imagine yourself
in some arbitrary place, arbitrary time, and you can think that through.
Is that something maybe that rat circuits would be less able to do,
or you just want to leave that for future research? I would leave that for future
research, but I think that we, I mean, I don't want to claim that rats are small humans.
That's pretty clear, right? And beyond language, you know, when we imagine the
future, we're imagining decades,
we can imagine centuries ahead, we can imagine, you know, normal humans plan
about 10 years ahead, typically, a typical human, you know, why is a human going to college?
Well, they'll tell you because I want to go get a job in five years,
right? So, typical humans are planning years ahead. Rats are not planning years ahead.
So, I think it's more a question of scale that's changing.
And I mean, we're not surprised to see that there are similarities in other organs, right?
So, why should we not expect to see similarities in these cognitive structures,
right? Well, it could also be just a difference of degree, right?
Yes. It doesn't need to be the
case that the fundamental processes are qualitatively different. Exactly.
But then the question is, so we have an idea now on, let's say,
the basic memory dynamics of hippocampus as you have assessed it experimentally.
And we have these different kinds of replay events forward and backward.
But to whom's benefit? So I'm here, I'm the rat, I'm running in a maze.
I'm at the reward side. I have a forward or backward, uh, replay. Um, and,
Who's going to process that? Am I throwing this out for my neighboring or for
my other cells in the hippocampus? I'm assigning this to other brain areas?
The short answer is we don't know. Okay. We do know that other structures are
listening to those events.
We know, for example, that the ventral striatum, the nucleus accumbens,
which is a structure involved in evaluation and motivation,
has reward information in it that those reward cells also replay.
That's Karian Lansing's data from Cyril Pennartz's lab, where they saw that
the sequence of hippocampal replays, I think during sleep states,
if I'm remembering right,
actually triggers immediately following the appropriate ventral striatal information,
which actually suggests that it's not just...
Because in fact, they had different flavors, and they saw the correct flavors
being decoded, which again suggests actually it's more than just space,
that it actually contains space and these other information as well,
at least in ventral striatum and accumbens.
It's known that cortical systems are, as I said, listening to these replay events.
But it's quite possible that the replay events are actually also affecting hippocampus internally.
That would not surprise me. But I don't think it's known at this point.
So when we talk about the human literature on hippocampus, we usually use the word episodic memory.
And essentially, you're saying rats have episodic memory and that space is one
part of that. That's right.
And one of the things that I think is very interesting is that I would say actually
that it's mostly about episodic future than episodic past.
There's some very beautiful kind of discussions coming out in the human literature
about hippocampuses being necessary for imagining the future,
which is an episodic, it's called episodic future thinking.
And one of the things that's interesting is, of course, episodic memory,
episodic past thinking, is notoriously fragile.
It's very easy to manipulate that by framing questions in strange ways,
or not even strange ways, just even normal questions, can change what you think you remember.
And I think, and I think that a lot of the human literature now is coming to
this, is that the reason that episodic memory is so fragile is it's not really a memory.
It's actually an imagination of the past.
And in the same way that you imagine the future, you're actually taking these
pieces of memory and putting them back together.
And because you're reconstructing, you're rebuilding that past using hippocampus.
This is why you need hippocampus for episodic memory, is that you're actually
doing this kind of episodic past thinking.
Well, one of the theories of what the hippocampus might be doing is not so much
about events or processes in time, but this idea that it's doing pattern completion.
So you are given some information through your sensory systems and from what
your hippocampus encodes about maybe where it is in space, it's able to fill that out.
So that kind of pattern completion aspect of what the hippocampus is doing is
another thing that it's contributing beyond being able to forecast future events.
But that's the whole point of that future event is you need to pattern complete
it because what you need to do is you need to take the pieces and put them together
and complete the pattern sometimes in novel ways,
sometimes in not novel ways, Basically, you have to, as I said,
the reason you have the memory is to construct that future so that you have
to take those pieces and you use pattern completion processes to rebuild that system.
When people talk about autobiographical memory, though, they often talk about
involuntary autobiographical memory, which is where they're referring to the
fact that something about the current context reminds me of a past event.
And one explanation of that is it's about reinterpreting the current context
in relation to things that have happened in the past.
So I think maybe you push it too far to say it's about imagining the future
because it's also about understanding the present.
Yes, absolutely. And in fact, I would very much argue that one of the reasons
we have episodic past thinking and episodic memory is so we can reinterpret that past.
And we can say, oh, that's what that really meant.
Yeah, but so in some sense, you're sort of making a contrast to the more traditional
view of memory, where it's like a storehouse of the facts that you encounter, right?
Right, exactly. And then basically what you're saying is, well,
maybe these so-called facts are more malleable to change than we thought they would be.
Yes. Like memory is continuously constructive. That's right.
There are still, if you want, ingredients of that past in there.
Yes. That's not completely arbitrary.
Absolutely. Okay. So I don't think, so if you say, look, it's focused on the
future, is that not, let's say, a bit of an overstatement in that sense?
I think it does both future and past. But I actually think we have to be careful
because, you know, we're talking as if the future is completely open and malleable
and plastic and all that.
But the fact is, we use those same ingredients to predict the future.
You know, when I came to give the talk, this is the first time I've been in
Barcelona, it's the first time I've been in this auditorium,
but I've given talks to other people with similar audiences and similar backgrounds, right?
So I said to myself, okay, I can take my ingredients from all these other pieces
and I can use this to construct that future.
And again, I'm using facts to do it. I think that's why you construct with those facts.
You're absolutely right. I mean, the goal of memory is not to imagine what it
could have been. And you could do that.
That's a counterfactual question and one of the things that people do.
In fact, we now know it's one of the things rats do. But it is also true that
sometimes your goal is to try to get as accurate a memory as possible.
The key is that that's actually hard.
Which is the whole, I mean, that goes back to Elizabeth Loftus. Right.
No, but isn't the cool consequence or an interesting consequence of that,
that maybe if you just take a hippocampal centric view on memory. Right.
The hippocampus just gets information from other extra hippocampal areas.
And this might come from sensors. It might come from other memory systems.
And it's just sitting there chunking out episodes.
So in some sense, you could argue, well, for that memory system,
if you look at CA3, which is really a core memory system of the hippocampus,
whatever information I dump in there, whether it comes from a past experience
or a current event, I actually don't care.
I just compress this together in an episode irrespective of the origins of that piece of information.
And that would mean that as much as I'm predicting my past, I'm also predicting my future.
If you want, always remodeling my past, given this mixing of current states
with past states. Would you agree with that?
I think that's a very plausible explanation, and it's probably the most likely one.
But at this point, I think it's also very important to say, we don't actually
have the connection between these mechanisms.
And the underlying circuitry. That is, we don't know what it is.
And this is the point you're asking about the oscillations.
We don't actually know what it is about those fundamental circuitry that is
enabling these processes.
We know these processes exist.
We know that they're there in hippocampus. We know hippocampus is doing them.
We know a lot about the circuitry of hippocampus.
But at this point, that connection is still actually an open question.
But the interesting thing is that these hippocampal areas are equally predicting
the past in the sense that if prediction errors relative to past events would
exceed a certain acceptable threshold, you just change them.
Would you agree with that? Say that again? Well, for instance,
if you see sweeps in hippocampus that reflect memory, now you are in a position
to actually measure the accuracy of that memory.
You can pose the question, well, was the rat exactly in that position or was it sort of there?
Is it like already, if you are massaging this memory to the current state?
So in the data you have on that, do you see really,
let's say, accurate factual replay of
past events that you really know x y is identical yeah
or do you already see a massaging of that
we already see the massaging there you go absolutely no the
the replay events are definitely not completely veridical the question is which
we don't know is how much of that noise matters right right how much of that
noise is actually reflecting,
a or i should say not noise how much of that difference is
actually reflecting a change and how
much of that difference is actually just noise right exactly
and we know that it's not i mean biology is messy right but how much of that
is on which side you know is actually a very interesting open question okay
so in your data you showed us a couple of examples one of those was was what
you call Vicarious Trial and Error,
which is an idea going back to Tomlin, but also before that,
I think it was Mertzinger and Gentry, 1931.
So a very nice example of how when the rat gets to the top of a junction in
the maze, it might glance left and you can see a play out of what might happen
if it was to go down that left tunnel.
But in a way, that's evidence to me that, yes, he can plan ahead and he can
use past experience in order to fill out what could happen in the immediate future.
It's a little bit like the involuntary autobiographical memory ideas that you're
reminded of stuff which is relevant to your current choice.
But then you're also talking about situations where he was at a feeding point
and then he would just replay being in other positions in the maze.
So that perhaps is stronger evidence that the rat can imagine himself in other
positions in the world. I mean, would you go that far?
Well, I would actually say we don't know how involuntary that forward sequence is. Right.
Right. So, in fact, to be honest, my gut is it's much more voluntary than involuntary.
Why are you saying that? To be honest, mostly because when humans are doing
this kind of deliberation moment, it's extremely voluntary, and it's very much very cognitive.
In humans, reaches conscious, tends to reach consciousness.
It tends to be, you compare it, for example, to a more automated behavior,
where in a human, you've switched from this kind of very flexible,
slow, deliberative mode to a very habitual kind of automatic mode.
In fact, it tends to be less reaching immediate consciousness.
And it often is kind of a, well, you ask, for example, a sports player,
why did you do that? They say it felt right.
And in fact, you ask a sports player, we'll talk about being in the zone when
things just kind of click.
And I think what they're doing is recognizing that their habitual system,
their procedural system is kind of working correctly.
So I don't know how much of the rat forward sequence is voluntary versus involuntary.
And I don't know how much of the replay happening is voluntary versus involuntary.
It's possible, and there are a lot of computational models that have suggested,
that the events happening at the The feeder sites is just what happens if you
have the right connection structure and you put noise into the system and it
kind of percolates over.
Okay, so the events happening at the corners in the maze are more cognitive.
And what you're saying is that perhaps the evidence at the feeder sites is not perhaps active.
I want to think about if I was in that part of the maze, it's just the noise
in the Bacampus could give rise to these sorts of patterns.
So that's the current theory, but let me give you a very interesting data point,
which I still don't know what to make of it, which is that we had an experiment.
And on this experiment, the animal would sometimes go only to one side of the
maze for many, many trials, and then would go to the other side of the maze,
and vice versa. And it would do.
But the point is, we could ask, how recently had it been on the current side
of the maze or on the opposite side of the maze?
And what we found is that it was more likely to imagine in these kind of,
during these events happening at the feeder sites, this thing we're talking
imagination insight kind of things,
was much more likely to be on the opposite side of the maze when the animal
had not been there recently.
And that's strange that doesn't fit the computational models of how we had understand
the circuit to be and suggest that somehow whatever role this is playing it actually,
depended on a lack of activity a lack of recent experience now the animal knew
the maze very well it had been on this maze many many days but on this day it
was say mostly on the left side,
we tend to see more right-side replace, which is strange.
And one possibility is that it's doing it to try to keep everything balanced,
and trying to say, well, you can't forget what's over there.
I may need to know it at some time.
But again, you run into this mechanism problem. What's the mechanism of it?
It doesn't sound like such a complicated phenomenon to interpret from a network
perspective if you would allow dynamics of, let's say, short-term plasticity,
where you say, look, I have known trajectories.
I'm suppressing their responses.
I have, let's say, lateral interactions among different locations in this environment I visited.
Visited they can now become active because
blah blah we can tell each other
stories like that's right right yes so then given that
i can if you want not trivialized but
sort of interpreted in mechanical terms why don't do you find this such an important
data point because the short-term plasticity story that we would want to tell
that we would have assumed to tell is based that recent Recent activity would
make a cell more likely to fire.
And in order to explain this specific data point, you have to suggest that the
cell is less likely to participate.
No, but I can just use an habituation component. The question then is,
why on other tasks is the more recent stuff the stuff that gets replayed?
So the problem here is that I can explain this experiment by creating these
kind of habituation, adding in habituation parameters and stuff like that.
But there are other tasks where actually the most recent stuff seems to be replayed.
So we now have to say, now our parameters for our mechanistic explanation depend
on task in strange and complex ways. Right, okay.
And so, yes, that's possible.
But why?
It would not be a satisfactory story because it would be ad hoc given one task only.
Exactly. But then my point is, why would you bring in an interpretation that
uses the notion of voluntary?
I was doing that more to point out that it wasn't just simply this noise model.
Okay, okay. I understand.
I don't know how voluntary these things are, because I'm not sure what the word voluntary means.
No, right. That's what I want to ask you. What would be the signatures of voluntary
operations in the preparation you look at?
What I would say is that when we look at human sequences and we identify some
sequences of voluntary and not voluntary in humans, which is its own debate,
but once we've made that categorization,
then we can say, what's the information processing in the brain structures in
humans under these, quote, unquote, voluntary conditions?
And then we can say, okay, does the rat show the same information processing
in the same brain structures?
And so then the question is, well, you could either argue humans are not voluntary,
or you can argue that rats are voluntary, but you're no longer allowed to argue
humans are voluntary and rats are not.
Right. So I think in the episodic memory literature, people are using that in a very specific way.
So involuntary memory is when things just come into your mind,
and you didn't intend that to come into your mind.
And it may be an unpleasant memory of something that's associated with what's
happening. Whereas voluntary autobiographical memories, when you specifically
try to remember what you were doing last Friday or Christmas Eve or whatever,
so you're working to retrieve that memory.
And that would be very interesting, obviously, if we had any evidence at all
that a rat could do something like that.
Well, one of the data points that we're still kind of finalizing the data on,
so I don't want to make too much of it yet. We're still looking at this.
We have an abstract at SFN, for example, talking about this.
So it's going to be out soon.
Is that it looks like hippocampus is being pulled during these moments by other brain structures.
So it looks like, in fact, prefrontal cortex in particular is basically saying
to the hippocampus, go find stuff.
So in a sense, it's possible that that could get closer to your voluntary story,
that some other system is actually saying to hippocampus, I need to remember this now.
Go figure this out for me you need some way
of ignoring the the sensory context and
saying this is the context i want you to consider uh construct a a scenario
starting from there but is there a way to actually i'm trying to think if there's
a way so what we'd want to do is have a way of essentially shutting off the
sensory context and still seeing that at this moment.
And that's hard to do because it's really hard to shut off sensory context in a controlled way.
It does enter the hippocampus over very specific pathways that are anatomically
well-defined and not mixed.
Yes. So this might suggest that this allows you to, let's say,
segment these input streams.
Yes. Right? Biasing towards, let's
say, a memory component or a sensory component or an action component.
Yes. Would you buy that? Yes. Okay. Absolutely.
It's a very hard experiment to do, though. But yes. We're hoping that you will do it soon.
But now, we looked at this. So Tony mentioned this vacation.
Error is trial and error. Vacation is trial and error. And I was getting confused here.
But it's not, you just not only looked at that behavior for historical reasons.
You have tied it down in a very specific way to the memory dynamics and the
task performance of the animal.
So that means it's only in a very specific moment in the task that rats actually
display this behavior. It's not that they do it all the time.
So what is exactly the structure that you found there?
So actually, Tolman actually saw this as well, that vicarious trial and error
tends to happen during early learning stages,
during stages when the animal knows the environment that he needs to work on
or that it needs to work on, because both males and females do vicarious trial and error.
That it knows the environment that it needs to work on, but it doesn't quite
know what to do on that environment.
And once it actually gets enough experience that it no longer,
essentially no longer needs to search, and it says, I know at this moment I'm
going to go left, then the vicarious trial and error goes away.
And we can put it back. We can reinitiate it by actually doing a reversal reversal,
where we force the animal and say, well, what you've been doing all this time doesn't work anymore.
I think of actually, you know, my classic example is driving to work.
The first time you're driving to work, you're planning, you're paying attention,
you're thinking about where you're going to go, you're doing all this deliberative stuff.
But if you do it every day for weeks, months, or years, suddenly you're doing
it, you're doing it in your sleep, you're doing it, you know,
you're driving your friend to your office instead of the airport because you
got that into a good conversation.
You know, luckily in Minnesota, the airport's close enough that that didn't
mean he missed his flight.
He was from New York and was really worried where that would have been a disaster.
Right, exactly. Um, I don't know,
But actually, so for example, in my drive to work, they actually closed one
of the roads, and suddenly I had to recognize at this one intersection,
which was actually significantly before the road got closed,
that I needed to turn a different way.
And it was actually very interesting kind of seeing my own internal system about
identifying at this moment, I have to, I found myself doing a lot of vicarious
trial and error, luckily in my head and not with the car.
But, you know, actually this kind of reversal does trigger that.
You were hoping for the chocolate to fall from the sky as well.
It would be nice if it was European chocolate, not American chocolate.
Yeah. But look, so the point is you said, okay, deliberation requires both search and evaluation.
Yes. And you see this vicarious trial and error as a signature of search.
Yes. That's correct. Yes. Okay.
So are you suggesting with that that this is really an active strategy to obtain
information from the environment? No. Okay.
That's an open question. I don't think so, and I don't think so in large part
because we have also seen vicarious trial and error in situations where the
reward site is actually out of sight of the animal,
so down tunnels that the animal has to go.
And in our environments, the environmental signatures are very, very different.
It's not a discrimination cue. So it's not a lot of difficulty for the the animal
to know what's on the left side or what's on the right side.
So I think this is actually more a reflection of an internal process in which
the animal is trying to figure out what the consequences of going down that path are.
So it's not so much that they're trying to trigger the sensory,
although it might be that they're trying to physically trigger this involuntary memory sequence,
or it may just be that actually the voluntary memory sequence,
if I can use that distinction the voluntary memory
sequence is actually triggering an initial motion
right that is kind of saying well okay let's start
to go wait no i don't want to go left right and i think
that may well be what a lot of this vicarious trial and error is
okay but so so you're saying it's not necessarily an orienting response to obtain
information it's more like an action that you use as a recall cue well it's
either an action as a recall cue or an epiphenomenon of the of the The internal orienting response,
which is also driving kind of the beginning of the action. Right.
And I don't know which one it is.
Or it doesn't need to be an exclusive choice. It might be both,
right? That's right. That's right. Mm-hmm.
But then, can we really call that deliberation as such in these experiments, right?
So also in your case, so the road to the airport or to your work is closed,
and now suddenly you have to sort of find a new route.
So you could say, well, look, I'm just obtaining sensory information. I'm consulting memory.
I'm performing actions. It could also be like you search through a list of alternative
options as opposed to really, really actively modeling the consequences of future actions.
Well, we know that they are in fact actively modeling the consequences of those
future actions because we know that there are direct representations through those options.
What I think the real key is that we also know that there's reward extra information,
particularly in the ventral striatum, particularly in the areas of ventral striatum
that hippocampus projects to, that reflect those reward information.
So not only is there a representation of those future options,
but there's also a representation of the value or the reward that they're going
to get at the ends of those options, which suggests that it actually is a search
and evaluate process. Right. Okay.
So now, we learn a lot now about the specific memory dynamics of the hippocampus,
but as you already indicated, hippocampus doesn't operate in isolation, right?
It is part of a circuit of a loop, essentially.
It's a loop with many other parts of the brain. And at some point,
you showed us at least how, in your view, this loop is interconnected with some
other key areas in cortex and subcortical areas.
So how do you see that overall loop play out? What do the different stages in this loop do?
Well, the key, I think, that we have, I mean, there's issues of the loop from
the circuitry perspective of how does information get in from,
for example, entorhinal cortex and the different parts of entorhinal cortex.
And we could certainly talk about that.
But I think at this point from this deliberation moment, what we know is that
hippocampus contains the information searching through that future.
That it reflects that information.
And the evidence, at least both from humans when they have damaged hippocampi
and from other animal experiments where people have manipulated hippocampus
suggest that hippocampus is critical to that future construction.
We know that the, we have some evidence that there's prefrontal information coming in.
We have some information that the ventral striatum is doing evaluation,
but at this point how they all interact on a moment by moment basis is actually
something that we don't know right now.
And that's actually something we're very excited to start going to look for.
But now in that circuit that you delineate,
Which of these four different regions you mentioned would have the biggest impact
when it would be lesioned?
Would it be the hippocampus? So it would be ventral striatum?
Would it be prefrontal cortex?
They would all have different impacts. Of course. But which one would be the most critical?
Well, critical for what? Solving the task.
I think they're all involved in solving the task. They're all involved differently.
So, for example, I think that without ventral striatum, although,
to be honest, we have not done this experiment, I would predict that without
eventual striatum, you'd have a lot of trouble differentiating values of rewards, right?
And we know from other people have found that it's critical for recognizing evaluation.
We know that, again, from other people, that orbitofrontal cortex,
which is another structure involved in a lot of this, is very critical when
you have different flavors, when you have to integrate across multiple reward information, right?
That it's very important when you're doing logic from A implies B,
B implies C, C implies I get reward, so I need to actually go to A.
These kinds of logical chains, you need orbitofrontal cortex to be able to make
these kind of logical chains.
The how these structures actually
are i mean they're all critical for the task and
the key is that they're going to each play a different computational role and
so the way i look at it is it's more a question of how does the computation
of the animal change when you've taken that out of the circuit yeah but you
also indicated yourself during your talk that for many of these tests,
you can just remove the hippocampus and you can still solve it.
You might be a bit less efficient, it might take you a bit longer to acquire,
but the way we test at least the rat brain,
In many of these tasks, you can actually manage your hippocampus. Yes.
So I think the key here is that, and this is why we have multiple decision systems, right?
Because in fact, there are many ways to solve many of these problems.
One of my favorite examples is Varga Khatam's data on children who have damaged hippocampi.
I believe they have congenital issues where they have no hippocampus.
They actually do okay in school, but it turns out that they do okay in school
by working completely semantically, and they work in school by a very different process.
They actually are able to pass their classes, and they do well and all that
stuff, but they have a completely different process by which they solve their behaviors, right?
And I think that's why we have these many, many systems, right?
Because evolutionarily, right, we didn't usually have a doctor we could go fix, right?
So, when a person solves or an animal solves a task without a hippocampus,
they do it very differently.
And in fact, you can provide, if you get the right probe trial,
you can actually construct probe trials whereby they behave differently, right?
One of my favorite examples is the Tolman-Hull debate on the T-Maze,
where they trained animals to go from the south arm to the west arm of a T-Maze.
And they both did it identically. And if you look at the path from the south
arm to the west arm, they look the same.
But if you put the animal on the north arm, so I should say,
Tolman argued that the animals were going to the place.
Tolman said, I know where I am. I'm at the south arm. I know where I want to
be. I want to be at the west arm. How do I get there? This is Tolman's explanation.
Hull says, no, no, no, it's all stimulus response. The animal says,
I'm on the maze. I turn left.
Now put the animal on the north arm of a plus. Paul Jay,
So Tolman's animals will say, I'm on the north arm, I want to be on the west
arm, I want to make a different action, turn right and go to the same place.
Hull's animals will say, I'm on the maze, turn left. They end up at a different
location. It's not that one of these is right and one of these is wrong.
It's that this is two ways of solving the task computationally.
And in fact, turns out that what happens is that early on, the animals look
like Tolman. and late, the animals
look like Hull and they actually switch from one system to the other.
But the Tormund-like system can be training the other one.
There's models by, for instance, Richard Sutton where he has reinforcement learning
going on but he also builds a forward model and very much in the same way as
your rats, he plays sequences through the forward model and uses that to train
the reinforcement learning system.
Yes. So, and do you see that as happening then in the rat brain? So, I suspect yes.
We know, for example, we've done models, we've built models where this kind
of consolidated replay will train up a downstream structure that can learn a
more habit-based kind of action chain story.
One of the experiments I've always wanted to run, I've never actually run it,
mostly because I just haven't done it yet.
And if somebody else wants to do it, it's fine, is to run one of these T-Maze
plus maze experiments where the animals switch from one to the other,
but actually to train them for only a limited time and then put them back in
their home cage and leave them for several weeks and then test them. Do they switch?
Right? Is it actually the experience that creates the switch?
Is it the mental part that creates the switch?
Is it time? We don't know that answer.
But we know that when somebody is learning from, actually, the example we just
came up with, I was just talking to an interview from Sports Illustrated,
which is an American magazine.
They're asking me about American football players who have to learn these big,
large playbooks, which of course are all Xs and Os and declarative memory.
And yet they have to actually execute it in procedural memory on the field.
And so there's this very interesting transition that these people have to do,
right, by imagining and simulating and practicing and learning and do that transition.
It definitely happens. What the exact processes are is a fascinating question.
But David, I would like to...
Come back to your earlier remark where you said, well, yes, it's true,
you can lesion hippocampus, animals can still perform the task,
but maybe that just shows redundancy.
But isn't that a little bit too easy? Because you could argue,
look, hippocampus does stand out on anatomical grounds.
It's a very unique kind of organization that you will actually not really find
in that way anywhere else in the brain.
As you said yourself the mind is the brain so that should imply that that anatomical
structure that structure is telling us something about a unique function yes
and I want to the second part is that it's a two pronged attack okay so ahead
the second the second prong here is that.
Maybe this implies that the tests we are using to understand the function of
that structure are just not sensitive enough yes maybe the tests we're using
are just two let's say course to really titrate out the specific contribution of hippocampus.
And as a result, we are forced into this redundancy interpretation because our
test is just not sensitive enough.
I want to be careful with the word redundancy. I'm sorry if my use of the word
redundancy implied that they were all equivalent. I don't mean that at all.
The computation by which the other systems actually perform is very different.
And so an animal, for example, I mean, giving the Tolman Hull example that I
showed, an animal with a hippocampal lesion, in fact, looks Hullian very quickly,
much more quickly than an animal actually with dorsal striatal damage,
who kind of stays Tolmanian for a longer time, right? Right.
I, the task, the key is that these tasks are not clinical tasks and we should
not be saying that the Morris water maze is a hippocampal task. It's not.
The hippocampus is necessary to solve the Morris water maze in a very specific way.
But yet, and so if you train the Morris water maze in a different way, right?
For example, you train the animal for months and months, or you train the animal
using this very large platform shrinking down to a small platform,
you can train an animal to do the task without a hippocampus, right?
So the key is to actually think of what is the computation the hippocampus is
performing? how does that computation then get used in this task?
And can we construct a probe trial which will differentiate that computation
from other computations?
And I actually think, in fact, when I say redundancy, what I mean is many of
the things we have to do in life
are able to or depend on are able to be solved in these multiple ways.
But I don't mean to imply that they're being solved in the same way at all. Okay.
So the last set of experiments you presented to us, which I thought were really
very exciting, was dealing with the notion of regret in rats,
which seems very counterintuitive. Yes.
So what does regret really mean in the case of a rodent?
So let me be very careful with what we actually found in this data,
because, of course, it's easy to kind of go way overboard with the term.
What we found is in situations in which we would expect it to induce regret
in humans, the animals behave differently, and their neurophysiology is different.
That behavior proves that they understand their own agency, that they understand
that they made a mistake, and they recognize that mistake.
And so we were able to differentiate that. And we were able to show that during
those moments, there are representations of the previous event,
that is the previous moment when they made the previous decision.
And whether the animal feels regret at that moment is kind of the thing that
makes all the popular media happy, right?
But the truth is what we actually found is there's different information processing
happening in conditions where the animal recognizes it made a mistake by its own agency.
And a case where the animal experiences a similar set of cues or an equivalent
set of cues, but makes a mistake not by, the mistake is not of its own agency.
And that we then saw that the information processing tracks that counterfactual,
which is critical to human regret.
But maybe we should try to understand the task a little bit better, right?
So we have a circular arena or a little tunnel through which the animal runs. It's actually open.
Okay, it's open field? Well, it's a circular arena, but it's a racetrack. Okay, yeah, right.
It's a racetrack. It's a racetrack. So there's cues everywhere is kind of the key.
So at four equally spaced points along this circular track,
you then have little zones, which you call restaurants if you want,
where the animal can sample a certain type of food reward or a certain flavor.
Do they smell it or do they eat it?
They eat it. They actually eat it. And they're different flavors,
and each flavor remains at a constant location throughout the entire training.
So the animal knows that in the southwest corner, there is going to be chocolate.
Exactly. And now you saw across the animals you tested, they have individual preferences.
Right. So there's an important step before we get to the individual preferences,
which is that every time the animal encounters one of these zones,
enters a restaurant, we like to say, there's a tone,
a pit, where the pitch of the tone tells the animal how long he's going to have to wait for food.
And that allows the animal to make a decision to either stay or go. Right.
Right? And so then, because animals have thresholds for each of these different
flavors, we can identify their revealed preferences. So the animals will stay.
If it's less than, if the offer, right, the cost of this restaurant,
of this food pellet is going to be small enough, then the animal will stick around.
And if it's going to be too long a delay, the animal skips it.
Right. And so that threshold allows us to measure how willing the animal is
to spend its time for that food.
Yeah. And what we found is that different animals had different thresholds for
different flavors, but that they were consistent with an animal.
Right. So that one animal would wait a long time for chocolate,
which tells us that that animal is willing to spend more time on chocolate than on other things.
Right. And also what's interesting there, in the behavioral signature,
they either decide to wait or they move on.
It's not that they start waiting and then interrupt the waiting. Correct.
So now what's the longest waiting time these rats are supposed to,
are willing to suffer? So for two of the rats, we only went up to 30 seconds
because that gave us enough range.
But for two of the rats, actually, the rats were willing to wait every time
at 30 seconds. And so we went up to 45 seconds.
And that was long enough. But
we've seen rats wait some of the trials even 45 seconds for some cases.
But then you also impose a timeout. So a total of 60 minutes to just consume
whatever they can get. That's right. And then the point is that you're manipulating
now these waiting times because you know the threshold value.
That's right. So in that way, you can create, if you want, disappointment or regret.
That's right. The difference being disappointment is when you have unexpectedly,
let's say, changed the property they might find that is different from what
they thought it would get.
Right. Well, it's actually a random distribution from the 1 to 30 seconds.
So it's kind of on the tail end of that distribution, that they just kind of,
they know it's somewhere in this thing, and they kind of get the bad end of the deal.
So a good deal will be a short waiting time for a reward you like,
and a bad deal is a long waiting time for a reward you don't like.
That's right. Yeah. So, but now, so now we understand the task,
and then the question is,
Is it fair to interpret this in terms of agency? Because in some sense,
agency implies that there is
a knowledge of the causal relationship of the agent with the environment.
So you can say, yeah, I did it. It was my choice. That's right.
So what you observe is that when animals hit this point where they get,
let's say, they have to wait longer than expected.
Right. By their own choice. You see certain signatures that you interpret as
indicating regret. So what are these specific signatures?
Well, the key is actually, you have to actually look, in order to get the regret
on this task, you have to actually look at a pair of samples.
And they have to actually have skipped a good deal to reach a bad deal.
And it's the skipping a good deal that, so we know by the fact that they either
stay or go, that they're making a decision, right? And the fact that they skip
a good deal means they've made a decision that is against their preferences.
And now they encounter a bad deal. And because they're time limited,
it means they've messed up.
They've made a mistake and they've erred where they should have taken the good
deal. And they're not allowed to go backwards.
Once they've left a deal, the deal is rescinded. No more, you know,
it's only good while you're in the restaurant. Once you leave,
you have to, you know, basically you have to go all the way around and try again,
and it could be a completely different deal.
So what we see is that at the moment when they have this mistake,
where they've made a mistake and now they hit a bad deal,
they'll stop, they look backwards, and at that moment, the orbitofrontal cortex
and the ventral striatum represent the moment of entering the previous restaurant.
And in the same way that we talked at the beginning of the podcast about this
decoding, we're doing the same decoding operation, and this time not for the
location of the animal, but for entering each of these different zones.
And so we can say this is a good representation of the previous zone. Mm-hmm.
Which is a mental time travel to that moment. Yeah, so in some sense,
what the mental time travel entails is that you recall, let's say,
the value, essentially, of that other zone.
That's right. So it's more like you call up a reference, like,
okay, but if I would have stuck it out on the other side, this is what I would have gotten.
Well, so what's interesting is that we didn't actually see a very strong,
although there's very strong representations of reward in these structures,
That is, at the moment of reward, the cells fire, you know, a subset of cells
will fire massively for each different flavor,
telling us very differentiable what each reward is.
At this moment of, quote, regret, unquote, the animal did not represent the
reward it should have gotten. That is, it didn't represent the reward.
It actually represented the entry point into the other restaurant.
Yeah, but wait, if you decode that from the ventros triatum,
as you said earlier, that must reflect some sense of valuation.
Or not? That's a hypothesis.
But given the literature, it would be consistent. Given the literature,
it's quite likely that it's some sense of valuation.
Actually, I suspect the orbital frontal cortex is also some sense of valuation.
Exactly right. Yes, absolutely.
But we don't know that. What we know is that at that moment,
there's a representation of that moment in the task. Right.
Whether there's actually a valuation judgment associated with it would be very, very interesting.
Interesting one of the things i'd really like to look at is
whether there is some relationship between
the self-firing patterns and the the value of
each of the rewards but it's very noisy and it's very hard to decode with the
limited data we have right exactly my hope is that if as we gather more data
we'll be able to actually determine whether in fact there's value representation
at that moment if it's just the moment that's being represented and what right
and of course one One of the interesting questions is, what's Hippocampus doing at that moment? Yeah.
But now you could also argue that your task is like a rat gambling task and
that you misinterpret it in some sense, right?
Because I could also say, well, if I'm the rat, I'm here standing in front of
this, the zone with the banana flavor, which I really detest.
I want to get to the chocolate.
So I don't care what he offers me. For me, it's not a bad deal because I want
to get to the chocolate. And then at the chocolate, you offer me a bad deal.
So I'm not regretting anything I did with the banana because I'm not interested in banana.
But you would still in relative terms just interpret the local event because
you say well you decided not to
wait there and relative to the delay i was giving you it was a good deal,
and you still didn't take it and that the chocolate gave you a bad deal but
the rat is saying look i'm not interested in banana i don't care about your
deal i want to just get the chocolate well but we don't see the rat behavior
look like that that is the rats actually take all four deals when When they're good deals, right?
It's not, their thresholds, the differences are five seconds out of the 30.
The other thing is that the next offer after, let's say it's chocolate to banana
to cherry to plain, after the banana, it's gonna, he's got two more offers before
he's coming back to the chocolate.
And in fact, we know from these representations that when he's left the banana
and he's done with that, he's thinking about the next one, which is the cherry,
not the one that he previously came. So normally, it's going to be representing
what's next in my sequence.
So I don't think... The point is that only in these very specific conditions
do you see this representation of the previous choice.
And one of the things that I think is important about this task is because there
are four options, it's not just, well, I'm not thinking about...
I don't want to be where I am now.
It's actually thinking about every specific case of that previous option.
Whereas again, normally the end will be thinking about the next option.
So, you've presented evidence that rats can feel disappointment,
maybe regret, that they can mentally time travel, potentially even quite far
in the future, and imagine the rewards or punishments they might get.
So as a scientist, you must obviously think, well, what are the implications
here for how we use animals in our research?
Yeah, well, I think that the key, I don't think it changes anything,
because I think the question still has to be, how do we make sure that our animals
are treated as well as possible?
That we want to make sure that every experiment we do is fully justified.
That we want to make sure that unless we're studying a stress condition,
we don't want to be stressing our animals.
I mean, I know I think differently under stress.
So if I want to understand how normal behavior is happening,
I want to understand how a non-stressed animal.
So I think it's always a question of you know
really thinking about this question and always asking yourself is
this experiment important is this
actually something that has to happen and I think that's a valid question and
I think that we every scientist I know asks that question as they're doing experiments
I guess so that one thing that we might have done in the past is think well
it's a rat it's not going to worry about what's happening to it tomorrow but you know we
now have this extra consideration perhaps that these animals are able to think
back on things that have happened in the past or maybe look forward to events
in the future and therefore we have to be more careful.
Yes, but to be honest, I think that intuitively, we've always known this.
You ask a pet owner, that pet owner has always believed that their animals are
emotional, intuitive, you know, connecting.
And I think the whole animal experimental question has always included that. And I think it has to. do.
I think that it's easy to say, well, yes, there's been a historical science,
but we go back to the Harlow experiments, those horrible Harlow experiments.
He was arguing the importance of this from an emotional perspective, that this is critical.
I mean, there's wonderful data.
Deborah Blum in her book Love at Goon Park talks extensively about this.
I think it's one of the the best books on the whole animal issue,
talks about how every infant who has survived the NICU unit,
the neonatal intensive care unit, owes its life to Harry Harlow and those nightmare experiments.
Because the NICU units changed, and the survival rate went from basically nothing
to tremendous, to very successful.
Because people started actually, they started actually touching the infants,
right? Before that, it was assumed that you couldn't touch an infant because
it'd be a sterile problem.
But then human infants need contact.
So we changed. We changed the NICU units.
And those NICU units changed because of Harry Harlow and those experiments that
are incredibly hard to justify.
So this is a very difficult and complex issue, and is one that I think needs
to be thought of in terms of the importance of this, and how do we connect that up?
And to be honest, I'm not sure that this data changes.
It convinces scientists that, you know, the animals really do this,
and I think it tells us a lot about how these processes work,
which is, to me, the really important thing.
That we understand a lot more, I think now, about how these processes work,
which means, of course, we can start to ask what happens when they go wrong, right?
And in fact, a lot of human psychiatry, for example, is fundamentally dependent
on breakdowns in computation, right?
The way to really think about psychiatry, I think, is to think of it as failure
modes of an engineering system.
But we can't know what the failure mode is until we know what the engineering
system is. So where do you see the impact of this work in, for instance,
treating human mental illness?
Well, I think that, for example, if we can identify the fundamental structures,
that is the fundamental computational components that are driving things like
psychiatry, then we will actually be able to understand treatments better.
We'll understand what the actual symptoms are.
One of the things I like to say, we've been doing a lot of work on addiction,
actually, in trying to understand what is human addiction.
And I like to say addiction is a symptom, not a disease. that there's actually lots of diseases,
lots of dysfunctions in the decision-making and other systems that we can identify
from first principles now that we know how these computations work.
We know better how these computations work.
And that then leads us into being able to start to say, well,
okay, that's not a cocaine addict.
That's somebody who has an evaluation problem in the deliberation system.
Now, of course, we have to figure out how do you fix an evaluation problem in
your deliberation system. But at least we know what it is, right?
And there are many, there are cases, there's discussion happening.
I mean, right now, this is exactly where the field is, with this new term called
computational psychiatry.
I have to say, I'm not fond of that term, but it is the term.
And the idea is to take this new computational understanding of decision making
systems, and really this engineering view of the brain, right?
The brain as a system that's It's doing things explicitly and has a physical
process, running a physical computation,
and identifying where the failure modes, where the fault lines are,
and then connecting that up. And there's a conversation happening.
My hope is that this can change treatment and even definitions of what some
of these dysfunctions are.
The short answer is it looks like the answer is yes, but we don't know it yet.
I suspect we're five or ten years away.
And one of the hopes, I guess, for the more distant future is that one of the
treatments we might be able to have is to replace damaged circuits with artificial
circuits, what people call neuroprostheses.
And people are talking about hippocampus as a potential target for neuroprothetics.
I mean, what do you think about that? Is that a realistic possibility?
Oh, I certainly think neuroprosthetics are a realistic possibility.
I think actually the first things that are going to come, though,
are going to be better understandings of learning systems.
And that as we understand how these systems learn and modify themselves, right?
I mean, anytime you interact with somebody, you are changing the brain,
right? The fact that somebody remembers anything means the brain has changed.
So I think there's ways to do it that are less physically invasive,
and I think those are going to happen first.
I think that we're going to see a lot kind of small... I think the first things
that are going to happen is small modifications of treatments,
where we say, oh, well, actually, this treatment depends on working memory computations.
So if we just added a working memory training component, that will change how this treatment works.
I think that's going to be the first steps, because neuroprostheses require
not only the understanding of the computational process, but a second engineering
piece of how do you actually interface with the brain, which is non-trivial.
And lots of people are working on that, but I think there's a complicated second
step that's going to have to happen there. Mm-hmm.
So now, in the beginning, when you sort of had to, in a nutshell,
define what you're trying to achieve, you said that you're trying to decode
the mind from brain states.
Yes. Right? But if we now look at the data….
And let's say a bit more detached cynical perspective. I can say,
oh, great, but you've been decoding neuron states from brain states.
Where is the mind, right? How do we get that link to mind?
And I see that you're trying to get there also looking at these high-level constructs
like regret, for instance, right? Or agency, right?
But this is also if you want creating a risk because now if people,
if you would not be able to really nail that, Because these contracts are just very complicated.
You're left in this position that all you've been doing then is decoding neuron
states from brain states.
Right. So how do we really cross that bridge? How are we going to do that?
How confident are you that we're actually close in reaching that goal?
I'm actually pretty confident that we are reaching that goal.
You're absolutely right.
There's danger in using these terms, particularly when we have about half of
the term, which is I think what's happening in regret, for example.
We don't have evidence that the animals feel the emotion of regret.
We talked a little at the talk whether we could actually figure out how to do that.
But for deliberation, I'm much more confident, actually, that we have the pieces of that. Yeah.
I think the key here is that this new viewpoint, and it really is,
to be honest, new in the last 20 or 30 years, that the way to understand the
brain is as a computational device,
and not just in some sort of digital computation, but in the mathematics of
analog components, that it's actually performing some sort of fundamental computational process.
And that's that what we call mind is, in fact, also a computational process.
And that doesn't diminish, in my view, the who you are of a person.
I'm very happy to be commander data. Sure, I have no problem with that. Right?
And so, but if we have that computational process, then we should be able to access it.
And so to me, the key is that those processes make very specific predictions
about what the neuron state should look like.
And so we can go in and look for those neuron states, checking those predictions.
And I think I would not want to say that we just can go in and look, right?
I think the key here is a full interaction of theory and experiment.
We need a theoretical neuroscience.
That would be very good, yes. I agree. But then what is interesting,
though, is that now you do – actually, you mentioned the word computation a lot. Yes.
So apparently you see that as a bridging level of description.
Are you having in mind a specific set of computational operations?
Or do you use it in a loose way, like some form of transformation?
I mean in a loose way, in some form of transformation. I do not mean specific.
What I mean is that mathematics and computation about information is the correct
way to describe this process.
And certainly we can also describe the process pharmacologically,
we can describe it chemically, we can describe it physically.
But I think that the way to understand what the brain is doing.
Is through a transformation of information.
So the idea is to take, in some sense, a computer science view,
which is that you have representations, you have algorithms.
The algorithms are not necessarily digital algorithms. The representations are
not necessarily digital representations.
But that you have, the question is, what are the representations?
How are those representations transformed from structure to structure?
How are those representations encoded? And that that language,
that mathematical language, is the bridge to connect kind of psychological states with neural states.
So then before we get to the finish line, in some sense, you could also look
at your description of the brain as making it actually fairly simple.
Because we have these different modules that perform certain operations.
I have a memory in my hippocampus. I have valuation in my obitofrontal cortex, ventral striatum.
I have some rule-based integration, a prefrontal cortex.
And as long as I just decode what these different subsystems do,
I can just glue them together and I understand how the brain works.
I don't think the glue is so simple. Okay, tell me. Is that where the secret lies?
No, I think there's important questions in all these components.
Both, I mean, to be honest, I want to be careful about saying,
you know, that we've solved the brain.
We certainly haven't solved the brain. There's a lot of work to do,
right? To say that we've identified that there exist future representations
in hippocampus doesn't tell us how those future representations are generated. We talked about that.
So there's a whole question of how does this computation happen?
One of the things that's very exciting in terms of the gluing is there's some
very exciting data coming on suggesting that there's dynamic gluing.
That structures will talk to other
structures by matching oscillations and
by sometimes they'll connect up and sometimes they won't
uh one of my favorite stories is the
neuromodulator stories in the invertebrate literature in which neuromodulators
basically completely rewire the network cells that were oscillating suddenly
are not oscillating cells that were inhibitory are now excitatory it's almost
like you have multiple networks hippocampus works the same way,
In the presence of acetylcholine, the entorhinal cortex is driving most of the inputs.
The recurrent inputs in CA3 are weak but have stronger LTP.
In the absence of acetylcholine, right, hippocampus is doing more internal generation.
The entorhinal inputs are weaker. The recurrent connections are stronger.
And hippocampus is driving more to the deep entorhinal outputs, right?
This is work, among other people, by Mike Hasselmo in the 1990s.
And so you've got this where these computational states are really complicated, right? Right.
So I don't want to trivialize the module story, right?
But I still think it's a computation question, right? So the advantage of the
acetylcholine from the Mike Haslamo story is that it prevents interference during storage.
It's fundamentally a computational explanation for this process.
Okay. So tell me, David, look, you're really leading the pack in a lot of this work, I have to say.
Thank you. And this sort of system-level understanding of cognitive properties of rats.
So if we'd like to follow in that in your tradition, what would be the radish
law of brain science that we should adhere to? Yikes.
That's a hard one. Bring in everything?
I think to me, the key is to be able to bring in theory, to bring in the computation,
to bring in the experiments, to have all of it talk to each other,
and to try to actually build this conjoint interaction.
And to say, you know, how does the, you know, how does this,
I mean, you really want a full loop is to me the key.
You want the theory making a prediction that you then test with the experiment,
that you connect up with the modeling, that, you know, then changes your theory.
And this whole cycle and thinking of it as a system is to me the key,
though I'd hate to call it the Reddish law.
Okay good then look tony here likes traveling
um and i don't think he has been
to minnesota yet so four years from now i'm gonna ship him
to minnesota uh cool low cost
but he's gonna get there one way or the other um and
he's gonna he's gonna have a piece of paper in his hand that said i came here
to test the hypothesis so what's the one hypothesis that your prediction that
that you want to make today that Tony will come and check out four years from
now to see whether you really tested it and what kind of outcome you found.
Specific prediction specific prediction um.
I'm not sure I could give one quickly like that, but what I could tell you is
what the key question is that we'd like to be asking, which is, how do you actually.
Integrate and decide when you have a conflict between decision systems?
To me, that's the ignorance question.
I really love the Stuart Feierstein ignorance point, that what science is about
is questions, right? And finding the question.
And to me, particularly questions that you didn't know were questions before
you started working, right?
To me, that's the question that I didn't know was a question, right?
Until I actually had the point, you know, until I, it wasn't me,
until, you know, the field had gotten to the point where we had these multiple
decision systems, asking how you interact between decision systems is a meaningless question.
To me, that's the the question i would love to know if tony comes and says that's
the question have you answered it i would be ecstatic if i if i had an answer
to that all right great david radish thank you very much for this conversation
thank you for having me thank you,
are we up yeah that was intense the csn podcast was produced by the convergent
science network of Biometrics and Biohybrid Systems,
a project funded by the European Sevens Research Framework Program.
For more interviews, recorded lectures, or upcoming conferences in the field
of biometrics and biohybrid systems, go to csnnetwork.com.
Music.