I have a conviction that the brain is understandable.
You look inside these AI models, and there's
some clarity coming up, but it's also super confusing.
So I believe that there's a huge landscape of
different architectures that are able to solve
these tasks, and we have two points. There's
Chachi PT and Dali, and there's the brain. I
think the brain is on one extreme of efficiency
and understandability, and that's what I want.
That's what I care about. I'm John Foxe, Director
of the Del Monte Institute for Neuroscience here
at the University of Rochester, and I'd like
to welcome you to another episode of Neuroscience
Perspectives. I'm absolutely thrilled to be joined
by renowned neuroscientist Professor Doris Tsao.
Her pioneering research has transformed our understanding
of how the brain processes visual information.
And her lab helped us discover that it takes
only a few hundred neurons to encode any given
face. For reference, as many of you will know,
there are billions of neurons in the brain. Dr.
Tsao is a professor of neurobiology. at University
of California, Berkeley, and she's been the recipient
of numerous awards, including most recently the
Kavli Prize, one of the most prestigious international
science prizes there is. She's also been elected
as a very young scientist to the National Academy
of Sciences and was named an investigator of
the Howard Hughes Medical Institute, which is
among the highest of distinctions for biomedical
research. Doris, thank you so much for joining
us here on Neuroscience Perspective. I'm dying
to get into things with you and ask you a bunch
of questions about yourself. But let's kick off
with your research. You know, why faces? Why
the visual system? Where did that all come from?
probably high school when I realized that the
visual system, the neurons are what generates
our perception of everything. But I came to faces
kind of by accident. You know, I was a graduate
student. I was recording in V1. I was initially
interested in this question of how the brain
represents space. and I wasn't getting anywhere,
so I got into fMRI, and I read about Nancy Kanwisher's
work that reported a face area in the human brain,
and it just seemed like a fun thing to do. Let's
just show faces and objects to a monkey and see
if monkeys also might have a face area. And then
you found it. they didn't just have one face
area. They had a whole system of face areas.
Yes, yes. And so it took off on its own. You
know, every question kind of naturally led to
the next question. It's like, yeah, so there's
six face areas. What are they each doing? Are
they each doing something different? Yeah, yeah,
yeah. Tell us about these six face areas. You
know, why six in a monkey? for faces yeah we
we wondered about that for a long time and um
a few years ago i think we discovered the answer
basically there's a whole map of object space
so object space you know same way we have 3d
space we can represent you know different points
and locations so you This part of the brain called
IT cortex represents objects also in this more
abstract space that's also two -dimensional,
and there's repeated copies of it, and the representation
becomes more and more invariant, and so you have
these multiple stages, and that's why you have
these multiple copies of this map, and faces
are one part of that map, and so when you just
look at faces, you see these six face patches.
I got you. Okay, so it's six levels of complexity?
Is that the way to think about it? Yeah, I think
that's... But highly... They're part of a circuit,
right? Yes, yes. So they're interacting to evolve,
and eventually then... an individual face I mean
I think we all remember these stories about like
the Jennifer Aniston neuron and that you know
when you get down to the level of like dissociating
two very similar faces from your life The representation
there is actually very limited in terms of the
number of neurons that are involved. Is that
fair to say? Or is that not the right way to
think about it? Yeah, so face carries so much
information. There's information about the physical
features. There's information about the identity,
the expression. And I should say this is still,
we're still figuring all this out, right? It's
still early days. But indeed, you don't need...
that many cells to represent the identity, which
is kind of surprising. You look at a face, it
just, you know, it feels so complex, right? It's
like ineffable, like who knows how many neurons
that would take. But it turns out that with just
50 numbers, and this is something that computer
vision people discovered with like 50 numbers.
you can, like, recreate the face, right? So you
can really compress the information down. And
it turns out that the neurons use a code that
is very similar to one that the computer vision
people figured out for how to compress the information.
Gotcha, yeah. So this sort of dimensionality
reduction is instantiated neurons in much the
same way that one can do it with a formula in
silico in a computer. Exactly. And, you know,
in recent years, like with Dali, right, where
you can, like, you know, specify any prompt and
it can create a picture. of anything that you
can possibly imagine um that idea of being able
to compress reality has really taken off right
and they're kind of an existence proof that you
know like like the space of all possible for
example 500 by 500 you know rgb images it's like
it's more than the number of atoms you know in
the universe right but you know with just this
like one gigabyte you know neural network like
you can create any like any picture of reality
and the reason is because reality is much lower
dimensional than the space of all possible images
right so much of that space is just like random
noise right but then there's like Yeah, the space
of reality, of objects. And there's beautiful
ways to discover those spaces that these machine
learning people are figuring out. And obviously
the brain has figured out how to do that with
its limited neurons. And so that's what we're
trying to figure out. You know, and all the buzz
is about AI at the moment. And you made a case
yesterday, and I'd be interested to hear you
sort of extrapolate on that a bit, that actually
it's still very important for us to understand
the neurobiology of vision because we have a
lot of insight. of how that's achieved to provide
to AI folks and that there's still a place for
neuroscience in the face of AI. I mean, it is
incredible. You know, our brain runs on 30 watts
while you hear about them, like, OpenAI building
nuclear reactor farms to power, you know, next
generation chat GPT. So I think the brain is
so much more efficient. I have a conviction that
the brain is understandable. And, you know, you
look inside these... AI models, there's some
clarity coming up, but it's also super confusing.
So I believe that there's a huge landscape of
different architectures that are able to solve
these tasks. We have two points, you know, there's
Chachi PT and Dali and there's like the brain.
I think the brain is on one extreme of efficiency
and understandability. Right, right. And that's
what I want. Right, these AIs are built on the
part that you can take brute force, super fast
computers and a lot of energy and just churn
through billions of calculations. Exactly. And
they just impose this one arbitrary architecture.
You know, what is the likelihood that that is
the architecture, right? Like, I just really
believe that the evolution figured out something.
tricks that we can still learn from. Here's a
thought I was thinking about technologies and
the way in which you sort of have pulled together
multiple technologies to address your question
and we're sort of living in this like extraordinary
time right particularly over the last couple
of decades as you've sort of built out your career
where you know fMRI came online high field magnets
extraordinary electrophysiological techniques
how much is that driving and do you see Do you
see AI, for example, now interfacing with that
and really driving forward science in a whole
new way? Oh, yeah. I mean, absolutely. I'm sure
you know this quote from Sidney Brenner, right?
Progress in science is driven by new tools, new
phenomena, and new ideas in that order of importance.
That's good. Sadly, it's true. And I was just
so lucky when I was in Boston as a grad student.
landed at Mass General Hospital where there were
all these experts on fMRI, you know, the Martinez
Imaging Center, and that was, like, so important,
right? Because these face patches, like, just
sticking an electrode into the temporal lobe,
you would never find them. Or you might, but
you couldn't then find it again. Whereas this
was, like, you know, it just gives you a systematic
way, right? fMRI gives you this bird's -eye view
of the brain so you can see all the, you know,
loci that are important, then you can go in and
study the single neurons in detail. Right, right.
Around those same times. people like Greg McCarty
and Ina Puse, Truett Allison, were using ECOG
and seeing fusiform face area. Yes, they were
the pioneers that really suggested there was
something profound down there. It's amazing.
I mean, having done some of those recordings
myself, when you have an electrode sitting over
the fusiform face area and you show it a face,
I mean, it announces itself. It's really amazing.
Okay, good. So we share that. Yeah, yeah. When
did you decide you wanted to be a scientist or
you think this is where I'm going to end up?
You know, I wasn't interested in science very
much as a kid. I was not really a curious child.
I played with Barbie dolls. Really? Yeah. I love
to read. And so I always found the stories, the
biographies of scientists very inspiring. These
were people who came from humble backgrounds,
but just by working hard. they could you know
achieve glory so that seemed like a very accessible
path to glory to me um and then It was after
I got accepted to Caltech. And, you know, I was
worried I was going to fall behind. And so my
senior year, I started reading the Feynman Lectures
on Physics, Volume 1. And that was when I understood
what science was about. It was so beautiful.
It's just like this freedom to ask any question
you want and use your brain and think through
it logically and come to an insight. Those Feynman
Lectures, you can get them on YouTube. And he
has this fantastic Brooklyn accent. And they're
really wonderful. He's such an entertainer. Recommend
them to people. You don't need to be a physicist
to enjoy Richard Feynman telling his stories.
Amazing. Now, but your mom and dad were academics?
So, yeah, we came to the U .S. So they were graduate
students at the University of Maryland. My mom
was a computer programmer. I wouldn't call her
an academic. I think she was my mom. And by my
dad, you know, he was like he, you know, during
the Cultural Revolution, he was sent off to Manchuria
to like chop trees. And he always knew that that
period would end and there would be a new day.
And so he, you know, brought a thick stack of
math books with him, like studying. functional
analysis in the forest. And so, you know, after
the end of the Cultural Revolution, you know,
he got into Peking University and then we came
to the U .S. And so, yeah, he's a mathematician
and always, like, just deeply curious. You know,
he's 82 years old now. He works harder than I
do. Like, whenever I go in his room, he's, like,
watching a YouTube video, like, learning a new...
Recently, he got excited about this area of math
I've never heard of called sheaf theory. And
it's, like... Apparently, it's about how you
can, everything is defined by its relation to
other things. And he's telling me how that's
like the essence of Marxism. And so that's the
kind of person he is. So mathematics to political
theory. Yeah. Just a brilliant mind. Yes. And
an inspiration then to you both, both the practical
part of computer science that your mom. brings
to the table and the inspiration yes yes you
know my mom actually like programmed some stimuli
for me when I was yeah setting these bases yeah
so yeah so a big move though to come from China
to the US in the in the 80s was it yeah 1980
1980 so right yes yeah and uh how much of that
do you remember you must have been extraordinarily
young at that point I remember yeah I remember
this night we were in Hong Kong I remember tasting
chocolate for the first time on that journey
somehow um yeah the U .S. you know when we first
came my grandfather he had been here for many
years and so he would have these like American
culture evenings, apparently. He would teach,
because I came with my cousins and their family,
so he would teach them about all the American
traditions, like this is football and here's
the rules and stuff. Wow. Were you five or six
or something? I was four. You were four, right.
And yet, very profound memories of that, obviously,
because it was such a big change. And how much
of your very early childhood in China do you
remember? Yeah, vague, vague glimmerings. And
you studied mathematics, though, in university,
right, as well? Yeah, I studied math and biology.
I really love math. Yeah, I wish I could, like,
if I had a secret wish, it would be, like, if
I could go do a math PhD, yeah. Look at that.
Well, maybe you will, like your dad at 82. Yeah.
That's outstanding, yeah. So then you went to
Caltech, you did your undergraduate there, and
then you went to Harvard for graduate school.
And then you ended up in a very important lab
or very famous lab, right, with Marge Livingston.
Tell us a little bit about that and the impact
Marge had on you. Marge is an incredible scientist.
She's such a true scientist. You know, the first
time I met her, like during the graduate school
interviews, she was recording and she kind of
like. had to finish her experiment then she looked
up at me and said hello and then she was like
so um just direct you know she said what are
you interested in i told her something vague
about you know i want to understand neural circuits
and she said well if i were interested in neural
circuits i would go to larry katz's lab maybe
this is not going so well but you know she just
like intuitively understood me you know it was
yeah i am very shy i was very shy i remember
i was like Marge, can I talk to you about something?
Because I wanted to join her lab. And I didn't
have to say anything more. She said, I ordered
an amplifier for you already. Yes. So it was
wonderful. She just does her own experiments
every day. And she's taught us by example. Yeah,
true. one of the great leading vision scientists
in the history. She's incredible. She started
as a Drosophila neuroscientist, discovered these
learning mutants, and then she went on to, she
became so interested in vision, she felt like
she found her calling. And she's also done incredible
work understanding the relationship between art
and vision, how artists... And so her mentorship
style, she's sleeves rolled up, in the lab making
recordings. Oh, yeah, yeah. And have you taken
that into your own lab? How do you approach it?
Not as much as Marge. Yeah, I don't know how.
I think I'd like to. Yeah, I do try, but just,
yeah, time management. That's an amazing trajectory.
And then you did something. A little strange,
too, because, you know, most people, you know,
the American science engine. But you took off
for Germany, right? And I went to, was it Bremen?
Bremen, yeah. And what was the, like, why? What
was the decision there and how did that work?
I got a fellowship from the Humboldt Foundation
to set up my own little lab. This was, you know,
I got my PhD and I spent two more years in Marge's
lab. But then, yeah, it was like an amazing opportunity.
So you didn't do an official postdoc. Yeah, yeah.
So, you know, we found these space patches and
it just like there's so many questions. And so
that was like an amazing opportunity to keep
studying this on my own. Yeah. So off to Germany.
And I'd always been interested in German culture.
You were. Okay. Where does that come from? I
think it started with music. Like my father,
he was always listening to Beethoven. I see.
Yeah, yeah, yeah. And was that culture shock?
Was it different? Did you settle easily? I loved
Germany. It was like a much simpler place. You
know, Bremen's a small town. There's like one
department store in Karstadt. So there wasn't
much to do. So I found a violin teacher. I fell
in love with the violin again. I loved it. Yeah,
yeah, yeah. It's a brave move. I mean, to get
out of your PhD and your own milieu and the place
that you've sort of trained and to take off for
Europe to a foreign country. launch a science
career I mean and are there lessons in that for
example for for your trainees is there something
about that streak in you that's willing to take
a chance that's part of how you approach science
I mean every path is unique but I think um yeah
I've never worried too much about I've just never
worried too much you know about the future I've
always had this like irrational exuberance and
so I try to like tell my trainees yeah yeah just
like follow follow your heart what you want to
do everything else will take care of itself yeah
yeah you gave a wonderful talk yesterday and
I really appreciated the part where you made
sure to point out exactly who the folks were
that did the work, the graduate students and
the postdocs, and give credit where credit's
due. And it's obvious that you've put some very,
very talented people through your lab. Oh, yeah.
Everything is due to them. It is such a privilege
to be able to work with these brilliant young
people. Right. It begins and ends with that sort
of vibrancy in the ranks and training people.
There's nothing better, right? There's nothing
better than to be slacking furiously about a
new idea. with a student and just fantasizing,
it's, yeah, it doesn't get better than that.
I love it, love it. Here, I read a quote, just
going back to mentorship, and we were talking
about some of these leading elders in the field,
you know, and this is a quote, I love this. It
said, all of us desire to find a mentor who sees
us as we wish to be rather than as we are. That's
quite profound. Would you unpack that for us?
What's that? What drove that? That was about
Marge, my PhD advisor. I felt that's how she...
treated me like i was like a child not her like
you know oars women for her grant yeah right
right um not just yeah the engine room right
yeah she like even when i was a young faculty
member like you know caltech was starting out
and she she said i said something about i was
like still studying math and she she sent me
an email and she's like you know i i you should
keep doing that i believe you can do something
great you know in that realm and it's like she's
always encouraged me like this side of me that's
not like maybe conventional I don't know maybe
is it fair to say then you know there's there's
a pragmatics part of mentorship in a PhD you
need to learn this material you need to know
how to do this technique you need to be able
to and then there's the the humanocentric piece
of it there's the development of the individual
through these you know early 20s into a fully
formed human and and that real mentorship sees
those two things equally yes like I think that
like you know everyone in our field comes in
with this like ambition right this raw ambition
and like you have to nurture that like yeah and
i think like somehow you know like on the outside
i think it's like this like soup crazy shy you
know girl like just shoveled hair and all that
and it was like but when march like saw this
like you know this is like inner flame and she
like nurtured that and i i i hope i i do that
for my trainees because they yeah they have their
wild dreams and i think it's so important right
yeah yeah developing the human as well The shyness.
Tell me about the shyness. Is it gone or is it
still there? It's totally still there. But you've
learned to conquer it. I've learned to cope with
it. We're talking right now somehow. I've seen
you give in talks so many times and you're a
very confident presenter. And I think if somebody
just saw the talk... they wouldn't know the shy
Doris. Oh yeah. No, when I go to a conference,
like there's always like this moment when hotel
lobby, I see all these colleagues and I just
like feel this dread. Oh my God. Even the most
extroverted people. Well, I think even the most
extroverted people get that. Oh really? I do.
Yeah. And then, yeah, I do. That's true. Oh,
I certainly do. Yeah. I was very, very shy youngster.
I sometimes say that to people and they're like,
no, that's ridiculous. But, but it was, you know,
but, and I, and I, I must say that's a, that's,
I often have that, like, you know, you walk into
the reception at one of those meetings and I
go, oh, gosh, what if I'm just standing there
alone as well? Just like, you know, you wade
in, I suppose. Yeah. So you've had to conquer
that as part of it. Because obviously the other
pieces, you know, there's the doing the science,
but the communicating part is equally important,
right? Yeah, yeah. I have to. Yeah, again, thank
my mentors for that, like teaching me how to
give talks and so on. You know, we're living
in strange political times for science. And they
sort of, you know, the public perception of science
as, you know, there was a time I think I read
recently, you know, if you go back 10, 15 years,
public trust in scientists was in the high 90%.
98 % of people had complete trust in the system.
I think those numbers would not be there today.
What are we doing wrong and how do we get at
that? How do we get at science communication
that brings our public along with us? Do you
have thoughts around that? Yes, I think we need
to tell people about our science, make many more
efforts to do that. And I think this podcast
is a great way to do that. I myself am like writing
a book now. I'm trying to link neuroscience,
philosophy, and self -help. Because I think that
there's this whole genre of neuroscience -based
self -help that's around how you can sleep better,
how you can eat better, and so on. But I'm interested.
I believe that the mind -body problem, insights
into consciousness, into free will, those insights.
don't live in some pure realm, right? They're
about the brain and the brain is what we used
to behave with. And they have like profound implications
for how we should live our life. And I want to
work that out. And I, so yeah. So you're waiting.
solidly into this space to bring neuroscience.
Yes, I think yes. And I hope then to go out into
the Oakland Unified School District where there
are many underrepresented students who may not
be thinking about careers in neuroscience, but
they do care about how should I live my day tomorrow?
Do I have free will? I think they would be totally
captivated by that question if they could discover
that there's a link between that and electrical
activity in the brain. Maybe they will change
their career trajectory. Outstanding. Thank you
so much. Thanks so much for spending some time
with us and allowing us to get to know you a
little bit better, Doris. You're a major star
in the field. Look forward to watching you in
the decades ahead and see how the work evolves.
Thank you so much. This was fun.