Welcome back, everybody, to another episode of
You Make Me Sick. Today, joined by a very, very
special guest. We're joined by Dr. Adam Yala.
Dr. Yala is an assistant professor of computational
precision health at UC Berkeley and UCSF. His
research focuses on developing machine learning
methods for personalized medicine and translating
them into clinical care. His previous research
is focused on two areas, predicting future cancer
risk and designing personalized screening policies.
His breast cancer tool, Mirai, did I pronounce
that correctly? It's a valid way to pronounce
it. I used to say Mirai. Mirai? Mirai. All right,
Mirai. Has been tested at 43 hospitals from 14
different countries. Adam's tools now underlie
prospective trials, and his research has been
featured in the Washington Post, the New York
Times, and the Boston Globe. He's also been published
in a number of scientific journals, including
the Journal of Radiology. Journal of Empirical
Methods and Natural Language Processing, the
Journal of Clinical Oncology, Nature Medicine,
and the Journal of Breast Cancer Research. Dr.
Yala got his undergrad, his master's, and his
doctorate all from MIT. At MIT, he worked in
the Jameel Clinic and the MIT CSAIL. He's here
today to talk about not only his work, which
is fantastic as far as cancer detection using
AI, but also just to touch a little bit on artificial
intelligence in healthcare, some of the boundaries
that we see right now, some of the barriers that
may be stopping AI from being introduced into
healthcare and some of the possibilities. So
Adam, thank you so much for joining us today.
Thanks for having me. It's a pleasure. So I guess
I'll kind of start with, you know, artificial
intelligence is pretty much everywhere. It's
in almost all aspects of life right now. And
for people who are listening, I think sometimes
there's some confusion as to what artificial
intelligence is. And I was wondering if maybe
you could explain exactly what's meant by AI
and then how your work may utilize it a little
bit differently than something compared to chat
GPT or grok or perplexity. Yeah. So broadly,
I define AIs as such technologies as designed
to learn patterns. As I specify behavior, which
might be through data, like here is a mammogram.
This person got cancer five years from now. That
is a pattern in the data. Now find me a way to
predict that pattern and replicate it. If you
think of chat GPT, like the analogy there is
like I have language. What's the next sentence?
What's the next word? And that's a very flexible
set of concepts because the world is filled with
a combination of data A, data B. Can you learn
the mappings between them? In the way in which
my work is different from what we often see in
industry, we have a lot of work in industry that's
like, how do you do something people already
do, just maybe faster, maybe more consistently?
How do you automate known behavior? That can
be very valuable and has a lot of use cases in
healthcare. A lot of what I spend time thinking
on is how do you do things that humans can't
do? So to go back to the example of cancer risk,
people can't look at the mammogram and say, this
person's going to get cancer in three years.
and here's a subtype. That's not a thing people
can do. But there's a pattern in that data. And
so we're developing technologies to kind of find
it to make something useful out of it. And I
think a lot about how can we can come up with
new kinds of workflows that are better than the
ones that are there now to make outcomes better.
So kind of with that, I'd like to touch on your
work that you've done. So you did some work at
MIT and now at your own lab at UC Berkeley. Initially,
I think it was a... a project called Sibyl that
you would work on. And this was using kind of
some of the information you were just talking
about, getting a number of, I think it was CAT
scans that you followed through. And by feeding
that into an algorithm, seeing if they could
predict out years in advance lung cancer, correct?
Yeah. So that model was in, I forget which year,
it was a little bit after the breast cancer work.
But the concept there is very similar. It's like,
so let's step back, what's the problem we're
trying to solve? In every screening program,
there's kind of two things you want to do. You
want to read the current scan well, and then
you want to figure out when to get the next scan.
Because if you read it well, cancer is already
gigantic. It's already kind of too late. So you
need to have like foresight to have known to
get the scan when it was still small. And for
that, you need to get the risk model. And that's
what we're trying to do in lung cancer with the
LDCTs. And so, yeah, let's talk about kind of
your lung cancer research too. Sorry, your breast
cancer research. Your current model that you
use right now is actually being used in a lot
of hospitals. I was kind of curious with regard
to that. So it's a deep learning model. And I
know that the study that I read, you had just
a large amount of data you had to pull from.
So I think a lot of people, and maybe you can
talk a little bit about this, when actually creating
one of these deep learning models, just how much
data you have, will that affect exactly how...
I guess how accurate the model will predict.
So I think there were, was it 150 ,000 different,
was it mammograms or CAT scans that you guys
used? Yeah, it was 150 ,000 mammograms. So based
on that, and you feed, would you need, I guess
the question I'm trying to ask here, and maybe
you can explain this too, is based on that, how
well were you able to kind of create an algorithm
or put it in the algorithm and predict cancer
risk in the future? based on that larger volume
of mammograms. Yeah, so it's easy to understand
these numbers in relative terms because people
don't have a good baseline of what is a good
model versus a bad model. I'll give you first
raw numbers and we'll contextualize them. So
normally the way we think about cancer risk models,
we use something called ROC, AUC as a metric.
It's a number between 50 and 100. 50 is random,
100 is perfect. And what it means, like if someone's
going to get cancer, what are the odds you're
going to have a higher score than someone who's
not going to get cancer? That's its probability,
50 to 100. The state of the art and what's been
like used nationally for a long time, like the
national like NCCN guidelines, the ACR guidelines
or whatever, are all built on models that have
an AUC of like 60. So I guess better than random,
but like a little bit, you know, not a huge amount.
I bet it's what we have today. And like, because
age by itself and breast density and these other
kinds of people are used to just aren't, don't
have all that much information. Mirai, the model
that I proposed in that work that we've been
validating a bunch of places now in these projected
trials is like ballpark 75. Now there's still
room to grow. And even though we use a lot of
data, like we use data from like 50 ,000 patients,
150 ,000 mammograms, in the scheme of things,
compared to the data that we generate in this
country, it's nothing. right like we take like
what 40 million mammograms a year uh and if you
look at the way like in deep learning we talk
a lot about overfitting like what's what's the
performance of the training sets versus the kind
of testing sets that you you're generalizing
to like the model is still like math they could
massively benefit from much more data than they
have now and so i think like we're still kind
of an infancy of the space which people to do
much much better than we are now And with the
current, I know that you're in hospitals right
now. With that data that you're collecting, I'm
assuming that'll get fed back in and kind of
help to just improve on your algorithm and Mirai
right now, correct? Ish. So most of the data
that exists is always data in the past because
you don't just need to have the mammogram. You
need to know what happened to this person within
the next five years. And so let's say the trials
that were running. you're collecting like tens
of thousands of new scans, you're getting high
risk scores, you're figuring out who's going
to get an MRI. For that data to be able to be
used in the training pipeline will take like
years. And it's a small fraction compared to
all the data accumulated across the last couple
of decades. So really like the way to kind of
scale data sets is more by getting more people
on board because you want to get the last 10,
15 years of data. Whereas the next month of data
is like a drop in the bucket. You see what I
mean? Sure. Yeah. So just very, very small, like
you're saying, compared historically to all the
data you can collect and pull from. Another great
thing about this study that was brought up, a
lot of times, sometimes there are ethical concerns
with using AI in healthcare that there's bias.
And by bias, I mean, if you just have one population
with the same geographic area, then there's a
possibility that they're going to be different,
have different outcomes than somewhere else.
But for this study, you guys actually geographically
selected multiple hospitals in multiple geographic
areas around the world, correct? Yeah, yeah.
It started off with a lot of cold emails, and
some dear friends helped me out to find more
possible collaborators. I flew to Taiwan. I met
Gigan Lin, who's a great collaborator at Chang
'e Memorial Hospital. So we did a validation
there. I flew to Sweden, met Frederik Strand,
who's a great collaborator. We validated at the
Karl Linska Institute. uh and we've since now
validated across ballpark 40 hospitals for more
than 14 countries uh and what we're finding is
that as you kind of go across the things are
working out of the box uh so we know that the
finding that we're finding like the pattern that
we learn mgh mammograms we originally built the
model do generalize they are general which is
good uh but uh so it's kind of always two threads
like given that technology you have today how
do we make it most useful? What are the trials
we can run now? How can we deliver on the promise?
And the other third has been a lot of time. I
was like, okay, well, we didn't peak in 2019.
We can do better. And how do we build it next?
So how did you guys come about when you first
started theorizing what you could do using imaging?
And this probably goes back to your days at MIT.
What was the initial kind of catalyst for, well,
if we can incorporate artificial intelligence,
using historical radiographical data you know
what outcomes could we find could we help detect
future cancers could we find could ai find things
on the radiographs that a radiologist would miss
uh like when this oh when you first started your
work what was kind of the initial thought process
behind what you guys could actually create yes
i mean a lot of the uh i started working with
cancer because of my pc advisor guida's personal
experience with cancer and the The idea of we
could cash cancer better if we knew who's going
to get it is not a novel idea that we just came
up with. People have been thinking about this
in one way or another since like the 60s. And
like even in the 60s, people were saying, oh,
there's this thing in the mammogram. Like it
looks a bit different if someone is going to
be riskier versus not. And then eventually, like
across decades, decades of work, they eventually
hit like federalization in 2019. And like coming
from the AI world, it's like. You know, you go
on Amazon, and you can bet that, like, so many
of the decisions you've made ever are influencing
what's shown to you, your ranking and your ads
and whatever. And, like, inside the healthcare
machine, there's very little information used
to say anything about anything. And every time
you kind of look more deeply at a problem, like
how we thought about risk, and you kind of quantify
it and view it as a prediction problem, you realize,
like, there's just a lot of room to do better.
Like, even, like, one of the most important decisions
we make right now is who do you screen in the
first place? before you even get the mammogram,
what should someone start getting a mammogram?
That is a decision. That's a prediction problem,
right? Like it's a true positive. You give someone
a mammogram, they eventually got cancer, so they
caught it earlier. And it's a negative if you
didn't give them a mammogram and they got cancer,
so they get caught later, or you gave it to them
and they never had cancer, right? It's a waste
there because all you did was bother them. That's
a classifier, right? You can evaluate how well
you predict that. And we only use age. Which
is, yeah, it's kind of a crude metric, you know?
Do we only know your age? Especially if you're
like a patient of this hospital, you know, so
much more. And we don't use any of it. And so
when I meet people who are like outside of screening
guidelines, like before 40, they have late stage
breast cancer, like we could have done better
for those people if we just built the right technologies.
When you have people in screening that like still
get late stage breast cancer, still could have
done better for those people who had the right
technologies. And so there's a lot of room to
do better. And the deeper you go into it from
a technology perspective, the more you're like,
the tools we have are crude. And even now, with
the best of AI we have now, even now we still
can't touch all of it. Because we have it actually
really big. We have so much about people. We
don't use it. When you're feeding this information
into your algorithm, is it just the radiographic
image? Or do you also have anything else? Have
you thought about or thought about adding? It
might be difficult to do, I don't know, but for
each mammogram also have, you know, lab work,
history, do you have family breast cancer history,
any kind of, I don't know, some people get genetic
testing, any of that information, could that
also be kind of added in, you know, to your algorithm
to help predict? Is that something you guys have
thought about or would that just be too much
at this point? We've looked into it. So that
part is, okay, so. let's say you're combining
the factors people know matter. So family history,
questionnaires about like reproductive health,
there's a person breastfeed and whatever, like
all this kind of like detailed questionnaire
information. If you add that back to the image,
it barely makes a difference. Like you go from
75 to 76, it's like it's not a big gap. And I
think it's because the questionnaires themselves
are too coarse. And we did some experiments.
It can actually predict a lot of these factors
from the image. So, like, if someone had children
before, it can predict from the image. If someone
breastfed before, it can predict from the image.
Like, there's a lot of things like that that
are, like, readable from the mammogram. And so
adding it back into this questionnaire doesn't
really help. I think there's other stuff that
we're not currently incorporating that were the
challenges of having all the data in the same
place that's not currently there that I think
would be powerful. So we had this collaboration
with some colleagues in Denmark where we're building
risk models looking at, like... their entire
health trajectory, because they have these big
national registries, to predict when someone
should start screening. And that stuff is pretty
good. And that is much better than things like
age, and that should be more complementary to
the mammogram. But the classic epidemiological
questionnaire stuff, that's underlying the various
cancer risk models, doesn't add all that much
compared to just the image. Which is pretty impressive
when you think about it, because you would think,
you know, even having a little bit history, I
thought it would sway it a little bit more. But
I guess just the radiographic image seems to
be kind of where you're getting most of that
data from. How, as compared to, you know, a radiologist
who would read this, how well does the AI kind
of detect it in comparison? And I'm sure, I know,
I think I read that you guys, I can't remember
if it was this study or it might have been the
Sybil study. So you had a radiologist kind of
take a read as well, and you compared it with
the AI read. And the AI read is very close to
what the radiologist would catch. Do they catch
more things than the radiologist could catch?
Yeah, so the lung cancer model is a little bit
newer. So there's two questions. One is how do
you find current cancer versus future cancer?
Radiologists. as it is their above part of their
job, are pretty good at finding current cancer,
at least the ones that are specialized in fellowship
training, et cetera. And in the evaluation that
we did, Seibel was fairly close to that performance.
And Mirai is not as good as a breast imaging
radiologist, partially because they rely more
on priors and stuff. That model is a little bit
older. Today, you can get quite a bit closer
to that than was possible when I first did this.
When you try to understand future cancer, People
can't really do this. It's something they're
trying to do, but really they can't. That's why
people came up with notions like breast density.
And in there, you do way better. And so that's
where the technology has been focused. The detection
capability for both MIRA and SIBO are kind of
like an ancillary add -on that kind of came out
of the process. The core focus is understanding
how do you understand who's going to get cancer
and what can you do about it. So that's how we
were trying to change the workflow. We weren't
trying to change how someone's reading it today.
is how do you schedule the next person so you're
getting rid of the best possible time. In the
current phase, we're doing more of everything.
So now we're also trying to improve the workflow,
help the person read it, make them more efficient,
making better caching stuff in addition to all
these other longer -term components. But that's
the newer generation of stuff. Do you think that...
You have, what did I say, 14 different hospitals
now we're trialing. your algorithm to detect
breast cancer. Do you see a day where this is
something that's going to be used just as kind
of almost like the gold standard as far as predicting
breast cancer in women? Or is it something you
feel is, you know, is that years away? Is that
something that could happen relatively soon?
The real challenge we face now is more I mean,
there's lots of protective trials that we're
running, and I think there's a lot of stuff to
learn clinically about how to use these tools.
And these things take time. You need to follow
patients for years. You need to roll with consent.
There's no way to supercharge that stuff in speed.
But the bottleneck to have this widely accessible
is more regulatory than anything. So the FDA
has some constraints. The way the law is written
is that if an algorithm looks at an image, even
if it's doing the same thing as what is currently
not a device, it becomes a device. So the classical
risk model that people use today that looks at
density and whatever, which looks at an image
in a way, but you enter the density yourself,
that's not a device. And that's what's in the
guidelines. Because you look at the image directly,
that's a device now. And now it needs to have
expensive filings, you need to have a commercial
structure around it to make it feasible. And
I think that has been the major roadblock in
order to have this more widely accessible. and
we're thinking of ways to amortize that cost
because it's hard to... Basically, the commercial
structure, which is only caused by the FDA, is
hard. It means doing this in the open, free way
that we wanted to do it becomes structurally
impossible, which means now we need to lump it
into a larger commercial thing, and we're thinking
about the right way to do that. Yeah, I was going
to ask you, that's actually a question, because
often when medications are first developed, it
takes, you know, years. It could be almost seven
to 10 years of trials and then just review just
to receive approval from the FDA. Are you looking
at that kind of timeline for this type of technology?
Do you think it'll be a shorter timeline? Have
you ever, I mean, I'm assuming this is probably
the first time you've had to go through this
type or these types of steps dealing with federal
regulations, incorporating AI into healthcare
and getting it passed. Do you think it could
take that long to actually implement this? As
far as like almost up to a decade? I think that
is the default in healthcare today. And it is
the grand challenge of one's work to make it
not take that long. And to at least have the
second model and the third model not take that
long and to get faster and faster. I mean, there's
a lot of bottlenecks across the process. Risk
models are used for many things. One thing we
use them for is to say who gets MRI. Finding
the partnerships to even stop those MRI trials
took years. Now that we have them, it takes years
to know patients. It takes years to follow them.
And you're already approaching this kind of like,
to do the final trial readout, I think it will
take closer to, it's a couple of years from today.
And in that, we'll have been thinking about this
for at least five years. And we're super grateful
to the partners that we found. BCRF is a fantastic
sponsor. And we'll not be able to do the study
without them. But at a system level, we can't
have this happen for every time we have a new
model. We need to have faster turnaround to get
information to helping people. Because even when
we have that, it doesn't solve the FDA problem.
The FDA does not even look at trials. It just
looks at other enumerated things. So there's
a lot of work to do across every access. How
do you make this thing financially sustainable
so you can finance improving it and have it accessible
at scale and go fill the regulatory fees they
have to go across to make this widely available
is a grand challenge. In addition to generating
the kind of evidence that we want to redesign
the next generation of cancer guidelines. Yeah,
so your lab at UC Berkeley, you focus on three
major themes just to provide customized care
to cancer patients. I was wondering if you could
explain kind of, you know, each of these themes
and how they kind of work together with regard
to actually that customization of care for patients
with using your technology. Yeah. So to give
that kind of the motivation we want to make possible,
there's a lot of problems in cancer. It'd be
a lot easier if you just had foresight. What
I mean by that is like, okay, if you knew who
was going to get cancer at a particular point
in time, it's easy way to do about it. You just
don't know. And so you get all of these, like,
advanced cancer stages. If you knew which therapy
you were going to respond to, and this was not
going to work, easy, you know? It's how you generate
that knowledge. And that kind of leads to kind
of three classes of questions that my group works
on. One I would broadly call improving predictive
capacity. So I mentioned, like, before, like,
with screening guidelines, like, people use their
age. There's more than that out there, right?
You could use, like, whole health trajectory.
But real data we have on cancer patients is multiple
gigabytes per person. That is way bigger than
anything we know how to model in AI today. Even
the big LLMs, they look at like 100 ,000 tokens.
Patients have billions of tokens. And so we need
to do quite a bit better at understanding things
at scale and super efficient models and to learn
the dependencies and patterns across this really
massive scale. So that leads to new work and
new kinds of neural network architectures, ways
to learn more efficiently, and all kinds of stuff.
How do you better deal with the fact that data
is gigantic? And you don't have 100 billion people.
You only have the people in the hospital that
are actually there. So it's big in that the samples
are really big, but you don't have that many
patients. And how do you learn efficiently in
that kind of regime? That's kind of one bucket
to give you better. And the way you tell us that
worked is now you beat Mirai, you beat Seibel,
you have tumor response models. You're better
predicting the various things we care about in
cancer. That's one piece, but it's not enough.
Because if you just have that, you need to do
something about it. And that's the second bucket
of problem is how do you make better decision
-making given this kind of predictors? i call
that control so i design new kind of screening
policies treatment plans and whatever and the
last bucket because even if you have that that's
not enough because you need if you have a better
predictor you have better action like a better
guideline people still can't use it uh so the
last bucket are called translation and then we're
working on like new kinds of death of trial design
how to make trials more efficient because you
get on many of them uh what's the right kind
of pipeline to kind of take this to the end consumer
and all this kind of stuff it's kind of three
broad buckets from prediction to control to translation
That's a big spiel. No, no, no, no. It's actually,
you explained it really well. And I think that
it's sort of personalization of care. I know
that you were involved in another study. So where
I work, we use Epic, which is a medical record
health system. People come in, say you come to
our emergency department and you have a fever
and a high heart rate and your white blood cell
count is high. It'll set up a red flag in our
system just from those indicators that you might
have a process called sepsis. So a whole body
infection. can be life -threatening. Based on
that, it'll trigger certain labs that need to
be done. It'll notify a team of physicians. It'll
notify the nurses on certain tasks that need
to be done. And I'm pretty sure you worked on
a project, and I don't know if it was using Epic,
but there were a couple of health medical record
systems, and you were trying to figure out which
had a better predictor for sepsis based on just
certain indicators or certain points of data.
And I can't remember the study correctly if it
was... if you had a better prediction with fewer
points of data because it confounded the results,
or if you had a better outcome with more points
of data because it gave you more information.
And if that was kind of, I'm trying to remember
which was the better predictor for sepsis. Yeah,
so this is a collaboration with Stat News with
Casey Ross. And what we were looking at is how
do you even evaluate these predictors? So normally
we have these sepsis models. There's many of
them. You get told like, here's an AUC, here's
a number, here's a sensitivity specificity of
your model. And we wanted to test and show in
that work is like, does that number mean the
same thing next year and the year after that?
When you build something on the output of the
health system, like you would, it's not a causal
model. What I mean by that is like, you do stuff
because, you know, there's guidelines and you
get told like, oh, for this, give them that.
You have a workflow. that workflow generates
data. And you learn from the output to that workflow,
the kind of data that's generated to predict
who has sepsis. But if you change your workflow
next year, the model could stop working because
now you have totally different data that's linked
to that kind of current outcome. Because you
don't have like this like defense logical understanding.
You're kind of going after the correlates that
are built through the healthcare system. What
we showed if you replicate something that's close
to the epic sepsis model, goes to the model size,
and you built it in like 2012. Then you ran it.
It goes from an AUC of, like, 90 or something
to 50 across, like, a 10 -year period. And we
did this with using the MIMIC data set. And it
was, like, in the beginning, like, oh, Beth Israel
bought another health center. And they expanded.
And they added this. And this is this change.
And this particular order set change. And then
you kind of, like, you just kind of see the numbers
kind of slowly veer down. And it kind of shows
an open challenge in this kind of, like, building
models off the EHR. where unless you have like,
and everything that's not changing, right? Like
meaning the way we collect, the way we collect
the data is self -informative. And if we change
the way we collect the data, then the model might
not mean the same thing anymore. So it always
has to be retraining and like finding a new thing
and consistently validating all this kind of
stuff. So I thought consistency essentially is
where, so when you have so many things that change
or variables might change, the model probably
gets worse at predicting. Yeah, because. Am I
right? is the 2012 pattern, not the 2022 pattern
or the 2025. Right, so completely different from
10 years, yeah. Which must be tough, like from
your standpoint, that must be a tough thing to
try to correct when you're trying to, you know,
trying to create the most efficient and accurate
kind of algorithm. I mean, you can try to find
the stuff that's stable or that's not changing
and then model only those that will be more steady.
But then you could lose out on the stuff that's
changing but useful. And so it just puts you
into a perspective. We kind of have the current
mindset of you build an algorithm once, it's
done, put a bow on it, ship it out, you're done.
For some classes of problems, like imaging, that
can actually work because an X -ray is an X -ray
is an X -ray. It's not changing all that much.
It's not like we're inventing new ribs. There's
not a lot of change that's happening there. The
EHR, stuff that's like the output of human workflows,
is not like that. And so it requires a different
kind of framework where you're like building
stuff all the time. And there's a model of like,
you know, we have seasonal flu vaccines and it's
a little bit more like that because you don't
have one flu vaccine. You solve all the flu.
It's changing. And so is. So getting back to
the kind of the. personalization of cancer plans.
There was a recent article in Frontiers Oncology
in 2023, and it discussed the use of AI to actually
help develop cancer -fighting drugs by identifying
tumor biomarkers. I don't know if you're familiar
with this study or this type of emerging technology,
and do you think this is something that could
be actually incorporated or used in conjunction
with, you know, I guess this would kind of be
past what me or I would do because that's...
predictive ability, but for cancers, do you see
AI helping to be able to treat tumors just by
detecting biomarkers? Yeah. I mean, this isn't
my area, but there's like a huge amount of excitable
topic in that space. The drug discovery pipeline
is long and complicated. There's a lot of steps
within it. And any individual, one of those links,
you can imagine ways for AI to help. Like if
you have a target you're trying to hit. finding
the molecule that will help you hit that target
is more efficient than ever because you can build
models, pick the mapping. There's many things
like from the range of like better ways to do
docking, how do these two things actually bind
together or not? Given one thing, how will it
fold in the first place? Like there's every part
of that kind of spectrum from like target identification,
which looks more like SysBioE down to given a
target, what do you do about it? It's getting
more efficient. Now the fundamental challenges
of like... I found a target that maybe works
on a mouse, maybe doesn't work on a person. Remains
the same, but I'm sure people are working on
that too. So it's not like we've solved drugs.
Drugs are hard for the fundamental reason that
we're not doing random trials on people. I mean,
if you find something like, we're getting better
at solving in virtual bottle systems faster,
which is amazing. We need importance, and we've
got people working on it. And I have lots of
friends working in different areas of this problem
that I think are going to be super impactful.
But the... The drug world is like a very complex
one. And like, let's say we get better at solving
this petri dish and this one and this model system
and whatever. It's good because then we can cross
the bridges faster. But the fundamental challenge
of like drugs in Ontario remains true. But this
is not, this is the, I'm a, I'm a, passenger
is the wrong word. I'm someone that's like ancillary
to the field. I do some work in the space, some
pharmas. And I think it's a fascinating space
in structure. I think it's a super exciting time
to be in it. But it's not my main bread and butter.
No, yeah. For me, I was just more curious about
as far as the applications of AI. AI, for me,
it can be pretty scary for some people. There's
some people that predict it's going to be the
end of humanity. What I will say is that if you
look at the past century, with regard to healthcare
anyway, there have been three public health measures
that have really improved just overall lifespan
and mortality. There was water sanitization,
vaccine development, and then the invention of
antibiotics. Do you think that AI might be instrumental
and maybe even the next fourth, you know, kind
of could be the fourth kind of public health
measure when it's implemented correctly to help
save lives and just kind of preserve lives in
general? I think it's not guaranteed, but it
could be if you play our part right. I think
there's things like both on the innovation level
and the systems level to make that possible.
i mean ai is and like like a vaccine is a particular
thing uh a antibiotic is a particular thing and
it will work for a particular set of things ai
is a broader empowering technology and so like
it should help us shorten the gap of finding
all the next steps and the right interventions
from decades between to like much much faster
so she gave us like a thousand moonshot ideas
rather than just one and like that qualitative
change in speed is what we're looking for, but
it implies the right investments. And we're not
fully there. How much investment is there really
in the health AI space at scale? It's not that
big. There's a lot of non -profits that help
support AI bio. There's the Ark Institute. There's
many things of this kind that have very big,
hundreds of GPUs coming together to better understand
the genome. similar kind of very big, expensive,
bold plays in the more clinical delivery space
are hard to find. And really, we're still in
a space that's at its infancy that has relatively
little investment, and there's not gigantic Google
-level financial players out of it. There's a
lot of investment in the biotech sector, partially
because we know farmers can do well, there's
a proven model, the tracks is there. I think
in the health AI space, we're still in a formative
stage to build the first major success stories
that can improve the case and catalyze that investment.
And really right now, we're still in early days
of this field. We have it in a society as well
to be small. I mean, look at the mammogram case.
The biggest models that are academic and public
are like 100 ,000. There's 40 million a year.
Do we have big national efforts to put these
things together? No. Hospitals barely share.
The compute that's available, I mean, pennies.
We're playing small. We need to play bigger.
There's plenty of room for growth then. As far
as AI's role in medical professions, I'm a nurse.
I work in a hospital. I've been doing it for
over 20 years now. AI is slowly being introduced
into certain aspects. I was wondering, I had
a few questions regarding the possibility of
AI's role just in future medical professions
and just possibly your opinion. So telehealth
is used widely now, especially when COVID started,
it really became more popular. And it's usually,
it's pretty well accepted just for preliminary
diagnosis of non -life -threatening conditions.
It still requires appointments. Appointments
cost similar to, you know, in -clinic visits.
Do you say that AI could someday just be as effective
as a telehealth visit by, and this is again for
non -life -threatening, you're calling because
you have a cough and sneezing or you have a rash.
Or you could theoretically be talking to artificial
intelligence, show them images, give them your
symptoms, and they could help provide a diagnosis?
Yeah, I think so. I mean, I'm sure plenty of
people do this already. Now, does it work or
not? Is it actually effective? It's TBD. But
people already do this kind of thing with the
general purpose VLMs you have from the field
of chiropractic PT now. I think to... catalyze
the space and make it actually effective you
need to have like the same way that like you
know if you get some very bad advice that's harmful
from a clinician there's like you know medical
legal liability and there's like a lot of stuff
there's a whole system around here to kind of
like how do you do qc as a society of the quality
of medical care my preference isn't gonna get
better but there is a system uh and i think uh
it is a matter of time until the systems become
mature enough that people are willing to kind
of put the money where their mouth is to kind
of take a liability for errors and, uh, monetize
it accordingly and do things to kind of make
that kind of service available. I think the nature
of the profession, like any profession will change.
And I can tell like, you know, I code, I've experienced
this in my training. What it's like for me to
code now is so different than it was for me to
code five years ago. It's like night and day.
And like, And it would be hard. I mean, I could
do it. I could code the way I used to, but I
feel like a barbarian, you know, it's like going
back to less efficient, right? Yeah. Yeah. I
mean, like it may even make you think differently
because like I'm more used to the AI models now.
And so I expected to do particular things and
I kind of like, and it means that my net throughput
can go much bigger. And like my, what I think
my leverage is as an individual person building
is much bigger. And I think it is an open challenge
of how do we do this in care? Because, like,
the general situation you have in care is that
we're always understaffed. Like, you tell me
you want an appointment with anything, it takes
forever. Like, I'm trying to get an appointment
right now. It's getting tough, you know? And
there is an opportunity, if you do well enough,
to make the system serve people better. I think
it's an exciting opportunity. But, again, it's
definitely going to involve a lot of change.
There's no way to avoid it. Yeah, I mean, kind
of as a follow -up to that, like, I was wondering,
like, Just how much, like, if you get to that
point where it's, you know, relatively effective
and efficient, it could definitely reduce, like,
wait times in emergency departments. People wouldn't
be going to the emergency department because
they felt like they were sick or had the flu
or something or it wasn't, you know, overly pressing.
The other thing, as you kind of brought up, like,
you know, the insurance industry has always got
their fingers in everything. And it's like, as
far as the costs. saving measure like at what
point would it become more or less cost effective
to do this i think that all depends on you know
like i said who the regulatory issues all those
things um in hospitals in general oh sorry go
right ahead yeah i was just saying that like
uh can we make it technically possible to live
in this kind of care for sure will we fail to
deploy because of some weird nuance of our incentives
and our insurance system Maybe. To me, those
are different questions because the insurer does
not necessarily have a great incentive to make
care all the much cheaper because I usually think
to help them increase the revenue is better for
them. And they have the profit cap stuff. It's
a quite complicated system. I think it's what
I think. I feel like it's ours to lose. Can we
make this possible? Yes. Is there the feasibility
of better care? Across almost every sector, the
answer is yes. It's like... Can we play the card
right enough to make it actually feasible and
cost effective within our system? Can we change
the system to remove the necessary roadblocks?
That to me is like the societal challenge because
we totally mess it up. Oh, I'm sure we would.
And there's a lot of people who wouldn't want
that to happen in the first place because there's
a lot of money to be lost with regard to that.
So in hospitals, physicians who typically, you
know, once they reach a certain stage, once they're
past their residency, essentially, they start
to make pretty decent money. But physicians,
they diagnose, they prescribe treatments, they
perform surgeries. They often spend little time
with their patients in general. You might see
a physician five to 10 minutes a day if you're
actually in the hospital. As a cost -saving measure,
could hospitals at some point use AI in some
of these aspects in care as far as the diagnosing
and prescribing treatments or even surgeries?
I have a question about surgeries and robotic
surgery as well, but I'll get to that in a second.
Just as far as AI, could they replace physicians
in a lot of these even smaller hospitals that
don't have the staff or don't feel like they
can afford the physicians with regard to prescribing
treatments or even just doing the assessment?
This will play out differently across different
types of places. I think of the solution when
you're like two hours away from the next hospital.
will necessarily look different than it is if
you're in the city and the hospital is like a
15 minutes bus ride away. So that'll be a little
bit diverse. I think we're already seeing some
version of this. It's generally true people are
burnt out. You see too many patients for too
little time. You don't have time to think about
them as much as you want to. It's very hard.
And the AI products that are getting the most
traction, stuff like Average that's doing passive
summarization, they'll give you back more time.
And so I think like... Will there be more and
more of this where you can kind of delegate more
and more responsibilities to more and more systems
so you can kind of have control over what you
think about and spend your intellectual power
where it's most needed? I think that's true.
And in places where you really can't get anyone,
is that basic safety better than Googling? I
would hope so. So I do think there's room to
grow there, but I think the implementation model...
will likely look very different in very rural
places like where I'm at right now compared to
SF. And I guess that could be really advantageous.
I know that a lot of community hospitals, they
will use telehealth for specialists. So in that
same vein, if you're an hour or two hours away
from a specialist and you need to talk to them
right away, I think that's somewhere where AI
might be able to come in and... at least aid
the physicians that are there in helping to make
a diagnosis or provide the correct treatment.
The question I had with robotic surgery, so there
was an article in January of 2024 in the Journal
of Robotic Surgery, and it highlighted the potential
of AI use in robotic surgery. I know that some
places are using a little bit of AI right now,
but do you think that there's a future where
AI -guided robotic surgery could actually become
a primary surgical tool in hospitals? With regard
to providing just more accurate surgery, I know
that robots have more dexterity than humans,
smaller areas than humans. Do you see that as
something that's a possibility in the future
or that might happen? I mean, I know the surgical
robot companies are making a lot of progress.
I mean, I think there's different classes of
improvements. One of them is, is it so good that
it's fully autonomous? Or is it so good that
it could make, because a bunch of people, like,
okay, what makes a really great surgeon? They
do a lot of surgeries. And some people are good
at very specific neuro type of surgeries. And
if you get it at a general place, it will not
be very good because they don't do them that
often. So you have much higher rates of complication.
Can we upscale people at scale? There's a different
version of this in radiology. There's a lot of
general radiologists that read everything. Okay,
well, how good are they then in recognizing the
rarest type of cancer that occurs in one in 15
,000 patients? Less compared to the one that's
like specialized in just that. And I think there's
a class of opportunities that to me is more like
short -term feasible of how do you upskill a
broad swath of people to operate at a higher
quality bar, which is a different bar of performance
than it is to say be fully autonomous. Like if
you have a Tesla or one of these like self -driving
car companies or whatever, adaptive cruise control
is pretty useful. It's pretty great. It changes
how I drive on my like, you know, eight hour
journeys and whatever. and like the tesla autopilot
version thing uh is like would i actually not
pay attention at all no but it like makes life
better and i think there's a lot of version of
this across domains including radiology and surgery
uh but it's you know it's a dangerous thing to
predict any particular time of any of these things
you know science takes what it takes take the
time that it takes yeah not trying to put you
in an awkward position uh answering any of these
i just want to pick your brain I think like the
robotics world generally has a harder time because
you have less demonstration data. In radiology,
we see more of the full world. We don't see all
of it, but we see more of the context literally.
Not everywhere records all the information and
all the kind of dexterous motion. And so it's
harder to, like you don't have as much data for
these robots to build on. And it tends to be
very conservative by design, right? Because it
only takes the robot messing up a couple of times
to get a lot of PR that can kind of shut down
the company. And so they're necessarily conservative.
And I think... But I think there's a lot of cool
work happening in the control world. There's
a lot of really cool companies being built around
general robotics. And if they do well, I think
it's quite possible you get richer capabilities
in the medical side. So kind of getting back
to the question I had prior to this, just with
physicians as well as what they do. And I kind
of brought up the fact they tend not to spend
a lot of time with the patient themselves. They
might be five to 10 minutes a day. They have
a lot of patients to see. Conversely, nurses,
which is the role that I do. The orders the physicians
write, the treatments, we're the ones who actually
perform those. We provide monitoring of the patients.
We're often there for emotional support. Nursing
itself is just kind of hallmarked by compassion
and empathy. Do you see a day where AI, say,
becomes autonomous and you have AI performing
the tasks of humans and you have AI -driven robots
essentially performing care? Do you ever see
a day where... they could reach the kind of emotional,
you know, those characteristics of empathy or
sympathy. And beyond that, do you think people
would actually be receptive to being taken care
of by, you know, fully autonomous robot nurse?
I think, like, couldn't LLM write a better eye
chart message than a person? Probably, because
people are busy. And the LLM could write you
a big paragraph. explaining all the details,
and, like, there's very compassionate. And when
people are rushed, and, like, when you're, I
don't know about you, but, like, me at my most
stressed is not my most elaborate to well -thought
-out email that shows a lot of compassion and
connection. When it comes to the point of seeing
the person in the bedside, I have a hard time
imagining that something kind of hits, like,
this kind of weird pseudo -synthetic thing. Like,
I don't know, like, a metal box that's trying
to look human would be more creepy than just
a metal box. And there's only so much that can
comfort you. But over phone, I think there's
a lot of... It is already true. There's already
been studies showing this, where people rate
LLM -generated reports back as more compassionate
and more patient than the ones written... There's
been anti -evidence here and there of that, which
is not surprising. If you have five minutes versus
infinite GPU flops, you can write poetry. It's
not that hard. But the actual in -person thing,
like holding someone's hand, I think there's
a deeply human thing there. Which is good to
know. With regard to that, so automation, I think
a lot of people are kind of fearful of automation.
It's something that is becoming just more and
more prevalent. I think AI is getting better
at kind of helping automation become more like
humans. Do you ever think that... there'll be
a day where, you know, most of the medical profession
is completely just run by automation. Do you
think that, I know you just said you can never
really take the human component out, but do you
think that humans will kind of be reduced in
their capacity and what they do or have to do?
Or could they even, you know, that could be beneficial
at some point because you're not doing the tasks
that are taking up time away from more direct
patient care or kind of providing better patient
care. I, I, The way I think of it is that we
don't have nearly enough humans to care for people
at the scale that we want. Like, basically, like,
in supply and demand, the problem is we don't
have nearly enough supply. The demand is way,
way higher. And so we end up with a product that
we give to patients that is much worse than it
could be, both in terms of, like, the user experience
of it and, like, sitting in a waiting room forever
and the apartment never opens the thing it is.
Like, it's not the best commercial experience,
let's just say, on average. I think AI ideally
will give us a lot more leverage to have the
existing supply, which is hard, more reasonably
meets demand and have people that are very skilled
be able to serve way more people and play where
they have highest leverage. Now, it's true in
theory that you could have such good tooling
that maybe you need less than existing supply.
I think we're just so far from existing supply.
It's not like you get an appointment for a specialist
same day. It's not in that world. We're just
not. People drive hours and you turn around time
for a callback, maybe several weeks. We're in
a very slow world. I think there's not enough
of almost every specialized profession. There's
always, how do you get more nurses in the profession?
How do you get more radiologists? How do you
get this? I think for the actual demand of people
wanting to take on these professions that are
hard and very normal, we can help them be able
to be more effective. It's a positive view because
how do you empower people to do more and better?
But like, could we mess up and make the experience
much worse and have it even more grinding? And
for sure, it's also possible. And, you know,
build it a particular way. It's going to be a
spicy time. So recently, there's a congressional
representative named David Schweikert. He introduced
a bill called H .R. 238. And this proposed that
AI could actually prescribe medication under
FDA and state approval. argument towards this
is that AI could actually help prevent medication
errors, it improves efficiency, and it can help
personalize treatment. Do you think that's a
technology that's actually currently available?
And do you think it would be capable? Do you
think that performing that task as far as prescribing
medication based on, say, a set of lab data or
an assessment or some other information that's
fed into a model? I mean, I think if you take
the word AI out of this, it would make sense.
Let's say you have a test you have a bacterial
infection give me a box of antibiotics like why
do you have to sign an extra form you know like
uh these things of like it's roads people do
it over and over again you see a thing you do
the thing it's not like a fancy ai model you
have this kind of infection here's what i normally
give you for it just get out of my way uh makes
sense you know like uh and So, like, I think
there's probably low -hanging fruit of that to
streamline workflows in a way that feels obvious
and low -risk. Now there's more complicated scenarios
of, like, you know, you're an oncologist and
you're figuring out which kind of chemo regimen
to do and you have all these toxicities and,
like, is palica going up? Is it because it's
responding or not responding? These kind of very
subtle things. It's quite possible that we'll
kind of create more information. and make an
easier decision than was possible before, something
that we're working on, I think is going to be
very exciting. But in the near -term future to
mid -term, I don't see that kind of thing being
fully automated, unless it's a place where the
standard is so bad that that automation is still
an improvement. That's the thing about healthcare.
It's so diverse, and you have really high resource
settings and such low resource settings. So the
way I think about automation, fundamentally it
depends. If you don't have access to an oncologist,
then yes, this thing that's maybe not as good
as one is better than none. But if you have one,
then you should work to help make them better.
And you shouldn't deliver worse care. So how
do you help an individual person? And especially
in the global health scenario, when you have
one for every X thousand cancer patients, then
the way to think about the solution has to change.
So I think public perception, too, is something
that... there's ethical considerations and public
perception when it comes to artificial intelligence
in healthcare. There was a recent JAMA article.
It was actually a commentary and there was a
survey and they surveyed over 2000 people. So
not a huge sample size, but there was a lot of
mixed trust in artificial intelligence role in
healthcare. The scores were, you could score
zero to 12 with zero being like no confidence
and 12 being confident in artificial intelligence
role in healthcare. And the mean score was about
5 .8. So people are kind of split as far as confidence
level for artificial intelligence to kind of
be incorporated in healthcare. Do you think there
are measures that can be taken to just improve
public trust or acceptance of AI? Do you think
that's something that is going to take a long
time to do? One, I think it's a funny scale.
I've never heard of a zero to 12. Zero to 12.
Yeah. I don't know. But I think I don't take
those numbers too seriously. I think people in
general. are very bad at estimating how they
will feel about technology that's not there.
So you ask people in the abstract, would you
use a language model for this before ChatGPT
came out? They were terribly predictive of how
they use it today. Fundamentally, what builds
trust? Efficacy. If you ask me, I just think
if the tools really work, building trust is just
a matter of actually delivering results that
are actually very good. And like fundamentally
user, like the experience of using the tool and
how well it works is what drives it. And if someone
could tell you, I felt this in my own work. People
tell me like, oh, I don't know if we can trust
what it means to have like an A algorithm do
cancer risk. I want an explanation, blah, blah,
blah. Hospital validation 10 later, no one asked
me that question anymore. Because it works. And
like, I think it's many things are the same way.
Like many things are scary in the abstract, but
in practice, if they're useful and you like them.
then you're useful and you like them. If they're
not useful, nothing will make you want to use
it. I think just the utility, if it's not useful,
people are just going to stop using it anyway.
It'll probably die off, I would assume. Yeah,
I think it's hard to have precise conversations
on this when the tool is abstract. Kind of following
up with the ethical considerations of it. The
studies you've done, like I said, you had one
that had over 150 ,000 scans. And it's kind of
another crucial topic when talking about AI and
healthcare. It's just patient privacy is huge.
It's something that we take very seriously here
in the United States. When you're using your
AI models, what kind of safeguards are put into
place to make sure that the information that
you're putting into your algorithm to kind of
come up with just research data? How is that
safeguarded for people trying to access that
information or hack into it or just the security
leaks in general? Yeah. And these settings, I
mean, like there is like all security is like
an onion. There's like many layers of security
that kind of like roll onto each other. So the
kind of like the first layer is like we, it's
not like we have like. servers that are sitting
in an office that you just kind of walk into
and plug in a flash drive. This is a secure data
center that's managed essentially by the hospital.
I couldn't walk in there myself even though I
bought the server. So there is layers of physical
and firewall constraints that's like we, even
though the data we work on is we usually anonymize
it before actually putting it into the research
workflow. Even though it is anonymized, we protect
it as if it wasn't anonymized. And so it lives
in a place of PHI compliance, all the kind of
standard IT security constraints accordingly.
In order to touch any of it, first you have to
have an IRB to say the risk of privacy leakage
is low enough for the kind of game that you have.
There's like minimal risk stuff, so you can actually
do this kind of large scale stuff. So you have
to get like ethical approval of the actual concept
in itself. You have to go through regular trainings,
both on the IT side and ethical training side.
There's like a bunch of layers of stuff. And
even if someone hacked into UCSF and got access
to our servers, the data is not analyzed. Which
is good. I know that here in the U .S., we have
a lot of regulatory oversight. Like you said,
you have all these layers. Do you know if other
countries or healthcare systems are kind of protecting
privacy for patients with regard to not just
artificial intelligence, but kind of data gathering
in general? Whereas the United States, are we
a little more strict? I'm not even sure if you
know. I'm not. I'm not sure. On the privacy side,
I usually defer to the privacy offers at the
hospital system to follow their recommendations
and rules. I do AI stuff. I have a lot of expertise
within that world and how the use case is. The
security world is its own infinite rabbit hole
of stuff. We have people that are good at it,
so they can tell me what to do and we do it.
I think in terms of the capability to build AI
systems, in health, where you have these privacy
constraints, there are trade -offs. In Europe,
you have these national health systems, and in
theory, you can actually build things across
the country and get a scale. It's hard to get
in the U .S. Because in the U .S., we're very
fragmented. You can build something for all of
Denmark, for all of the U .K., using the NHS
data. And that's cool. The downside in that system,
I'm finding through current collaborations, is
that... the investments in research is often
quite a bit smaller. And so like, you know, I
have a collaborator, they're using the resources
of Denmark, the GPUs they have access to is really
quite limited and like quite a bit less than
just my lab. And so like, you know, would you
need to build these tools as well? You need data
and compute to live in the same place. And it's
hard to find that almost anywhere. And this is
part of the kind of general conversation of like,
we ender invest in health AI. So beyond the ethics
component, too, you kind of just touched on this
briefly, but there's always kind of that fear
of litigation and liability. And it's almost
a question of who would be liable. Like, say
that, you know, you incorporate AI into the health
care system and it's used and somebody gets injured.
Who is actually and this is just kind of a question.
I don't even know if there's an answer, a correct
answer for this. Like who would be liable for
it? Would it be? you know, the provider or the
technology corporation or even just the programmer
themselves who created the AI? And do you feel
that the fear of liability is actually kind of
a barrier to incorporating AI into more healthcare
systems? I think it's one of those things like
it totally depends on the workflow. Let's say,
like right now, we already use AI in radiology.
We use it for dictation, right? You speak into
the microphone, it generates the report. Now,
if you sign off on a report and the thing misread
your words and like it put a not where there
shouldn't be a not, right? And you sign the report,
that's on you. Dictation software is the same
way if you like, if you slip and you like hit
the wrong part of the keyboard, that's on you.
You sign the thing, you make sure it's there.
And for many things where like you're signing
off, it's clear what the responsibility is. Now,
let's say you have an autonomous system that's
taking the full decision and it is signing the
report. then the maker of the system needs to
be liable, right? Because like, and it should
be priced accordingly and it should be able to
put the money in their mouth as it had their
own, you know, their own medical liability insurance,
et cetera. So I think like for many cases, it's
kind of straightforward. Like if, I think the
ambiguous circumstances are harder to come up
with. I mean, it's like, you know, let's say
someone's doing surgery and the scalpel breaks.
then who's liable in that circumstance? I imagine
it depends. Like, did it break within warranty?
You know, or did it break after, like, 15 years
of not being maintained and cleaned? Right? And,
like, I think there was, like, things like what
happens if it breaks in the kind of circumstances,
but I think a lot of settings that we're imagining
now where it is a decision aid, like, you know,
is Google liable if it doesn't give you the best
search results for the question that you have?
No. Should it be? Probably not. Like, you don't
have to use it. But if you're saying that's a
diagnosis, whatever Google's research is, that's
a product they're selling you. They should be
liable for that. So with regard to AI and machine
learning and future research, I don't know if
you can share. Are you concentrating everything
with your lab right now, just working on Mirai?
Or do you have other projects on the pipeline?
Are there things you can or can't talk about
with regard to future research? We're broadly
focusing on going bigger and broader. So both
Mira and Sybil are what I think of as highly
specialized point solutions. We're trying to
solve one particular problem, problems I deeply
care about and ones I want to get better at solving.
And to me, the future of the space is solving
more and more problems at once and having a broader
set of capabilities. One, because it makes you
more nimble. being able to address many potential
issues within the workflow and kind of like being
creative and how many types of things can you
help in at the same time. But two, because it
also amortizes the cost of translation and like
regulatory. So like I imagine that the future
of this commercially is going to be like you
go for filings for like 400 conditions at the
same time. Because then it becomes more cost
effective than doing them one at a time. And
so like technically, like to give a more concrete
answer. We're jointly modeling things across
many cancers, many imaging modalities, trying
to get a broader set of capabilities, including
the ones we've already been solving. Because
one, we think it'll make us better at solving
every individual one of these things, because
in every one of the things, by itself, you're
too data constrained. Together, you have more
data to play with, so there's more constraints
on the model for how to learn better. And when
it comes time to translate it out, translating
one big model and going through one big filing
is easier than going through a dozen small models.
The technology is fantastic because I know that
I think you had done another research paper that
you had done or worked on with regard to pancreatic
cancer as well. And there are certain cancers,
pancreatic cancer being one of them, as well
as stomach cancer, that a lot of times you don't
find malignancies until it's far too late. Pancreatic
cancer especially because it's got such a high
mortality rate. Even if you catch it early, it's
still just super deadly. But like stomach cancers
as well, because people don't present with symptoms
until you have that malignancy. And I don't know
if the same, you know, with the research you've
done on Sybil and with Mirai, if that same type
of technology can be used as well to help detect
these other cancers. Yeah, so there's the same
type. We're working on two things for that problem.
It's all the same flavor. You know, like once
you have imaging, you know so much. You get to
see. You get to atomically see what's going on.
You should be able to get better risk models.
And so there's one flavor of work that we're
working on. Like if you have a lot of incidental
imaging, you got an abdomen CT for an unrelated
reason, kind of figure out your high risk now
so that like for those people who happen to have
that imaging, we can do something useful for
them. The other bucket, when you don't have imaging,
how do I give you a better baseline prediction?
And that's all this like EHR modeling stuff,
which we already talked about before, has to
always be evolving because the EHR is like this
dynamic beast. And the future... I think it's
both. We should model everything. I think there's
been a lot of efforts that are more comp bio
plays where you say, like, I'm going to invent
a new panel, see a new blood test, maybe I'll
look for circular tumor DNA or something else.
And those are great efforts, and I think they
can be complementary. But the type of thing I'm
most interested in is, like, what can we do if
it's already widely collected? Because there
you have scale. Then it's an AI play. if you're
doing a new blood test and you have a new kind
of panel, you're going to read these four things
out or these 10 things out. You're going to have
maybe across as many years, 100 samples or 300
samples. It's too small. I mean, it can be really
valuable and that's great, but it's too small
for my skill set and what the top stuff in my
lab does to make the big difference. So we're
very much focused like where can we operate at
scale where the type of technologies that we're
really good at building can make the most difference.
So it leads you to like imaging and EHR stuff
that we've been collecting for decades. less
than the kind of like new novel experimental
platform that is by its nature new and small
and limited adoption. There is room to make AI
innovations there, but it's like a different
flavor of work and it's quite a bit harder. How
hard has it been for you when you're kind of
procuring data just to get, because you needed
so much data for these models? have you had difficulty
finding uh just facilities or just medical rec
just obtaining the medical records and do you
see that as kind of being another roadblock in
the future when you're trying to do these other
predictive models is just actually getting access
to the data that you need it is a major challenge
i mean i think that uh we're not bad at it i
spent a lot of time on it i don't know how uh
you know how hard is as i say it's hard enough
to spend a lot of my time on it i think about
it a lot i And it's so hard, I don't know how
to compare it to anything else. So it's that
level of hard. It's like tier one hard. And it's
not just about finding other centers. It's like
the kind of stuff that we're building and the
level of which we're trying to pull data is like,
I want everything in the health system that's
ever happened. I want everything. That's hard.
There's not a button. There's not people have
been asking for that for many years. And it's
very easy to export everything in an easily available
format. It takes a lot of time to pull it. It's
a basic, even internally. The basic piping to
download CTs at UCSF is not that fast. It takes
a lot of time. And we're spending a lot of time
kind of like, okay, well, what's the best way
to use the system as it is to kind of get the
volume that we need? It's challenging internally,
it's challenging externally, other places. And
fundamentally, because it's so much, it's so
laborious to get the system working, it's based
on relationships. Like not some random stranger
who like does not care about the project or like
what we're trying to achieve. It's going to go
through all the work to curate this data set
for us. You have to find partners that believe
in the mission, that you're aligned with. You've
got to build something together. They're willing
to go through the work with you to make it happen.
And it's a burden of love. It's a burden of love.
Another question. A lot of people don't realize
that not everybody has digital records as well.
So we still get a lot of outside hospitals that
will send us images that are captured on DVDs
or CDs. And I don't know how it is because you
had collaborations just with other parts of the
world. And I'm not sure as far as like digital
radiographs, like a lot of places can't upload
them pretty quickly. You were just saying it
takes a lot of time just to kind of actually
get the information and then like download it
into your system. Have you had trouble just getting
just the type of media that the imaging comes
on? Has that been a problem too, as far as integrating
that? Not even integrating, but just getting
the data and being able to upload that? it just
happens to think about the scale like moving
500 terabytes of images one place to another
it's just not it's not that easy because like
fundamentally you pull from the clinical system
well clinical systems they do clinical stuff
all day so it has to slow down so you don't mess
up clinical operations and like moving that much
thing when you don't have is like you know fast
links between places just like it's just slow
and like and if people haven't done that kind
of thing before beating the pipes is challenging
not at the patient level when someone wants to
like get a risk score, or they're curious about
the research, they want to try a model. Fundamentally,
as a patient, you don't have easy query access.
People get CDs. People sometimes mail me CDs.
It's too challenging as a patient to get access
to your data in a convenient format. At the institutional
level, that's not really the thing. PAC systems
have APIs, but depending on the hospital that
you're in, You don't have the bulk export enabled
in your PACS license. They're going to squeeze
you for more money. Or maybe you have it, but
you don't have an intermediate place to store
500 terabytes of storage. So it's not trivial.
It's not trivial because of the scale. With regard
to funding, has it been difficult to get funding
just as far as governmental, publicly, privately,
however you procure funding for your research?
And do you think that there's a way that you
could, I don't know, hopefully this will help
this podcast, depending on who takes a listen
to it, kind of spread the word about what you're
doing and how beneficial it can be and probably
will be. Have you found it hard to actually get
funding for what you're doing? Yeah, I think
that's one of the things. You never have a baseline.
You spend a lot of time on it. You work hard
on it. It is always challenging, and the cycle
time that we have in the system is quite slow.
So just today, I got my notice of award for the
R01 slash R37 that I wrote. Very excited for
that. That's going to hopefully give us quite
a bit of runway. From the time of writing that,
the first time I wrote it was June of 20 -something,
more than a year ago. I think it was, yeah, it
must have been June of 23. They had some comments.
We revised it. The first time we could resubmit,
given the revision, was February of 24. And we
did not receive that funding until today. Now,
I'm super, like, I mean, the fact we have the
NIH, there's nothing quite like it. It's an incredible
resource. But, like, practically the cycle time
of, like, having an idea, having the resource
to pursue that idea, median time is, like, 18
months. That's very slow in the pace of science.
Because by the time that comes over, you have
all these kind of new ideas. And you just can't
be very nimble. And that means it's hard to move
quickly. And practically right now for our work,
I can say for a fact, the rate limiting factor
is not ideas. It's not students building really
great stuff. It's compute. And it's going to
remain that way for the foreseeable future. And
so I think as a lab leader, what is my responsibility?
I'm trying to both create things in the world
of cancer. and develop useful things for patients.
And the best way for me to do that is to empower
my team to do that. And resourcing and getting
computers is a huge part of that. And when anything
that you do has a minimum year plus cycle time
to the normal pathways, it's hard to be nimble.
And I think there's a better system to be had.
So it's not easy. It's not easy for anyone. I
think things have gone relatively well. Relatively.
But compared to one's ambitions, compared to
my ambitions, not enough. Though that's more
dispositional than anything. I think I'm always
trying to think of ways of how can we move faster?
How can we deliver faster? To me, the biggest
risk is in 10 years, care is still the same.
That is failure. And the normal time for translating
anything is super slow. And so we need to find
ways to be faster, more aggressive. And for now,
compute is a limiting factor. And resourcing
is a big part of the game. A few kind of just
AI questions in general. So I know that quantum
computing is something I'm not a super technically
savvy person, but I've read a little bit just
about quantum computing and how it might be a
game changer as far as giving you more compute
power and just kind of speeding everything up.
I know that there... there are chips being processed
right now, but how far in the future do you see
like quantum computing being able to be incorporated
into artificial intelligence? And is that something
that could help speed up the process of a lot
of what you're doing? I'm not sure. I have some
friends that work in that space. And as far as
I understand, I haven't seen any kind of like
AI breakthroughs made conceivably possible by
that. As far as I know, the type of stuff that
quantum is really great for is like shor's algorithm
and stuff to kind of like you know factor primes
and stuff and uh and there's already post quantum
crypto algorithms that are out there so i personally
and i know that like if you're doing quantum
simulations and some type of molecular modeling
it's very useful uh but in the kind of stuff
that i do it's hard it's it's unknown uh maybe
this country comes around and it's great and
it's awesome that'd be cool uh but i don't uh
i don't see a light at the end of the tunnel
of that kind i think uh More and more, we're
getting the NVIDIA GPUs keep getting better and
better. And really, it's not a matter of technologies
not exist. It's having the money to buy it and
then doing stuff with it. This is kind of a ridiculous
question, but are you familiar with the simulation
theory? We're just a part of a simulation. And
if you are familiar with that, do you buy into
the simulation theory? I mean, like, there's
some things I think are, like, I don't even think
about because they're not actionable. You know?
So, like, from a probability standpoint, so if
you believe this is true, like, I mean, it's
plausible. If you create a simulation, sure,
I guess. Why not? Then, like, if you take that
premise and the odds of you being in a simulation
environment, it's not higher because you don't
know the probability density distribution. but
like it would be like believing you're in the
center of the universe to say that like you are
in the primal base reality or whatever but like
functionally I like whatever because if it is
or it's not true it changes nothing in life so
it's like there's many things of that kind like
like another version of this is like if there
is a if there is a god his favorite number is
some real number there is an incontestable infinite
number of those real numbers so no matter which
of them you believe in you're probably wrong
because what are the odds that it's true You
can play these games. There's one of infinite
possibilities. Because infinite possibilities,
it would be hubris to say that you're the one.
You have no reason to believe that your infinite
possibility is the one. But that's like saying
I could be wrong about something. You can always
be wrong about something. Whatever. It's actually
a great point of view. Another question I had,
and this is something that a lot of people theorize,
that eventually when AI becomes fully sentient,
that there's something called the singularity.
that is going to happen i i don't know if you
could like what your thoughts are on the singularity
and you could even tell like our listeners what
that means i this is concepts of like once you
get to a particular level capability then the
ai can prove itself at some exponential rate
and it's going to solve all things it's almost
like a messianic kind of argument of like you
know hockey stick goes up kind of thing i i think
that it assumes something about cost. It basically
assumes that there is no cost improvement, that
cost is not improving, is not increasing as well.
And these things are like unbounded and restricted,
which I think is probably an oversimplification.
Like, for example, like to build new chips, the
cost of a fab costs like 2x more or something
every single time. There's many, like, I'm skeptical
of the concept that there is no fundamental trade
-off that you can actually see given existing
resources. even if you had systems that were
able to like buy new resources it takes time
to build a new power plant and to create a new
data center there's time to like you know lay
more bricks and like manufacture more trips like
there's a bunch of physical time stuff there
and so the notion that you can get exponential
growth you have in fixed constraints and fixed
resources forever right so that's where you get
the singularity thing and now like you solve
all things it like seems to be a bit bold Like,
if you want to invent new drugs, you can put
that shit in mice. Mice take time to grow. You
know, like, I just think, like, the over -signification
there is, like, to me, feels like a stretch.
Will we get systems that will change the nature
of labor? You would hope so. I mean, the internet
did that. But then it sounds much less magical.
So, like, will we have things that will disrupt,
like, you know, disrupt the way that we work
and cause a magical transformation that will
be the biggest thing we remember of this century?
I think so. Does it look like a sci -fi movie?
I mean, what are the odds that the way we imagine
it happens to be what's going to happen? It's
continuous odds, so probably not. Do you ever
have any fears that with AI that it's going to
get to a point where it could become detrimental
to humanity? Do you think that we're, you know,
I would say smart enough? or cautious enough
that we'll take precautions developing ai that
we don't have to worry about it you know realizing
that humans are actually the cause of all terrible
things in this world so it decides to destroy
all humans do you think that uh are you more
optimistic or pessimistic towards the future
uh i'm i think we'll both do harm and good like
we do with basically everything i think we're
going to get more volatile with the good we can
achieve it's going to be more ambitious than
we currently thought And the bad we can achieve
is going to be worse than we can currently think.
And it's going to happen through both. It's going
to be this kind of running balance between the
two, and it's hard to predict what that one will
be. The internet has done a huge amount of good.
It has also done some harm. Social media has
done some good, and it's done some harm. And
I think AI is what I would call a broad empowering
technology that lets you do a lot of both. And
I don't like to say that we would do, to take
a purely optimistic view. would be utopian that
like we've we've exited human conflict that seems
unlikely uh and to take a broadly pessimistic
view would would be counter to the arc of like
psychological progress across the last like 80
years uh but still i view these things like i
very much of the mode of like i have a particular
problem trying to solve there's a classic they're
trying to do better and if we execute poorly
we could do some harm We're going to try very
hard not to. We're going to try to build all
the right systems to do so. We're also going
to be driving towards better care. That's why
you have to run trials, because not every trial
succeeds. That's the kind of name of the game.
On the broader societal level, there's lots of
weird stuff. I have no claim on the wisdom to
predict how this will interact with geopolitics.
All this kind of stuff is out of my lane. So
for anyone who's listening who might be curious
about going into the field, just of programming
or coding or just in AI in general, who has just
kind of a fascination with it, what advice could
you give them with pursuing that as far as where
they should go to school, what kind of material
they should read to get involved or just to learn
more about artificial intelligence and the type
of work that you do? Like anything, you learn
by doing, you learn by building. And I would
say it's a better time to build doubt than ever
before. As an individual person trying to build
something new, you have more leverage now than
ever before. The tools are better, the resources
are better, the compute, the raw hardware is
better than ever before. And so, like, less reading,
more building, you learn by doing. Now, if you
want to do the kind of stuff that I do, then
practically you need access to... collaborators
in a lab that has the right resources to kind
of do that. And so like, you know, reach out
to the faculty interested in working with and
like try to find opportunities. There's opportunities
in industry in addition to academia. But like
fundamentally someone to get into AI, build stuff.
It is easier than ever. And because it's easier
than ever to do something small, it means you
can be more ambitious than ever to do something
bigger. Like what two people working together
can build today is like, you know, nine and eight
came out five years ago. And it's going to continue
to feel that way for the foreseeable future.
And it's really cool. It's cool to see. It's
cool to see. Yeah, I'm fascinated by it. Do you
have just a large language model that you prefer
to use? Do you prefer to use ChatGPT or Perplexity
or Grok? Or is there one that people might not
know about that is actually more useful than
any of those? It depends on what you're doing.
I mean, I use LMS a lot in like prototype encoding
stuff. Not so much for research. I mean, like
not so much research for cool stuff when I'm
like trying to figure out new systems, data infer
thing. Both ChatGPT and Cloud are pretty good
at that kind of thing. Practically, as an IDE,
I use Cursor. Cursor's pretty great. I only use
it for stuff where I can use it given security
compliance issues or whatever. But that's...
I think as a whole, this is a different kind
of coding than was possible before. And if you're
doing something where you don't... For example,
I don't build web apps anymore. I used to when
I was an undergrad a long time ago. I don't really
do it anymore for a variety of reasons. i needed
one for small prototype to test out a particular
kind of workflow and like five years ago that
have taken so long you have to like learn react
and learn how to do npm packagement like all
this like little stuff but like you know just
like uh it is much easier to kind of like bootstrap
to minimum capability very quickly especially
when you don't know the library so it's cool
i encourage more people to get in the game yeah
i've never like i i wouldn't know how to code.
Is there, so somebody like me who's never coded
at all in their life, could they use one of these
LLMs to actually, would it help in teaching them
or would it just, are you actually able to feed
in kind of what you're looking for? I'm just
kind of curious as the process of coding, I understand
the basis for it, but how much could I, as someone
who has no experience with it, use one of these
LLMs to actually develop, say, a webpage or,
I don't know, I think you said it was for app
development. For things that are well covered
by the internet, there's a billion tutorials
on how to build a web page out there. For that
kind of thing, it really can. You can literally
copy and paste the error message back into the
chat and have it tell it to fix it over and over
again so it works. For things that are easy.
A web page is so well covered by the internet
that it's pretty straightforward. But it is not
a panacea. It's not being good at coding. What
it means, if you're good, you just have so much
more leverage. You can go so much further. But
if you don't know what you're doing, then at
some point, it's going to stop fixing it. And
then you'll be stuck. And even guiding it the
right thing, understanding what the error is.
As times get more competitive, the only defense
is always to be good. That's always what it is.
When times are good, you want to be excellent.
When times are bad, you want to be excellent.
Because resources are narrow or good. That's
the only path that's out. And so there's no kind
of panacea, but like... it does mean minimal
resourcing to build something is smaller than
ever before. So you can be more ambitious. I
think that's very cool. I have to jump the next
two minutes, unfortunately. Oh, sorry, man. Yeah,
I will let you go. I just want to let you know,
thank you so much for coming on. I want to let
our listeners know too. Looking at Adam's research,
it's excellent. I appreciate you and what you
do. I've always thought people who especially
have kind of... Just your drive, your ambition,
especially with, you know, your technical background
going to MIT and now you see Berkeley. The fact
that you use that to try and help other people
is fantastic. I think it's a noble, noble endeavor.
There's a lot of people with the same kind of
background who might, you know, try to be more
profit seeking. But what you're doing, I find
is excellent. It's so helpful. And I have a lot
of faith in your efforts that it's going to continue
just to grow and just be completely successful.
So thank you very, very much for coming on. One
last question. Have you ever seen the television
show Silicon Valley? I've heard of it. I have
not seen it. It's a great show. It's a comedy.
It's on HBO. It aired from, I don't know, it
goes like 2012 to 2014. I don't know. It's about
a decade ago. But it involves, it's all about
living in Silicon Valley. And just, this is more
about. like app developers trying to get known.
It's extremely funny though. So if you have a
chance, check it out. Anyway, Adam, thank you
so much for joining us. Did you have anything
else you want to tell our listeners? Anybody
you want to kind of, where people can find your
information? Yeah. My, like most faculty, I maintain
a website. I don't update it enough. Like also
all other faculty, you know, like in 20 years,
I'll have the exact same photo I have now. But
like addemail .org is my website. And if someone
wants to reach me, then like my email's there.
And if someone wants to either as a house hunter
get involved in your own research in some way,
you know, hit me up. Awesome. Thank you so much.
I appreciate to just all the time you took. Thank
you everybody for listening. As always, that
can be reached through Gmail at makemesickpod.
And I can also be found on Twitter or X at makemesickpod.
Thank you again, Adam, very much for coming on.
I appreciate the time. Remember, everybody, to
wash your hands. Data and dreams in his steady
hands Building a world where hope expands Dr.
Yala's got the spark, the vision so bold He's
reading the answers and the patterns they hold
A glow in the cold, a map to uncover Guiding
us closer, a life to recover The week after y
'all has got the spark He's weaving a fight,
a beacon of science in the darkest night Connecting
the pieces, defying the shroud Lifting the silent,
making us loud Oh -oh -oh -oh -oh -oh -oh -oh
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh
-oh -oh -oh -oh -oh Breast cancer secrets he
dares to seek Giving strength to the scared,
the lost, the weak Dr. Yala's got the spark,
a vision so bold He's reading the answers and
the patterns they hold A glow in the cold, a
map to uncover Guiding us closer, one life to
recover Dr. Yala's got the spark, a vision so
bold Quick Cover! Connecting the pieces, defying
the shroud, lifting the silent, making us loud.
With steady hands Crafting hope From lines of
commands Nearer I see Where shadows hide A beacon
of hope A guiding light your spark promise me
mapping risks with data's grace