WEBVTT

00:00:10.189 --> 00:00:12.289
Welcome back, everybody, to another episode of

00:00:12.289 --> 00:00:15.310
You Make Me Sick. Today, joined by a very, very

00:00:15.310 --> 00:00:19.230
special guest. We're joined by Dr. Adam Yala.

00:00:19.390 --> 00:00:22.309
Dr. Yala is an assistant professor of computational

00:00:22.309 --> 00:00:26.570
precision health at UC Berkeley and UCSF. His

00:00:26.570 --> 00:00:28.550
research focuses on developing machine learning

00:00:28.550 --> 00:00:31.030
methods for personalized medicine and translating

00:00:31.030 --> 00:00:33.969
them into clinical care. His previous research

00:00:33.969 --> 00:00:36.869
is focused on two areas, predicting future cancer

00:00:36.869 --> 00:00:40.329
risk and designing personalized screening policies.

00:00:40.829 --> 00:00:43.710
His breast cancer tool, Mirai, did I pronounce

00:00:43.710 --> 00:00:46.490
that correctly? It's a valid way to pronounce

00:00:46.490 --> 00:00:49.590
it. I used to say Mirai. Mirai? Mirai. All right,

00:00:49.590 --> 00:00:52.390
Mirai. Has been tested at 43 hospitals from 14

00:00:52.390 --> 00:00:55.149
different countries. Adam's tools now underlie

00:00:55.149 --> 00:00:57.149
prospective trials, and his research has been

00:00:57.149 --> 00:00:59.030
featured in the Washington Post, the New York

00:00:59.030 --> 00:01:02.149
Times, and the Boston Globe. He's also been published

00:01:02.149 --> 00:01:03.890
in a number of scientific journals, including

00:01:03.890 --> 00:01:06.459
the Journal of Radiology. Journal of Empirical

00:01:06.459 --> 00:01:10.840
Methods and Natural Language Processing, the

00:01:10.840 --> 00:01:14.079
Journal of Clinical Oncology, Nature Medicine,

00:01:14.400 --> 00:01:17.680
and the Journal of Breast Cancer Research. Dr.

00:01:17.819 --> 00:01:21.359
Yala got his undergrad, his master's, and his

00:01:21.359 --> 00:01:25.019
doctorate all from MIT. At MIT, he worked in

00:01:25.019 --> 00:01:29.680
the Jameel Clinic and the MIT CSAIL. He's here

00:01:29.680 --> 00:01:31.920
today to talk about not only his work, which

00:01:31.920 --> 00:01:34.680
is fantastic as far as cancer detection using

00:01:34.680 --> 00:01:37.319
AI, but also just to touch a little bit on artificial

00:01:37.319 --> 00:01:40.659
intelligence in healthcare, some of the boundaries

00:01:40.659 --> 00:01:43.239
that we see right now, some of the barriers that

00:01:43.239 --> 00:01:46.280
may be stopping AI from being introduced into

00:01:46.280 --> 00:01:49.040
healthcare and some of the possibilities. So

00:01:49.040 --> 00:01:50.640
Adam, thank you so much for joining us today.

00:01:51.680 --> 00:01:55.069
Thanks for having me. It's a pleasure. So I guess

00:01:55.069 --> 00:01:57.010
I'll kind of start with, you know, artificial

00:01:57.010 --> 00:01:59.030
intelligence is pretty much everywhere. It's

00:01:59.030 --> 00:02:01.590
in almost all aspects of life right now. And

00:02:01.590 --> 00:02:03.689
for people who are listening, I think sometimes

00:02:03.689 --> 00:02:05.310
there's some confusion as to what artificial

00:02:05.310 --> 00:02:07.590
intelligence is. And I was wondering if maybe

00:02:07.590 --> 00:02:09.930
you could explain exactly what's meant by AI

00:02:09.930 --> 00:02:12.409
and then how your work may utilize it a little

00:02:12.409 --> 00:02:15.430
bit differently than something compared to chat

00:02:15.430 --> 00:02:20.610
GPT or grok or perplexity. Yeah. So broadly,

00:02:20.789 --> 00:02:22.930
I define AIs as such technologies as designed

00:02:22.930 --> 00:02:25.870
to learn patterns. As I specify behavior, which

00:02:25.870 --> 00:02:28.409
might be through data, like here is a mammogram.

00:02:28.530 --> 00:02:31.069
This person got cancer five years from now. That

00:02:31.069 --> 00:02:34.129
is a pattern in the data. Now find me a way to

00:02:34.129 --> 00:02:36.930
predict that pattern and replicate it. If you

00:02:36.930 --> 00:02:39.250
think of chat GPT, like the analogy there is

00:02:39.250 --> 00:02:40.689
like I have language. What's the next sentence?

00:02:40.830 --> 00:02:43.389
What's the next word? And that's a very flexible

00:02:43.389 --> 00:02:46.069
set of concepts because the world is filled with

00:02:46.069 --> 00:02:48.129
a combination of data A, data B. Can you learn

00:02:48.129 --> 00:02:51.750
the mappings between them? In the way in which

00:02:51.750 --> 00:02:54.050
my work is different from what we often see in

00:02:54.050 --> 00:02:55.669
industry, we have a lot of work in industry that's

00:02:55.669 --> 00:02:58.530
like, how do you do something people already

00:02:58.530 --> 00:03:00.689
do, just maybe faster, maybe more consistently?

00:03:01.409 --> 00:03:06.090
How do you automate known behavior? That can

00:03:06.090 --> 00:03:07.789
be very valuable and has a lot of use cases in

00:03:07.789 --> 00:03:09.729
healthcare. A lot of what I spend time thinking

00:03:09.729 --> 00:03:11.150
on is how do you do things that humans can't

00:03:11.150 --> 00:03:14.349
do? So to go back to the example of cancer risk,

00:03:14.530 --> 00:03:16.530
people can't look at the mammogram and say, this

00:03:16.530 --> 00:03:17.750
person's going to get cancer in three years.

00:03:18.240 --> 00:03:20.240
and here's a subtype. That's not a thing people

00:03:20.240 --> 00:03:22.740
can do. But there's a pattern in that data. And

00:03:22.740 --> 00:03:24.240
so we're developing technologies to kind of find

00:03:24.240 --> 00:03:26.259
it to make something useful out of it. And I

00:03:26.259 --> 00:03:28.520
think a lot about how can we can come up with

00:03:28.520 --> 00:03:30.319
new kinds of workflows that are better than the

00:03:30.319 --> 00:03:32.439
ones that are there now to make outcomes better.

00:03:34.180 --> 00:03:36.719
So kind of with that, I'd like to touch on your

00:03:36.719 --> 00:03:39.860
work that you've done. So you did some work at

00:03:39.860 --> 00:03:44.419
MIT and now at your own lab at UC Berkeley. Initially,

00:03:44.580 --> 00:03:47.409
I think it was a... a project called Sibyl that

00:03:47.409 --> 00:03:49.629
you would work on. And this was using kind of

00:03:49.629 --> 00:03:50.789
some of the information you were just talking

00:03:50.789 --> 00:03:53.449
about, getting a number of, I think it was CAT

00:03:53.449 --> 00:03:56.009
scans that you followed through. And by feeding

00:03:56.009 --> 00:03:57.530
that into an algorithm, seeing if they could

00:03:57.530 --> 00:04:00.669
predict out years in advance lung cancer, correct?

00:04:01.870 --> 00:04:07.610
Yeah. So that model was in, I forget which year,

00:04:07.770 --> 00:04:08.909
it was a little bit after the breast cancer work.

00:04:09.710 --> 00:04:11.990
But the concept there is very similar. It's like,

00:04:12.009 --> 00:04:14.550
so let's step back, what's the problem we're

00:04:14.550 --> 00:04:17.079
trying to solve? In every screening program,

00:04:17.240 --> 00:04:18.480
there's kind of two things you want to do. You

00:04:18.480 --> 00:04:21.500
want to read the current scan well, and then

00:04:21.500 --> 00:04:23.079
you want to figure out when to get the next scan.

00:04:23.560 --> 00:04:25.959
Because if you read it well, cancer is already

00:04:25.959 --> 00:04:28.939
gigantic. It's already kind of too late. So you

00:04:28.939 --> 00:04:31.879
need to have like foresight to have known to

00:04:31.879 --> 00:04:33.779
get the scan when it was still small. And for

00:04:33.779 --> 00:04:35.220
that, you need to get the risk model. And that's

00:04:35.220 --> 00:04:36.519
what we're trying to do in lung cancer with the

00:04:36.519 --> 00:04:40.680
LDCTs. And so, yeah, let's talk about kind of

00:04:40.680 --> 00:04:42.720
your lung cancer research too. Sorry, your breast

00:04:42.720 --> 00:04:46.819
cancer research. Your current model that you

00:04:46.819 --> 00:04:48.399
use right now is actually being used in a lot

00:04:48.399 --> 00:04:51.360
of hospitals. I was kind of curious with regard

00:04:51.360 --> 00:04:54.899
to that. So it's a deep learning model. And I

00:04:54.899 --> 00:04:58.639
know that the study that I read, you had just

00:04:58.639 --> 00:05:00.899
a large amount of data you had to pull from.

00:05:01.040 --> 00:05:03.680
So I think a lot of people, and maybe you can

00:05:03.680 --> 00:05:06.339
talk a little bit about this, when actually creating

00:05:06.339 --> 00:05:09.360
one of these deep learning models, just how much

00:05:09.360 --> 00:05:12.639
data you have, will that affect exactly how...

00:05:13.269 --> 00:05:15.430
I guess how accurate the model will predict.

00:05:15.709 --> 00:05:18.410
So I think there were, was it 150 ,000 different,

00:05:18.470 --> 00:05:20.170
was it mammograms or CAT scans that you guys

00:05:20.170 --> 00:05:23.970
used? Yeah, it was 150 ,000 mammograms. So based

00:05:23.970 --> 00:05:27.410
on that, and you feed, would you need, I guess

00:05:27.410 --> 00:05:29.129
the question I'm trying to ask here, and maybe

00:05:29.129 --> 00:05:33.110
you can explain this too, is based on that, how

00:05:33.110 --> 00:05:36.990
well were you able to kind of create an algorithm

00:05:36.990 --> 00:05:39.209
or put it in the algorithm and predict cancer

00:05:39.209 --> 00:05:42.129
risk in the future? based on that larger volume

00:05:42.129 --> 00:05:48.569
of mammograms. Yeah, so it's easy to understand

00:05:48.569 --> 00:05:51.889
these numbers in relative terms because people

00:05:51.889 --> 00:05:53.870
don't have a good baseline of what is a good

00:05:53.870 --> 00:05:56.329
model versus a bad model. I'll give you first

00:05:56.329 --> 00:05:58.990
raw numbers and we'll contextualize them. So

00:05:58.990 --> 00:06:01.310
normally the way we think about cancer risk models,

00:06:01.410 --> 00:06:03.410
we use something called ROC, AUC as a metric.

00:06:04.569 --> 00:06:07.589
It's a number between 50 and 100. 50 is random,

00:06:07.709 --> 00:06:10.259
100 is perfect. And what it means, like if someone's

00:06:10.259 --> 00:06:12.560
going to get cancer, what are the odds you're

00:06:12.560 --> 00:06:14.000
going to have a higher score than someone who's

00:06:14.000 --> 00:06:15.980
not going to get cancer? That's its probability,

00:06:16.339 --> 00:06:21.560
50 to 100. The state of the art and what's been

00:06:21.560 --> 00:06:23.839
like used nationally for a long time, like the

00:06:23.839 --> 00:06:26.199
national like NCCN guidelines, the ACR guidelines

00:06:26.199 --> 00:06:28.920
or whatever, are all built on models that have

00:06:28.920 --> 00:06:32.959
an AUC of like 60. So I guess better than random,

00:06:33.100 --> 00:06:36.980
but like a little bit, you know, not a huge amount.

00:06:37.480 --> 00:06:40.000
I bet it's what we have today. And like, because

00:06:40.000 --> 00:06:42.339
age by itself and breast density and these other

00:06:42.339 --> 00:06:43.899
kinds of people are used to just aren't, don't

00:06:43.899 --> 00:06:47.879
have all that much information. Mirai, the model

00:06:47.879 --> 00:06:49.480
that I proposed in that work that we've been

00:06:49.480 --> 00:06:50.980
validating a bunch of places now in these projected

00:06:50.980 --> 00:06:54.959
trials is like ballpark 75. Now there's still

00:06:54.959 --> 00:06:57.939
room to grow. And even though we use a lot of

00:06:57.939 --> 00:07:00.459
data, like we use data from like 50 ,000 patients,

00:07:00.899 --> 00:07:03.779
150 ,000 mammograms, in the scheme of things,

00:07:03.800 --> 00:07:05.259
compared to the data that we generate in this

00:07:05.259 --> 00:07:07.149
country, it's nothing. right like we take like

00:07:07.149 --> 00:07:11.550
what 40 million mammograms a year uh and if you

00:07:11.550 --> 00:07:13.569
look at the way like in deep learning we talk

00:07:13.569 --> 00:07:15.810
a lot about overfitting like what's what's the

00:07:15.810 --> 00:07:17.430
performance of the training sets versus the kind

00:07:17.430 --> 00:07:19.529
of testing sets that you you're generalizing

00:07:19.529 --> 00:07:23.550
to like the model is still like math they could

00:07:23.550 --> 00:07:25.089
massively benefit from much more data than they

00:07:25.089 --> 00:07:27.310
have now and so i think like we're still kind

00:07:27.310 --> 00:07:29.050
of an infancy of the space which people to do

00:07:29.050 --> 00:07:31.980
much much better than we are now And with the

00:07:31.980 --> 00:07:34.220
current, I know that you're in hospitals right

00:07:34.220 --> 00:07:36.160
now. With that data that you're collecting, I'm

00:07:36.160 --> 00:07:38.199
assuming that'll get fed back in and kind of

00:07:38.199 --> 00:07:42.720
help to just improve on your algorithm and Mirai

00:07:42.720 --> 00:07:47.879
right now, correct? Ish. So most of the data

00:07:47.879 --> 00:07:49.860
that exists is always data in the past because

00:07:49.860 --> 00:07:51.500
you don't just need to have the mammogram. You

00:07:51.500 --> 00:07:52.980
need to know what happened to this person within

00:07:52.980 --> 00:07:56.860
the next five years. And so let's say the trials

00:07:56.860 --> 00:07:59.480
that were running. you're collecting like tens

00:07:59.480 --> 00:08:01.120
of thousands of new scans, you're getting high

00:08:01.120 --> 00:08:02.120
risk scores, you're figuring out who's going

00:08:02.120 --> 00:08:04.680
to get an MRI. For that data to be able to be

00:08:04.680 --> 00:08:06.639
used in the training pipeline will take like

00:08:06.639 --> 00:08:09.360
years. And it's a small fraction compared to

00:08:09.360 --> 00:08:11.100
all the data accumulated across the last couple

00:08:11.100 --> 00:08:13.759
of decades. So really like the way to kind of

00:08:13.759 --> 00:08:15.319
scale data sets is more by getting more people

00:08:15.319 --> 00:08:18.079
on board because you want to get the last 10,

00:08:18.120 --> 00:08:21.100
15 years of data. Whereas the next month of data

00:08:21.100 --> 00:08:24.279
is like a drop in the bucket. You see what I

00:08:24.279 --> 00:08:27.079
mean? Sure. Yeah. So just very, very small, like

00:08:27.079 --> 00:08:29.040
you're saying, compared historically to all the

00:08:29.040 --> 00:08:32.320
data you can collect and pull from. Another great

00:08:32.320 --> 00:08:34.679
thing about this study that was brought up, a

00:08:34.679 --> 00:08:36.500
lot of times, sometimes there are ethical concerns

00:08:36.500 --> 00:08:40.259
with using AI in healthcare that there's bias.

00:08:40.379 --> 00:08:44.159
And by bias, I mean, if you just have one population

00:08:44.159 --> 00:08:48.000
with the same geographic area, then there's a

00:08:48.000 --> 00:08:51.259
possibility that they're going to be different,

00:08:51.320 --> 00:08:52.879
have different outcomes than somewhere else.

00:08:53.470 --> 00:08:55.549
But for this study, you guys actually geographically

00:08:55.549 --> 00:08:58.049
selected multiple hospitals in multiple geographic

00:08:58.049 --> 00:09:00.970
areas around the world, correct? Yeah, yeah.

00:09:02.309 --> 00:09:05.009
It started off with a lot of cold emails, and

00:09:05.009 --> 00:09:07.009
some dear friends helped me out to find more

00:09:07.009 --> 00:09:10.409
possible collaborators. I flew to Taiwan. I met

00:09:10.409 --> 00:09:12.049
Gigan Lin, who's a great collaborator at Chang

00:09:12.049 --> 00:09:13.730
'e Memorial Hospital. So we did a validation

00:09:13.730 --> 00:09:16.649
there. I flew to Sweden, met Frederik Strand,

00:09:16.950 --> 00:09:18.929
who's a great collaborator. We validated at the

00:09:18.929 --> 00:09:22.730
Karl Linska Institute. uh and we've since now

00:09:22.730 --> 00:09:27.409
validated across ballpark 40 hospitals for more

00:09:27.409 --> 00:09:30.950
than 14 countries uh and what we're finding is

00:09:30.950 --> 00:09:32.570
that as you kind of go across the things are

00:09:32.570 --> 00:09:35.029
working out of the box uh so we know that the

00:09:35.029 --> 00:09:37.070
finding that we're finding like the pattern that

00:09:37.070 --> 00:09:39.529
we learn mgh mammograms we originally built the

00:09:39.529 --> 00:09:41.970
model do generalize they are general which is

00:09:41.970 --> 00:09:45.850
good uh but uh so it's kind of always two threads

00:09:45.850 --> 00:09:48.419
like given that technology you have today how

00:09:48.419 --> 00:09:50.100
do we make it most useful? What are the trials

00:09:50.100 --> 00:09:53.039
we can run now? How can we deliver on the promise?

00:09:53.519 --> 00:09:55.019
And the other third has been a lot of time. I

00:09:55.019 --> 00:09:58.100
was like, okay, well, we didn't peak in 2019.

00:09:58.220 --> 00:10:00.240
We can do better. And how do we build it next?

00:10:02.419 --> 00:10:05.100
So how did you guys come about when you first

00:10:05.100 --> 00:10:07.879
started theorizing what you could do using imaging?

00:10:08.399 --> 00:10:09.960
And this probably goes back to your days at MIT.

00:10:11.500 --> 00:10:13.879
What was the initial kind of catalyst for, well,

00:10:13.919 --> 00:10:16.179
if we can incorporate artificial intelligence,

00:10:17.259 --> 00:10:19.980
using historical radiographical data you know

00:10:19.980 --> 00:10:21.860
what outcomes could we find could we help detect

00:10:21.860 --> 00:10:24.759
future cancers could we find could ai find things

00:10:24.759 --> 00:10:26.799
on the radiographs that a radiologist would miss

00:10:26.799 --> 00:10:30.259
uh like when this oh when you first started your

00:10:30.259 --> 00:10:32.820
work what was kind of the initial thought process

00:10:32.820 --> 00:10:36.500
behind what you guys could actually create yes

00:10:36.500 --> 00:10:39.519
i mean a lot of the uh i started working with

00:10:39.519 --> 00:10:43.019
cancer because of my pc advisor guida's personal

00:10:43.019 --> 00:10:48.100
experience with cancer and the The idea of we

00:10:48.100 --> 00:10:50.080
could cash cancer better if we knew who's going

00:10:50.080 --> 00:10:52.779
to get it is not a novel idea that we just came

00:10:52.779 --> 00:10:53.980
up with. People have been thinking about this

00:10:53.980 --> 00:10:56.779
in one way or another since like the 60s. And

00:10:56.779 --> 00:10:59.120
like even in the 60s, people were saying, oh,

00:10:59.159 --> 00:11:01.019
there's this thing in the mammogram. Like it

00:11:01.019 --> 00:11:02.779
looks a bit different if someone is going to

00:11:02.779 --> 00:11:05.639
be riskier versus not. And then eventually, like

00:11:05.639 --> 00:11:09.340
across decades, decades of work, they eventually

00:11:09.340 --> 00:11:13.379
hit like federalization in 2019. And like coming

00:11:13.379 --> 00:11:15.970
from the AI world, it's like. You know, you go

00:11:15.970 --> 00:11:18.649
on Amazon, and you can bet that, like, so many

00:11:18.649 --> 00:11:21.549
of the decisions you've made ever are influencing

00:11:21.549 --> 00:11:23.409
what's shown to you, your ranking and your ads

00:11:23.409 --> 00:11:25.570
and whatever. And, like, inside the healthcare

00:11:25.570 --> 00:11:27.549
machine, there's very little information used

00:11:27.549 --> 00:11:30.710
to say anything about anything. And every time

00:11:30.710 --> 00:11:33.370
you kind of look more deeply at a problem, like

00:11:33.370 --> 00:11:35.330
how we thought about risk, and you kind of quantify

00:11:35.330 --> 00:11:36.490
it and view it as a prediction problem, you realize,

00:11:36.549 --> 00:11:39.230
like, there's just a lot of room to do better.

00:11:39.830 --> 00:11:42.389
Like, even, like, one of the most important decisions

00:11:42.389 --> 00:11:44.330
we make right now is who do you screen in the

00:11:44.330 --> 00:11:46.529
first place? before you even get the mammogram,

00:11:46.730 --> 00:11:49.250
what should someone start getting a mammogram?

00:11:49.970 --> 00:11:51.909
That is a decision. That's a prediction problem,

00:11:52.090 --> 00:11:54.230
right? Like it's a true positive. You give someone

00:11:54.230 --> 00:11:55.909
a mammogram, they eventually got cancer, so they

00:11:55.909 --> 00:11:59.350
caught it earlier. And it's a negative if you

00:11:59.350 --> 00:12:01.049
didn't give them a mammogram and they got cancer,

00:12:01.149 --> 00:12:03.990
so they get caught later, or you gave it to them

00:12:03.990 --> 00:12:06.009
and they never had cancer, right? It's a waste

00:12:06.009 --> 00:12:08.529
there because all you did was bother them. That's

00:12:08.529 --> 00:12:10.110
a classifier, right? You can evaluate how well

00:12:10.110 --> 00:12:14.529
you predict that. And we only use age. Which

00:12:14.529 --> 00:12:16.309
is, yeah, it's kind of a crude metric, you know?

00:12:16.690 --> 00:12:20.710
Do we only know your age? Especially if you're

00:12:20.710 --> 00:12:22.509
like a patient of this hospital, you know, so

00:12:22.509 --> 00:12:25.509
much more. And we don't use any of it. And so

00:12:25.509 --> 00:12:28.269
when I meet people who are like outside of screening

00:12:28.269 --> 00:12:30.269
guidelines, like before 40, they have late stage

00:12:30.269 --> 00:12:32.669
breast cancer, like we could have done better

00:12:32.669 --> 00:12:34.710
for those people if we just built the right technologies.

00:12:36.009 --> 00:12:38.269
When you have people in screening that like still

00:12:38.269 --> 00:12:40.110
get late stage breast cancer, still could have

00:12:40.110 --> 00:12:41.110
done better for those people who had the right

00:12:41.110 --> 00:12:42.470
technologies. And so there's a lot of room to

00:12:42.470 --> 00:12:45.509
do better. And the deeper you go into it from

00:12:45.509 --> 00:12:46.769
a technology perspective, the more you're like,

00:12:46.950 --> 00:12:49.289
the tools we have are crude. And even now, with

00:12:49.289 --> 00:12:52.350
the best of AI we have now, even now we still

00:12:52.350 --> 00:12:54.289
can't touch all of it. Because we have it actually

00:12:54.289 --> 00:12:55.990
really big. We have so much about people. We

00:12:55.990 --> 00:12:59.610
don't use it. When you're feeding this information

00:12:59.610 --> 00:13:03.070
into your algorithm, is it just the radiographic

00:13:03.070 --> 00:13:06.070
image? Or do you also have anything else? Have

00:13:06.070 --> 00:13:10.169
you thought about or thought about adding? It

00:13:10.169 --> 00:13:11.669
might be difficult to do, I don't know, but for

00:13:11.669 --> 00:13:14.669
each mammogram also have, you know, lab work,

00:13:14.750 --> 00:13:17.730
history, do you have family breast cancer history,

00:13:18.090 --> 00:13:21.669
any kind of, I don't know, some people get genetic

00:13:21.669 --> 00:13:24.090
testing, any of that information, could that

00:13:24.090 --> 00:13:27.909
also be kind of added in, you know, to your algorithm

00:13:27.909 --> 00:13:29.529
to help predict? Is that something you guys have

00:13:29.529 --> 00:13:31.309
thought about or would that just be too much

00:13:31.309 --> 00:13:34.990
at this point? We've looked into it. So that

00:13:34.990 --> 00:13:40.360
part is, okay, so. let's say you're combining

00:13:40.360 --> 00:13:43.659
the factors people know matter. So family history,

00:13:44.000 --> 00:13:47.179
questionnaires about like reproductive health,

00:13:47.500 --> 00:13:50.120
there's a person breastfeed and whatever, like

00:13:50.120 --> 00:13:51.460
all this kind of like detailed questionnaire

00:13:51.460 --> 00:13:53.840
information. If you add that back to the image,

00:13:54.019 --> 00:13:58.399
it barely makes a difference. Like you go from

00:13:58.399 --> 00:14:03.679
75 to 76, it's like it's not a big gap. And I

00:14:03.679 --> 00:14:05.059
think it's because the questionnaires themselves

00:14:05.059 --> 00:14:07.460
are too coarse. And we did some experiments.

00:14:07.639 --> 00:14:09.259
It can actually predict a lot of these factors

00:14:09.259 --> 00:14:11.759
from the image. So, like, if someone had children

00:14:11.759 --> 00:14:13.159
before, it can predict from the image. If someone

00:14:13.159 --> 00:14:14.279
breastfed before, it can predict from the image.

00:14:14.500 --> 00:14:15.960
Like, there's a lot of things like that that

00:14:15.960 --> 00:14:18.299
are, like, readable from the mammogram. And so

00:14:18.299 --> 00:14:20.299
adding it back into this questionnaire doesn't

00:14:20.299 --> 00:14:22.980
really help. I think there's other stuff that

00:14:22.980 --> 00:14:24.620
we're not currently incorporating that were the

00:14:24.620 --> 00:14:26.220
challenges of having all the data in the same

00:14:26.220 --> 00:14:27.879
place that's not currently there that I think

00:14:27.879 --> 00:14:29.759
would be powerful. So we had this collaboration

00:14:29.759 --> 00:14:32.299
with some colleagues in Denmark where we're building

00:14:32.299 --> 00:14:35.029
risk models looking at, like... their entire

00:14:35.029 --> 00:14:36.470
health trajectory, because they have these big

00:14:36.470 --> 00:14:39.690
national registries, to predict when someone

00:14:39.690 --> 00:14:41.870
should start screening. And that stuff is pretty

00:14:41.870 --> 00:14:44.470
good. And that is much better than things like

00:14:44.470 --> 00:14:45.970
age, and that should be more complementary to

00:14:45.970 --> 00:14:48.649
the mammogram. But the classic epidemiological

00:14:48.649 --> 00:14:52.450
questionnaire stuff, that's underlying the various

00:14:52.450 --> 00:14:55.129
cancer risk models, doesn't add all that much

00:14:55.129 --> 00:14:58.169
compared to just the image. Which is pretty impressive

00:14:58.169 --> 00:14:59.590
when you think about it, because you would think,

00:14:59.610 --> 00:15:01.070
you know, even having a little bit history, I

00:15:01.070 --> 00:15:02.549
thought it would sway it a little bit more. But

00:15:02.549 --> 00:15:04.669
I guess just the radiographic image seems to

00:15:04.669 --> 00:15:07.269
be kind of where you're getting most of that

00:15:07.269 --> 00:15:12.129
data from. How, as compared to, you know, a radiologist

00:15:12.129 --> 00:15:15.289
who would read this, how well does the AI kind

00:15:15.289 --> 00:15:19.990
of detect it in comparison? And I'm sure, I know,

00:15:20.029 --> 00:15:22.389
I think I read that you guys, I can't remember

00:15:22.389 --> 00:15:23.490
if it was this study or it might have been the

00:15:23.490 --> 00:15:27.860
Sybil study. So you had a radiologist kind of

00:15:27.860 --> 00:15:30.220
take a read as well, and you compared it with

00:15:30.220 --> 00:15:36.679
the AI read. And the AI read is very close to

00:15:36.679 --> 00:15:38.000
what the radiologist would catch. Do they catch

00:15:38.000 --> 00:15:39.500
more things than the radiologist could catch?

00:15:40.820 --> 00:15:44.659
Yeah, so the lung cancer model is a little bit

00:15:44.659 --> 00:15:49.779
newer. So there's two questions. One is how do

00:15:49.779 --> 00:15:51.200
you find current cancer versus future cancer?

00:15:52.500 --> 00:15:55.440
Radiologists. as it is their above part of their

00:15:55.440 --> 00:15:57.840
job, are pretty good at finding current cancer,

00:15:57.960 --> 00:15:59.279
at least the ones that are specialized in fellowship

00:15:59.279 --> 00:16:03.019
training, et cetera. And in the evaluation that

00:16:03.019 --> 00:16:06.580
we did, Seibel was fairly close to that performance.

00:16:07.519 --> 00:16:10.759
And Mirai is not as good as a breast imaging

00:16:10.759 --> 00:16:11.980
radiologist, partially because they rely more

00:16:11.980 --> 00:16:15.519
on priors and stuff. That model is a little bit

00:16:15.519 --> 00:16:17.539
older. Today, you can get quite a bit closer

00:16:17.539 --> 00:16:19.519
to that than was possible when I first did this.

00:16:20.799 --> 00:16:23.240
When you try to understand future cancer, People

00:16:23.240 --> 00:16:26.620
can't really do this. It's something they're

00:16:26.620 --> 00:16:28.460
trying to do, but really they can't. That's why

00:16:28.460 --> 00:16:29.799
people came up with notions like breast density.

00:16:30.320 --> 00:16:34.379
And in there, you do way better. And so that's

00:16:34.379 --> 00:16:37.039
where the technology has been focused. The detection

00:16:37.039 --> 00:16:38.940
capability for both MIRA and SIBO are kind of

00:16:38.940 --> 00:16:42.500
like an ancillary add -on that kind of came out

00:16:42.500 --> 00:16:44.600
of the process. The core focus is understanding

00:16:44.600 --> 00:16:46.200
how do you understand who's going to get cancer

00:16:46.200 --> 00:16:48.120
and what can you do about it. So that's how we

00:16:48.120 --> 00:16:49.720
were trying to change the workflow. We weren't

00:16:49.720 --> 00:16:51.500
trying to change how someone's reading it today.

00:16:52.220 --> 00:16:54.100
is how do you schedule the next person so you're

00:16:54.100 --> 00:16:56.940
getting rid of the best possible time. In the

00:16:56.940 --> 00:16:58.919
current phase, we're doing more of everything.

00:16:59.440 --> 00:17:02.940
So now we're also trying to improve the workflow,

00:17:03.179 --> 00:17:04.539
help the person read it, make them more efficient,

00:17:04.720 --> 00:17:06.220
making better caching stuff in addition to all

00:17:06.220 --> 00:17:08.660
these other longer -term components. But that's

00:17:08.660 --> 00:17:13.099
the newer generation of stuff. Do you think that...

00:17:13.099 --> 00:17:17.640
You have, what did I say, 14 different hospitals

00:17:17.640 --> 00:17:21.460
now we're trialing. your algorithm to detect

00:17:21.460 --> 00:17:23.920
breast cancer. Do you see a day where this is

00:17:23.920 --> 00:17:26.059
something that's going to be used just as kind

00:17:26.059 --> 00:17:28.619
of almost like the gold standard as far as predicting

00:17:28.619 --> 00:17:32.539
breast cancer in women? Or is it something you

00:17:32.539 --> 00:17:35.119
feel is, you know, is that years away? Is that

00:17:35.119 --> 00:17:37.039
something that could happen relatively soon?

00:17:38.319 --> 00:17:42.859
The real challenge we face now is more I mean,

00:17:42.880 --> 00:17:44.519
there's lots of protective trials that we're

00:17:44.519 --> 00:17:45.819
running, and I think there's a lot of stuff to

00:17:45.819 --> 00:17:47.519
learn clinically about how to use these tools.

00:17:47.680 --> 00:17:49.559
And these things take time. You need to follow

00:17:49.559 --> 00:17:51.240
patients for years. You need to roll with consent.

00:17:52.099 --> 00:17:54.140
There's no way to supercharge that stuff in speed.

00:17:54.680 --> 00:17:56.500
But the bottleneck to have this widely accessible

00:17:56.500 --> 00:18:01.259
is more regulatory than anything. So the FDA

00:18:01.259 --> 00:18:05.500
has some constraints. The way the law is written

00:18:05.500 --> 00:18:10.980
is that if an algorithm looks at an image, even

00:18:10.980 --> 00:18:12.519
if it's doing the same thing as what is currently

00:18:12.519 --> 00:18:14.660
not a device, it becomes a device. So the classical

00:18:14.660 --> 00:18:17.220
risk model that people use today that looks at

00:18:17.220 --> 00:18:20.759
density and whatever, which looks at an image

00:18:20.759 --> 00:18:22.539
in a way, but you enter the density yourself,

00:18:23.019 --> 00:18:25.220
that's not a device. And that's what's in the

00:18:25.220 --> 00:18:27.319
guidelines. Because you look at the image directly,

00:18:27.579 --> 00:18:29.500
that's a device now. And now it needs to have

00:18:29.500 --> 00:18:32.539
expensive filings, you need to have a commercial

00:18:32.539 --> 00:18:34.339
structure around it to make it feasible. And

00:18:34.339 --> 00:18:37.099
I think that has been the major roadblock in

00:18:37.099 --> 00:18:39.870
order to have this more widely accessible. and

00:18:39.870 --> 00:18:43.309
we're thinking of ways to amortize that cost

00:18:43.309 --> 00:18:48.170
because it's hard to... Basically, the commercial

00:18:48.170 --> 00:18:50.549
structure, which is only caused by the FDA, is

00:18:50.549 --> 00:18:52.289
hard. It means doing this in the open, free way

00:18:52.289 --> 00:18:54.529
that we wanted to do it becomes structurally

00:18:54.529 --> 00:18:57.269
impossible, which means now we need to lump it

00:18:57.269 --> 00:18:58.849
into a larger commercial thing, and we're thinking

00:18:58.849 --> 00:19:01.059
about the right way to do that. Yeah, I was going

00:19:01.059 --> 00:19:02.440
to ask you, that's actually a question, because

00:19:02.440 --> 00:19:04.559
often when medications are first developed, it

00:19:04.559 --> 00:19:06.980
takes, you know, years. It could be almost seven

00:19:06.980 --> 00:19:10.880
to 10 years of trials and then just review just

00:19:10.880 --> 00:19:14.059
to receive approval from the FDA. Are you looking

00:19:14.059 --> 00:19:16.059
at that kind of timeline for this type of technology?

00:19:16.259 --> 00:19:18.200
Do you think it'll be a shorter timeline? Have

00:19:18.200 --> 00:19:20.299
you ever, I mean, I'm assuming this is probably

00:19:20.299 --> 00:19:21.680
the first time you've had to go through this

00:19:21.680 --> 00:19:23.900
type or these types of steps dealing with federal

00:19:23.900 --> 00:19:27.380
regulations, incorporating AI into healthcare

00:19:27.380 --> 00:19:29.289
and getting it passed. Do you think it could

00:19:29.289 --> 00:19:33.069
take that long to actually implement this? As

00:19:33.069 --> 00:19:36.869
far as like almost up to a decade? I think that

00:19:36.869 --> 00:19:40.329
is the default in healthcare today. And it is

00:19:40.329 --> 00:19:43.789
the grand challenge of one's work to make it

00:19:43.789 --> 00:19:46.069
not take that long. And to at least have the

00:19:46.069 --> 00:19:48.210
second model and the third model not take that

00:19:48.210 --> 00:19:51.130
long and to get faster and faster. I mean, there's

00:19:51.130 --> 00:19:54.920
a lot of bottlenecks across the process. Risk

00:19:54.920 --> 00:19:56.660
models are used for many things. One thing we

00:19:56.660 --> 00:20:00.160
use them for is to say who gets MRI. Finding

00:20:00.160 --> 00:20:02.640
the partnerships to even stop those MRI trials

00:20:02.640 --> 00:20:05.299
took years. Now that we have them, it takes years

00:20:05.299 --> 00:20:07.279
to know patients. It takes years to follow them.

00:20:07.819 --> 00:20:09.460
And you're already approaching this kind of like,

00:20:09.579 --> 00:20:12.220
to do the final trial readout, I think it will

00:20:12.220 --> 00:20:16.920
take closer to, it's a couple of years from today.

00:20:17.119 --> 00:20:19.660
And in that, we'll have been thinking about this

00:20:19.660 --> 00:20:23.099
for at least five years. And we're super grateful

00:20:23.099 --> 00:20:25.319
to the partners that we found. BCRF is a fantastic

00:20:25.319 --> 00:20:27.839
sponsor. And we'll not be able to do the study

00:20:27.839 --> 00:20:33.819
without them. But at a system level, we can't

00:20:33.819 --> 00:20:35.259
have this happen for every time we have a new

00:20:35.259 --> 00:20:37.200
model. We need to have faster turnaround to get

00:20:37.200 --> 00:20:40.099
information to helping people. Because even when

00:20:40.099 --> 00:20:41.339
we have that, it doesn't solve the FDA problem.

00:20:41.579 --> 00:20:42.859
The FDA does not even look at trials. It just

00:20:42.859 --> 00:20:46.660
looks at other enumerated things. So there's

00:20:46.660 --> 00:20:51.109
a lot of work to do across every access. How

00:20:51.109 --> 00:20:52.230
do you make this thing financially sustainable

00:20:52.230 --> 00:20:55.589
so you can finance improving it and have it accessible

00:20:55.589 --> 00:20:57.509
at scale and go fill the regulatory fees they

00:20:57.509 --> 00:20:59.289
have to go across to make this widely available

00:20:59.289 --> 00:21:02.490
is a grand challenge. In addition to generating

00:21:02.490 --> 00:21:04.269
the kind of evidence that we want to redesign

00:21:04.269 --> 00:21:07.490
the next generation of cancer guidelines. Yeah,

00:21:07.529 --> 00:21:10.549
so your lab at UC Berkeley, you focus on three

00:21:10.549 --> 00:21:13.529
major themes just to provide customized care

00:21:13.529 --> 00:21:15.890
to cancer patients. I was wondering if you could

00:21:15.890 --> 00:21:18.349
explain kind of, you know, each of these themes

00:21:18.349 --> 00:21:20.730
and how they kind of work together with regard

00:21:20.730 --> 00:21:23.430
to actually that customization of care for patients

00:21:23.430 --> 00:21:29.430
with using your technology. Yeah. So to give

00:21:29.430 --> 00:21:31.569
that kind of the motivation we want to make possible,

00:21:32.029 --> 00:21:35.269
there's a lot of problems in cancer. It'd be

00:21:35.269 --> 00:21:38.410
a lot easier if you just had foresight. What

00:21:38.410 --> 00:21:39.809
I mean by that is like, okay, if you knew who

00:21:39.809 --> 00:21:40.849
was going to get cancer at a particular point

00:21:40.849 --> 00:21:43.769
in time, it's easy way to do about it. You just

00:21:43.769 --> 00:21:45.500
don't know. And so you get all of these, like,

00:21:45.500 --> 00:21:48.819
advanced cancer stages. If you knew which therapy

00:21:48.819 --> 00:21:50.460
you were going to respond to, and this was not

00:21:50.460 --> 00:21:53.359
going to work, easy, you know? It's how you generate

00:21:53.359 --> 00:21:54.920
that knowledge. And that kind of leads to kind

00:21:54.920 --> 00:21:57.160
of three classes of questions that my group works

00:21:57.160 --> 00:22:01.079
on. One I would broadly call improving predictive

00:22:01.079 --> 00:22:04.200
capacity. So I mentioned, like, before, like,

00:22:04.200 --> 00:22:06.000
with screening guidelines, like, people use their

00:22:06.000 --> 00:22:08.079
age. There's more than that out there, right?

00:22:08.119 --> 00:22:09.319
You could use, like, whole health trajectory.

00:22:10.039 --> 00:22:12.660
But real data we have on cancer patients is multiple

00:22:12.660 --> 00:22:15.890
gigabytes per person. That is way bigger than

00:22:15.890 --> 00:22:18.630
anything we know how to model in AI today. Even

00:22:18.630 --> 00:22:22.609
the big LLMs, they look at like 100 ,000 tokens.

00:22:23.789 --> 00:22:27.690
Patients have billions of tokens. And so we need

00:22:27.690 --> 00:22:29.829
to do quite a bit better at understanding things

00:22:29.829 --> 00:22:33.029
at scale and super efficient models and to learn

00:22:33.029 --> 00:22:34.509
the dependencies and patterns across this really

00:22:34.509 --> 00:22:36.690
massive scale. So that leads to new work and

00:22:36.690 --> 00:22:38.970
new kinds of neural network architectures, ways

00:22:38.970 --> 00:22:40.769
to learn more efficiently, and all kinds of stuff.

00:22:40.769 --> 00:22:42.329
How do you better deal with the fact that data

00:22:42.329 --> 00:22:45.720
is gigantic? And you don't have 100 billion people.

00:22:45.799 --> 00:22:47.000
You only have the people in the hospital that

00:22:47.000 --> 00:22:50.099
are actually there. So it's big in that the samples

00:22:50.099 --> 00:22:51.359
are really big, but you don't have that many

00:22:51.359 --> 00:22:53.160
patients. And how do you learn efficiently in

00:22:53.160 --> 00:22:54.319
that kind of regime? That's kind of one bucket

00:22:54.319 --> 00:22:55.859
to give you better. And the way you tell us that

00:22:55.859 --> 00:22:58.299
worked is now you beat Mirai, you beat Seibel,

00:22:58.420 --> 00:23:00.599
you have tumor response models. You're better

00:23:00.599 --> 00:23:01.940
predicting the various things we care about in

00:23:01.940 --> 00:23:03.900
cancer. That's one piece, but it's not enough.

00:23:05.440 --> 00:23:07.440
Because if you just have that, you need to do

00:23:07.440 --> 00:23:09.039
something about it. And that's the second bucket

00:23:09.039 --> 00:23:10.819
of problem is how do you make better decision

00:23:10.819 --> 00:23:13.809
-making given this kind of predictors? i call

00:23:13.809 --> 00:23:16.210
that control so i design new kind of screening

00:23:16.210 --> 00:23:18.490
policies treatment plans and whatever and the

00:23:18.490 --> 00:23:19.789
last bucket because even if you have that that's

00:23:19.789 --> 00:23:22.170
not enough because you need if you have a better

00:23:22.170 --> 00:23:24.190
predictor you have better action like a better

00:23:24.190 --> 00:23:27.410
guideline people still can't use it uh so the

00:23:27.410 --> 00:23:30.190
last bucket are called translation and then we're

00:23:30.190 --> 00:23:31.769
working on like new kinds of death of trial design

00:23:31.769 --> 00:23:33.109
how to make trials more efficient because you

00:23:33.109 --> 00:23:35.589
get on many of them uh what's the right kind

00:23:35.589 --> 00:23:37.650
of pipeline to kind of take this to the end consumer

00:23:37.650 --> 00:23:39.690
and all this kind of stuff it's kind of three

00:23:39.690 --> 00:23:42.269
broad buckets from prediction to control to translation

00:23:44.700 --> 00:23:47.680
That's a big spiel. No, no, no, no. It's actually,

00:23:47.740 --> 00:23:49.500
you explained it really well. And I think that

00:23:49.500 --> 00:23:52.680
it's sort of personalization of care. I know

00:23:52.680 --> 00:23:55.140
that you were involved in another study. So where

00:23:55.140 --> 00:23:57.599
I work, we use Epic, which is a medical record

00:23:57.599 --> 00:24:00.140
health system. People come in, say you come to

00:24:00.140 --> 00:24:02.900
our emergency department and you have a fever

00:24:02.900 --> 00:24:05.700
and a high heart rate and your white blood cell

00:24:05.700 --> 00:24:08.440
count is high. It'll set up a red flag in our

00:24:08.440 --> 00:24:10.160
system just from those indicators that you might

00:24:10.160 --> 00:24:12.839
have a process called sepsis. So a whole body

00:24:12.839 --> 00:24:16.119
infection. can be life -threatening. Based on

00:24:16.119 --> 00:24:19.220
that, it'll trigger certain labs that need to

00:24:19.220 --> 00:24:21.960
be done. It'll notify a team of physicians. It'll

00:24:21.960 --> 00:24:24.140
notify the nurses on certain tasks that need

00:24:24.140 --> 00:24:26.339
to be done. And I'm pretty sure you worked on

00:24:26.339 --> 00:24:27.940
a project, and I don't know if it was using Epic,

00:24:28.000 --> 00:24:30.099
but there were a couple of health medical record

00:24:30.099 --> 00:24:33.240
systems, and you were trying to figure out which

00:24:33.240 --> 00:24:36.940
had a better predictor for sepsis based on just

00:24:36.940 --> 00:24:39.019
certain indicators or certain points of data.

00:24:39.559 --> 00:24:42.980
And I can't remember the study correctly if it

00:24:42.980 --> 00:24:45.829
was... if you had a better prediction with fewer

00:24:45.829 --> 00:24:48.809
points of data because it confounded the results,

00:24:49.069 --> 00:24:51.509
or if you had a better outcome with more points

00:24:51.509 --> 00:24:53.450
of data because it gave you more information.

00:24:53.950 --> 00:24:56.349
And if that was kind of, I'm trying to remember

00:24:56.349 --> 00:24:59.690
which was the better predictor for sepsis. Yeah,

00:24:59.750 --> 00:25:02.789
so this is a collaboration with Stat News with

00:25:02.789 --> 00:25:07.509
Casey Ross. And what we were looking at is how

00:25:07.509 --> 00:25:09.130
do you even evaluate these predictors? So normally

00:25:09.130 --> 00:25:11.309
we have these sepsis models. There's many of

00:25:11.309 --> 00:25:13.960
them. You get told like, here's an AUC, here's

00:25:13.960 --> 00:25:15.859
a number, here's a sensitivity specificity of

00:25:15.859 --> 00:25:20.039
your model. And we wanted to test and show in

00:25:20.039 --> 00:25:23.359
that work is like, does that number mean the

00:25:23.359 --> 00:25:26.160
same thing next year and the year after that?

00:25:26.460 --> 00:25:28.400
When you build something on the output of the

00:25:28.400 --> 00:25:31.720
health system, like you would, it's not a causal

00:25:31.720 --> 00:25:35.059
model. What I mean by that is like, you do stuff

00:25:35.059 --> 00:25:37.500
because, you know, there's guidelines and you

00:25:37.500 --> 00:25:39.200
get told like, oh, for this, give them that.

00:25:39.539 --> 00:25:41.519
You have a workflow. that workflow generates

00:25:41.519 --> 00:25:44.420
data. And you learn from the output to that workflow,

00:25:44.599 --> 00:25:46.079
the kind of data that's generated to predict

00:25:46.079 --> 00:25:48.160
who has sepsis. But if you change your workflow

00:25:48.160 --> 00:25:51.559
next year, the model could stop working because

00:25:51.559 --> 00:25:53.019
now you have totally different data that's linked

00:25:53.019 --> 00:25:54.440
to that kind of current outcome. Because you

00:25:54.440 --> 00:25:56.279
don't have like this like defense logical understanding.

00:25:56.339 --> 00:25:57.859
You're kind of going after the correlates that

00:25:57.859 --> 00:25:59.859
are built through the healthcare system. What

00:25:59.859 --> 00:26:01.680
we showed if you replicate something that's close

00:26:01.680 --> 00:26:04.440
to the epic sepsis model, goes to the model size,

00:26:04.700 --> 00:26:08.349
and you built it in like 2012. Then you ran it.

00:26:08.410 --> 00:26:10.490
It goes from an AUC of, like, 90 or something

00:26:10.490 --> 00:26:14.930
to 50 across, like, a 10 -year period. And we

00:26:14.930 --> 00:26:16.349
did this with using the MIMIC data set. And it

00:26:16.349 --> 00:26:21.049
was, like, in the beginning, like, oh, Beth Israel

00:26:21.049 --> 00:26:23.210
bought another health center. And they expanded.

00:26:23.349 --> 00:26:25.069
And they added this. And this is this change.

00:26:25.250 --> 00:26:27.769
And this particular order set change. And then

00:26:27.769 --> 00:26:29.630
you kind of, like, you just kind of see the numbers

00:26:29.630 --> 00:26:33.109
kind of slowly veer down. And it kind of shows

00:26:33.109 --> 00:26:35.529
an open challenge in this kind of, like, building

00:26:35.529 --> 00:26:38.799
models off the EHR. where unless you have like,

00:26:38.839 --> 00:26:42.559
and everything that's not changing, right? Like

00:26:42.559 --> 00:26:44.160
meaning the way we collect, the way we collect

00:26:44.160 --> 00:26:46.579
the data is self -informative. And if we change

00:26:46.579 --> 00:26:48.180
the way we collect the data, then the model might

00:26:48.180 --> 00:26:50.240
not mean the same thing anymore. So it always

00:26:50.240 --> 00:26:52.200
has to be retraining and like finding a new thing

00:26:52.200 --> 00:26:53.619
and consistently validating all this kind of

00:26:53.619 --> 00:26:58.279
stuff. So I thought consistency essentially is

00:26:58.279 --> 00:27:00.019
where, so when you have so many things that change

00:27:00.019 --> 00:27:01.819
or variables might change, the model probably

00:27:01.819 --> 00:27:05.940
gets worse at predicting. Yeah, because. Am I

00:27:05.940 --> 00:27:09.650
right? is the 2012 pattern, not the 2022 pattern

00:27:09.650 --> 00:27:11.309
or the 2025. Right, so completely different from

00:27:11.309 --> 00:27:14.170
10 years, yeah. Which must be tough, like from

00:27:14.170 --> 00:27:16.789
your standpoint, that must be a tough thing to

00:27:16.789 --> 00:27:19.170
try to correct when you're trying to, you know,

00:27:19.190 --> 00:27:23.490
trying to create the most efficient and accurate

00:27:23.490 --> 00:27:27.170
kind of algorithm. I mean, you can try to find

00:27:27.170 --> 00:27:29.630
the stuff that's stable or that's not changing

00:27:29.630 --> 00:27:31.390
and then model only those that will be more steady.

00:27:32.569 --> 00:27:34.670
But then you could lose out on the stuff that's

00:27:34.670 --> 00:27:37.359
changing but useful. And so it just puts you

00:27:37.359 --> 00:27:38.859
into a perspective. We kind of have the current

00:27:38.859 --> 00:27:41.359
mindset of you build an algorithm once, it's

00:27:41.359 --> 00:27:43.380
done, put a bow on it, ship it out, you're done.

00:27:44.680 --> 00:27:47.440
For some classes of problems, like imaging, that

00:27:47.440 --> 00:27:50.119
can actually work because an X -ray is an X -ray

00:27:50.119 --> 00:27:52.160
is an X -ray. It's not changing all that much.

00:27:52.299 --> 00:27:54.900
It's not like we're inventing new ribs. There's

00:27:54.900 --> 00:27:58.819
not a lot of change that's happening there. The

00:27:58.819 --> 00:28:02.119
EHR, stuff that's like the output of human workflows,

00:28:02.480 --> 00:28:06.470
is not like that. And so it requires a different

00:28:06.470 --> 00:28:08.289
kind of framework where you're like building

00:28:08.289 --> 00:28:11.150
stuff all the time. And there's a model of like,

00:28:11.250 --> 00:28:14.230
you know, we have seasonal flu vaccines and it's

00:28:14.230 --> 00:28:15.650
a little bit more like that because you don't

00:28:15.650 --> 00:28:17.150
have one flu vaccine. You solve all the flu.

00:28:17.769 --> 00:28:23.430
It's changing. And so is. So getting back to

00:28:23.430 --> 00:28:26.670
the kind of the. personalization of cancer plans.

00:28:26.950 --> 00:28:29.329
There was a recent article in Frontiers Oncology

00:28:29.329 --> 00:28:32.170
in 2023, and it discussed the use of AI to actually

00:28:32.170 --> 00:28:34.609
help develop cancer -fighting drugs by identifying

00:28:34.609 --> 00:28:37.450
tumor biomarkers. I don't know if you're familiar

00:28:37.450 --> 00:28:40.410
with this study or this type of emerging technology,

00:28:40.549 --> 00:28:42.069
and do you think this is something that could

00:28:42.069 --> 00:28:45.210
be actually incorporated or used in conjunction

00:28:45.210 --> 00:28:48.250
with, you know, I guess this would kind of be

00:28:48.250 --> 00:28:50.309
past what me or I would do because that's...

00:28:50.490 --> 00:28:52.930
predictive ability, but for cancers, do you see

00:28:52.930 --> 00:28:55.950
AI helping to be able to treat tumors just by

00:28:55.950 --> 00:28:59.609
detecting biomarkers? Yeah. I mean, this isn't

00:28:59.609 --> 00:29:01.490
my area, but there's like a huge amount of excitable

00:29:01.490 --> 00:29:06.089
topic in that space. The drug discovery pipeline

00:29:06.089 --> 00:29:07.849
is long and complicated. There's a lot of steps

00:29:07.849 --> 00:29:10.509
within it. And any individual, one of those links,

00:29:10.630 --> 00:29:13.450
you can imagine ways for AI to help. Like if

00:29:13.450 --> 00:29:17.619
you have a target you're trying to hit. finding

00:29:17.619 --> 00:29:19.339
the molecule that will help you hit that target

00:29:19.339 --> 00:29:21.119
is more efficient than ever because you can build

00:29:21.119 --> 00:29:22.660
models, pick the mapping. There's many things

00:29:22.660 --> 00:29:24.880
like from the range of like better ways to do

00:29:24.880 --> 00:29:26.859
docking, how do these two things actually bind

00:29:26.859 --> 00:29:29.160
together or not? Given one thing, how will it

00:29:29.160 --> 00:29:33.019
fold in the first place? Like there's every part

00:29:33.019 --> 00:29:34.759
of that kind of spectrum from like target identification,

00:29:35.259 --> 00:29:39.079
which looks more like SysBioE down to given a

00:29:39.079 --> 00:29:40.839
target, what do you do about it? It's getting

00:29:40.839 --> 00:29:42.599
more efficient. Now the fundamental challenges

00:29:42.599 --> 00:29:45.500
of like... I found a target that maybe works

00:29:45.500 --> 00:29:48.339
on a mouse, maybe doesn't work on a person. Remains

00:29:48.339 --> 00:29:50.460
the same, but I'm sure people are working on

00:29:50.460 --> 00:29:52.859
that too. So it's not like we've solved drugs.

00:29:54.539 --> 00:29:56.660
Drugs are hard for the fundamental reason that

00:29:56.660 --> 00:29:59.740
we're not doing random trials on people. I mean,

00:29:59.740 --> 00:30:01.700
if you find something like, we're getting better

00:30:01.700 --> 00:30:03.640
at solving in virtual bottle systems faster,

00:30:03.819 --> 00:30:05.700
which is amazing. We need importance, and we've

00:30:05.700 --> 00:30:07.539
got people working on it. And I have lots of

00:30:07.539 --> 00:30:08.819
friends working in different areas of this problem

00:30:08.819 --> 00:30:09.720
that I think are going to be super impactful.

00:30:10.900 --> 00:30:13.910
But the... The drug world is like a very complex

00:30:13.910 --> 00:30:15.470
one. And like, let's say we get better at solving

00:30:15.470 --> 00:30:17.789
this petri dish and this one and this model system

00:30:17.789 --> 00:30:19.809
and whatever. It's good because then we can cross

00:30:19.809 --> 00:30:21.829
the bridges faster. But the fundamental challenge

00:30:21.829 --> 00:30:25.509
of like drugs in Ontario remains true. But this

00:30:25.509 --> 00:30:31.170
is not, this is the, I'm a, I'm a, passenger

00:30:31.170 --> 00:30:34.690
is the wrong word. I'm someone that's like ancillary

00:30:34.690 --> 00:30:37.109
to the field. I do some work in the space, some

00:30:37.109 --> 00:30:39.230
pharmas. And I think it's a fascinating space

00:30:39.230 --> 00:30:40.549
in structure. I think it's a super exciting time

00:30:40.549 --> 00:30:43.369
to be in it. But it's not my main bread and butter.

00:30:43.910 --> 00:30:47.630
No, yeah. For me, I was just more curious about

00:30:47.630 --> 00:30:52.670
as far as the applications of AI. AI, for me,

00:30:52.769 --> 00:30:55.630
it can be pretty scary for some people. There's

00:30:55.630 --> 00:30:56.769
some people that predict it's going to be the

00:30:56.769 --> 00:30:59.569
end of humanity. What I will say is that if you

00:30:59.569 --> 00:31:01.990
look at the past century, with regard to healthcare

00:31:01.990 --> 00:31:04.069
anyway, there have been three public health measures

00:31:04.069 --> 00:31:06.390
that have really improved just overall lifespan

00:31:06.390 --> 00:31:10.259
and mortality. There was water sanitization,

00:31:10.480 --> 00:31:12.779
vaccine development, and then the invention of

00:31:12.779 --> 00:31:16.140
antibiotics. Do you think that AI might be instrumental

00:31:16.140 --> 00:31:18.140
and maybe even the next fourth, you know, kind

00:31:18.140 --> 00:31:20.279
of could be the fourth kind of public health

00:31:20.279 --> 00:31:22.279
measure when it's implemented correctly to help

00:31:22.279 --> 00:31:24.900
save lives and just kind of preserve lives in

00:31:24.900 --> 00:31:31.140
general? I think it's not guaranteed, but it

00:31:31.140 --> 00:31:33.680
could be if you play our part right. I think

00:31:33.680 --> 00:31:35.279
there's things like both on the innovation level

00:31:35.279 --> 00:31:36.619
and the systems level to make that possible.

00:31:37.369 --> 00:31:40.750
i mean ai is and like like a vaccine is a particular

00:31:40.750 --> 00:31:44.809
thing uh a antibiotic is a particular thing and

00:31:44.809 --> 00:31:46.910
it will work for a particular set of things ai

00:31:46.910 --> 00:31:50.089
is a broader empowering technology and so like

00:31:50.089 --> 00:31:51.910
it should help us shorten the gap of finding

00:31:51.910 --> 00:31:53.710
all the next steps and the right interventions

00:31:53.710 --> 00:31:56.890
from decades between to like much much faster

00:31:56.890 --> 00:31:59.849
so she gave us like a thousand moonshot ideas

00:31:59.849 --> 00:32:03.319
rather than just one and like that qualitative

00:32:03.319 --> 00:32:05.279
change in speed is what we're looking for, but

00:32:05.279 --> 00:32:09.900
it implies the right investments. And we're not

00:32:09.900 --> 00:32:13.240
fully there. How much investment is there really

00:32:13.240 --> 00:32:17.099
in the health AI space at scale? It's not that

00:32:17.099 --> 00:32:19.740
big. There's a lot of non -profits that help

00:32:19.740 --> 00:32:25.079
support AI bio. There's the Ark Institute. There's

00:32:25.079 --> 00:32:27.500
many things of this kind that have very big,

00:32:27.619 --> 00:32:30.180
hundreds of GPUs coming together to better understand

00:32:30.180 --> 00:32:33.859
the genome. similar kind of very big, expensive,

00:32:33.880 --> 00:32:36.640
bold plays in the more clinical delivery space

00:32:36.640 --> 00:32:42.299
are hard to find. And really, we're still in

00:32:42.299 --> 00:32:43.819
a space that's at its infancy that has relatively

00:32:43.819 --> 00:32:46.880
little investment, and there's not gigantic Google

00:32:46.880 --> 00:32:50.299
-level financial players out of it. There's a

00:32:50.299 --> 00:32:52.619
lot of investment in the biotech sector, partially

00:32:52.619 --> 00:32:55.559
because we know farmers can do well, there's

00:32:55.559 --> 00:32:59.970
a proven model, the tracks is there. I think

00:32:59.970 --> 00:33:01.910
in the health AI space, we're still in a formative

00:33:01.910 --> 00:33:03.750
stage to build the first major success stories

00:33:03.750 --> 00:33:06.170
that can improve the case and catalyze that investment.

00:33:06.450 --> 00:33:09.390
And really right now, we're still in early days

00:33:09.390 --> 00:33:12.089
of this field. We have it in a society as well

00:33:12.089 --> 00:33:14.089
to be small. I mean, look at the mammogram case.

00:33:14.150 --> 00:33:16.390
The biggest models that are academic and public

00:33:16.390 --> 00:33:18.250
are like 100 ,000. There's 40 million a year.

00:33:19.190 --> 00:33:21.650
Do we have big national efforts to put these

00:33:21.650 --> 00:33:23.970
things together? No. Hospitals barely share.

00:33:24.769 --> 00:33:28.029
The compute that's available, I mean, pennies.

00:33:28.900 --> 00:33:30.440
We're playing small. We need to play bigger.

00:33:32.519 --> 00:33:35.539
There's plenty of room for growth then. As far

00:33:35.539 --> 00:33:38.720
as AI's role in medical professions, I'm a nurse.

00:33:38.759 --> 00:33:41.099
I work in a hospital. I've been doing it for

00:33:41.099 --> 00:33:45.660
over 20 years now. AI is slowly being introduced

00:33:45.660 --> 00:33:47.940
into certain aspects. I was wondering, I had

00:33:47.940 --> 00:33:50.019
a few questions regarding the possibility of

00:33:50.019 --> 00:33:53.220
AI's role just in future medical professions

00:33:53.220 --> 00:33:56.690
and just possibly your opinion. So telehealth

00:33:56.690 --> 00:33:58.869
is used widely now, especially when COVID started,

00:33:59.009 --> 00:34:02.369
it really became more popular. And it's usually,

00:34:02.410 --> 00:34:04.329
it's pretty well accepted just for preliminary

00:34:04.329 --> 00:34:07.650
diagnosis of non -life -threatening conditions.

00:34:08.769 --> 00:34:11.530
It still requires appointments. Appointments

00:34:11.530 --> 00:34:13.269
cost similar to, you know, in -clinic visits.

00:34:13.750 --> 00:34:16.949
Do you say that AI could someday just be as effective

00:34:16.949 --> 00:34:19.550
as a telehealth visit by, and this is again for

00:34:19.550 --> 00:34:21.329
non -life -threatening, you're calling because

00:34:21.329 --> 00:34:23.570
you have a cough and sneezing or you have a rash.

00:34:24.280 --> 00:34:26.599
Or you could theoretically be talking to artificial

00:34:26.599 --> 00:34:28.739
intelligence, show them images, give them your

00:34:28.739 --> 00:34:30.599
symptoms, and they could help provide a diagnosis?

00:34:33.480 --> 00:34:37.719
Yeah, I think so. I mean, I'm sure plenty of

00:34:37.719 --> 00:34:39.940
people do this already. Now, does it work or

00:34:39.940 --> 00:34:43.219
not? Is it actually effective? It's TBD. But

00:34:43.219 --> 00:34:45.659
people already do this kind of thing with the

00:34:45.659 --> 00:34:48.119
general purpose VLMs you have from the field

00:34:48.119 --> 00:34:51.920
of chiropractic PT now. I think to... catalyze

00:34:51.920 --> 00:34:54.699
the space and make it actually effective you

00:34:54.699 --> 00:34:57.340
need to have like the same way that like you

00:34:57.340 --> 00:35:00.039
know if you get some very bad advice that's harmful

00:35:00.039 --> 00:35:01.579
from a clinician there's like you know medical

00:35:01.579 --> 00:35:03.139
legal liability and there's like a lot of stuff

00:35:03.139 --> 00:35:04.400
there's a whole system around here to kind of

00:35:04.400 --> 00:35:07.139
like how do you do qc as a society of the quality

00:35:07.139 --> 00:35:09.320
of medical care my preference isn't gonna get

00:35:09.320 --> 00:35:12.039
better but there is a system uh and i think uh

00:35:13.230 --> 00:35:16.110
it is a matter of time until the systems become

00:35:16.110 --> 00:35:17.829
mature enough that people are willing to kind

00:35:17.829 --> 00:35:19.369
of put the money where their mouth is to kind

00:35:19.369 --> 00:35:23.570
of take a liability for errors and, uh, monetize

00:35:23.570 --> 00:35:24.929
it accordingly and do things to kind of make

00:35:24.929 --> 00:35:27.130
that kind of service available. I think the nature

00:35:27.130 --> 00:35:29.730
of the profession, like any profession will change.

00:35:29.909 --> 00:35:32.909
And I can tell like, you know, I code, I've experienced

00:35:32.909 --> 00:35:35.130
this in my training. What it's like for me to

00:35:35.130 --> 00:35:38.670
code now is so different than it was for me to

00:35:38.670 --> 00:35:41.489
code five years ago. It's like night and day.

00:35:41.909 --> 00:35:44.369
And like, And it would be hard. I mean, I could

00:35:44.369 --> 00:35:46.510
do it. I could code the way I used to, but I

00:35:46.510 --> 00:35:48.230
feel like a barbarian, you know, it's like going

00:35:48.230 --> 00:35:50.849
back to less efficient, right? Yeah. Yeah. I

00:35:50.849 --> 00:35:52.550
mean, like it may even make you think differently

00:35:52.550 --> 00:35:56.150
because like I'm more used to the AI models now.

00:35:57.329 --> 00:36:00.369
And so I expected to do particular things and

00:36:00.369 --> 00:36:01.889
I kind of like, and it means that my net throughput

00:36:01.889 --> 00:36:03.610
can go much bigger. And like my, what I think

00:36:03.610 --> 00:36:05.289
my leverage is as an individual person building

00:36:05.289 --> 00:36:08.110
is much bigger. And I think it is an open challenge

00:36:08.110 --> 00:36:11.909
of how do we do this in care? Because, like,

00:36:11.969 --> 00:36:13.469
the general situation you have in care is that

00:36:13.469 --> 00:36:16.769
we're always understaffed. Like, you tell me

00:36:16.769 --> 00:36:17.690
you want an appointment with anything, it takes

00:36:17.690 --> 00:36:18.690
forever. Like, I'm trying to get an appointment

00:36:18.690 --> 00:36:22.110
right now. It's getting tough, you know? And

00:36:22.110 --> 00:36:27.170
there is an opportunity, if you do well enough,

00:36:27.230 --> 00:36:30.989
to make the system serve people better. I think

00:36:30.989 --> 00:36:32.329
it's an exciting opportunity. But, again, it's

00:36:32.329 --> 00:36:33.329
definitely going to involve a lot of change.

00:36:33.469 --> 00:36:36.150
There's no way to avoid it. Yeah, I mean, kind

00:36:36.150 --> 00:36:37.670
of as a follow -up to that, like, I was wondering,

00:36:37.730 --> 00:36:40.139
like, Just how much, like, if you get to that

00:36:40.139 --> 00:36:42.099
point where it's, you know, relatively effective

00:36:42.099 --> 00:36:44.219
and efficient, it could definitely reduce, like,

00:36:44.239 --> 00:36:46.000
wait times in emergency departments. People wouldn't

00:36:46.000 --> 00:36:47.360
be going to the emergency department because

00:36:47.360 --> 00:36:49.119
they felt like they were sick or had the flu

00:36:49.119 --> 00:36:51.559
or something or it wasn't, you know, overly pressing.

00:36:53.079 --> 00:36:55.260
The other thing, as you kind of brought up, like,

00:36:55.320 --> 00:36:57.380
you know, the insurance industry has always got

00:36:57.380 --> 00:36:59.400
their fingers in everything. And it's like, as

00:36:59.400 --> 00:37:01.980
far as the costs. saving measure like at what

00:37:01.980 --> 00:37:03.760
point would it become more or less cost effective

00:37:03.760 --> 00:37:06.440
to do this i think that all depends on you know

00:37:06.440 --> 00:37:09.320
like i said who the regulatory issues all those

00:37:09.320 --> 00:37:13.619
things um in hospitals in general oh sorry go

00:37:13.619 --> 00:37:15.699
right ahead yeah i was just saying that like

00:37:15.699 --> 00:37:18.219
uh can we make it technically possible to live

00:37:18.219 --> 00:37:21.920
in this kind of care for sure will we fail to

00:37:21.920 --> 00:37:24.539
deploy because of some weird nuance of our incentives

00:37:24.539 --> 00:37:27.789
and our insurance system Maybe. To me, those

00:37:27.789 --> 00:37:30.230
are different questions because the insurer does

00:37:30.230 --> 00:37:31.929
not necessarily have a great incentive to make

00:37:31.929 --> 00:37:35.409
care all the much cheaper because I usually think

00:37:35.409 --> 00:37:37.829
to help them increase the revenue is better for

00:37:37.829 --> 00:37:42.190
them. And they have the profit cap stuff. It's

00:37:42.190 --> 00:37:43.550
a quite complicated system. I think it's what

00:37:43.550 --> 00:37:48.110
I think. I feel like it's ours to lose. Can we

00:37:48.110 --> 00:37:50.110
make this possible? Yes. Is there the feasibility

00:37:50.110 --> 00:37:52.090
of better care? Across almost every sector, the

00:37:52.090 --> 00:37:55.289
answer is yes. It's like... Can we play the card

00:37:55.289 --> 00:37:57.230
right enough to make it actually feasible and

00:37:57.230 --> 00:37:58.809
cost effective within our system? Can we change

00:37:58.809 --> 00:38:01.050
the system to remove the necessary roadblocks?

00:38:01.289 --> 00:38:03.789
That to me is like the societal challenge because

00:38:03.789 --> 00:38:07.030
we totally mess it up. Oh, I'm sure we would.

00:38:07.150 --> 00:38:08.230
And there's a lot of people who wouldn't want

00:38:08.230 --> 00:38:09.889
that to happen in the first place because there's

00:38:09.889 --> 00:38:13.150
a lot of money to be lost with regard to that.

00:38:13.230 --> 00:38:16.809
So in hospitals, physicians who typically, you

00:38:16.809 --> 00:38:19.250
know, once they reach a certain stage, once they're

00:38:19.250 --> 00:38:20.849
past their residency, essentially, they start

00:38:20.849 --> 00:38:24.300
to make pretty decent money. But physicians,

00:38:24.380 --> 00:38:26.960
they diagnose, they prescribe treatments, they

00:38:26.960 --> 00:38:29.619
perform surgeries. They often spend little time

00:38:29.619 --> 00:38:32.199
with their patients in general. You might see

00:38:32.199 --> 00:38:33.980
a physician five to 10 minutes a day if you're

00:38:33.980 --> 00:38:37.420
actually in the hospital. As a cost -saving measure,

00:38:37.559 --> 00:38:40.739
could hospitals at some point use AI in some

00:38:40.739 --> 00:38:43.699
of these aspects in care as far as the diagnosing

00:38:43.699 --> 00:38:46.880
and prescribing treatments or even surgeries?

00:38:46.900 --> 00:38:48.719
I have a question about surgeries and robotic

00:38:48.719 --> 00:38:50.820
surgery as well, but I'll get to that in a second.

00:38:52.139 --> 00:38:54.980
Just as far as AI, could they replace physicians

00:38:54.980 --> 00:38:57.639
in a lot of these even smaller hospitals that

00:38:57.639 --> 00:38:59.239
don't have the staff or don't feel like they

00:38:59.239 --> 00:39:03.039
can afford the physicians with regard to prescribing

00:39:03.039 --> 00:39:05.139
treatments or even just doing the assessment?

00:39:07.739 --> 00:39:09.579
This will play out differently across different

00:39:09.579 --> 00:39:12.320
types of places. I think of the solution when

00:39:12.320 --> 00:39:14.380
you're like two hours away from the next hospital.

00:39:15.539 --> 00:39:17.699
will necessarily look different than it is if

00:39:17.699 --> 00:39:19.420
you're in the city and the hospital is like a

00:39:19.420 --> 00:39:23.559
15 minutes bus ride away. So that'll be a little

00:39:23.559 --> 00:39:25.719
bit diverse. I think we're already seeing some

00:39:25.719 --> 00:39:29.300
version of this. It's generally true people are

00:39:29.300 --> 00:39:30.900
burnt out. You see too many patients for too

00:39:30.900 --> 00:39:32.159
little time. You don't have time to think about

00:39:32.159 --> 00:39:34.480
them as much as you want to. It's very hard.

00:39:34.780 --> 00:39:36.360
And the AI products that are getting the most

00:39:36.360 --> 00:39:38.360
traction, stuff like Average that's doing passive

00:39:38.360 --> 00:39:40.420
summarization, they'll give you back more time.

00:39:41.079 --> 00:39:43.369
And so I think like... Will there be more and

00:39:43.369 --> 00:39:45.730
more of this where you can kind of delegate more

00:39:45.730 --> 00:39:48.769
and more responsibilities to more and more systems

00:39:48.769 --> 00:39:51.769
so you can kind of have control over what you

00:39:51.769 --> 00:39:53.690
think about and spend your intellectual power

00:39:53.690 --> 00:39:56.530
where it's most needed? I think that's true.

00:39:56.909 --> 00:40:00.190
And in places where you really can't get anyone,

00:40:00.449 --> 00:40:04.929
is that basic safety better than Googling? I

00:40:04.929 --> 00:40:07.789
would hope so. So I do think there's room to

00:40:07.789 --> 00:40:10.250
grow there, but I think the implementation model...

00:40:10.559 --> 00:40:13.000
will likely look very different in very rural

00:40:13.000 --> 00:40:15.519
places like where I'm at right now compared to

00:40:15.519 --> 00:40:21.059
SF. And I guess that could be really advantageous.

00:40:21.059 --> 00:40:23.940
I know that a lot of community hospitals, they

00:40:23.940 --> 00:40:28.780
will use telehealth for specialists. So in that

00:40:28.780 --> 00:40:31.480
same vein, if you're an hour or two hours away

00:40:31.480 --> 00:40:33.099
from a specialist and you need to talk to them

00:40:33.099 --> 00:40:35.980
right away, I think that's somewhere where AI

00:40:35.980 --> 00:40:38.519
might be able to come in and... at least aid

00:40:38.519 --> 00:40:40.099
the physicians that are there in helping to make

00:40:40.099 --> 00:40:42.219
a diagnosis or provide the correct treatment.

00:40:43.300 --> 00:40:45.860
The question I had with robotic surgery, so there

00:40:45.860 --> 00:40:48.559
was an article in January of 2024 in the Journal

00:40:48.559 --> 00:40:51.199
of Robotic Surgery, and it highlighted the potential

00:40:51.199 --> 00:40:55.559
of AI use in robotic surgery. I know that some

00:40:55.559 --> 00:40:57.780
places are using a little bit of AI right now,

00:40:57.840 --> 00:40:59.500
but do you think that there's a future where

00:40:59.500 --> 00:41:01.980
AI -guided robotic surgery could actually become

00:41:01.980 --> 00:41:05.389
a primary surgical tool in hospitals? With regard

00:41:05.389 --> 00:41:08.389
to providing just more accurate surgery, I know

00:41:08.389 --> 00:41:11.590
that robots have more dexterity than humans,

00:41:11.789 --> 00:41:14.670
smaller areas than humans. Do you see that as

00:41:14.670 --> 00:41:16.230
something that's a possibility in the future

00:41:16.230 --> 00:41:22.670
or that might happen? I mean, I know the surgical

00:41:22.670 --> 00:41:24.250
robot companies are making a lot of progress.

00:41:24.710 --> 00:41:26.809
I mean, I think there's different classes of

00:41:26.809 --> 00:41:30.449
improvements. One of them is, is it so good that

00:41:30.449 --> 00:41:33.750
it's fully autonomous? Or is it so good that

00:41:33.750 --> 00:41:35.030
it could make, because a bunch of people, like,

00:41:35.030 --> 00:41:36.250
okay, what makes a really great surgeon? They

00:41:36.250 --> 00:41:38.610
do a lot of surgeries. And some people are good

00:41:38.610 --> 00:41:40.730
at very specific neuro type of surgeries. And

00:41:40.730 --> 00:41:42.050
if you get it at a general place, it will not

00:41:42.050 --> 00:41:43.170
be very good because they don't do them that

00:41:43.170 --> 00:41:44.949
often. So you have much higher rates of complication.

00:41:46.170 --> 00:41:48.769
Can we upscale people at scale? There's a different

00:41:48.769 --> 00:41:50.630
version of this in radiology. There's a lot of

00:41:50.630 --> 00:41:52.969
general radiologists that read everything. Okay,

00:41:53.010 --> 00:41:54.550
well, how good are they then in recognizing the

00:41:54.550 --> 00:41:56.210
rarest type of cancer that occurs in one in 15

00:41:56.210 --> 00:41:58.889
,000 patients? Less compared to the one that's

00:41:58.889 --> 00:42:01.079
like specialized in just that. And I think there's

00:42:01.079 --> 00:42:02.980
a class of opportunities that to me is more like

00:42:02.980 --> 00:42:06.699
short -term feasible of how do you upskill a

00:42:06.699 --> 00:42:09.480
broad swath of people to operate at a higher

00:42:09.480 --> 00:42:13.179
quality bar, which is a different bar of performance

00:42:13.179 --> 00:42:16.800
than it is to say be fully autonomous. Like if

00:42:16.800 --> 00:42:18.500
you have a Tesla or one of these like self -driving

00:42:18.500 --> 00:42:21.659
car companies or whatever, adaptive cruise control

00:42:21.659 --> 00:42:25.019
is pretty useful. It's pretty great. It changes

00:42:25.019 --> 00:42:26.760
how I drive on my like, you know, eight hour

00:42:26.760 --> 00:42:29.320
journeys and whatever. and like the tesla autopilot

00:42:29.320 --> 00:42:33.840
version thing uh is like would i actually not

00:42:33.840 --> 00:42:35.820
pay attention at all no but it like makes life

00:42:35.820 --> 00:42:37.699
better and i think there's a lot of version of

00:42:37.699 --> 00:42:40.000
this across domains including radiology and surgery

00:42:40.000 --> 00:42:42.900
uh but it's you know it's a dangerous thing to

00:42:42.900 --> 00:42:44.860
predict any particular time of any of these things

00:42:44.860 --> 00:42:46.719
you know science takes what it takes take the

00:42:46.719 --> 00:42:49.760
time that it takes yeah not trying to put you

00:42:49.760 --> 00:42:52.260
in an awkward position uh answering any of these

00:42:52.260 --> 00:42:55.860
i just want to pick your brain I think like the

00:42:55.860 --> 00:42:57.900
robotics world generally has a harder time because

00:42:57.900 --> 00:43:00.260
you have less demonstration data. In radiology,

00:43:00.320 --> 00:43:03.079
we see more of the full world. We don't see all

00:43:03.079 --> 00:43:04.500
of it, but we see more of the context literally.

00:43:05.739 --> 00:43:09.000
Not everywhere records all the information and

00:43:09.000 --> 00:43:10.380
all the kind of dexterous motion. And so it's

00:43:10.380 --> 00:43:12.300
harder to, like you don't have as much data for

00:43:12.300 --> 00:43:14.460
these robots to build on. And it tends to be

00:43:14.460 --> 00:43:16.300
very conservative by design, right? Because it

00:43:16.300 --> 00:43:18.539
only takes the robot messing up a couple of times

00:43:18.539 --> 00:43:20.039
to get a lot of PR that can kind of shut down

00:43:20.039 --> 00:43:22.780
the company. And so they're necessarily conservative.

00:43:23.000 --> 00:43:25.159
And I think... But I think there's a lot of cool

00:43:25.159 --> 00:43:26.920
work happening in the control world. There's

00:43:26.920 --> 00:43:28.039
a lot of really cool companies being built around

00:43:28.039 --> 00:43:30.340
general robotics. And if they do well, I think

00:43:30.340 --> 00:43:33.019
it's quite possible you get richer capabilities

00:43:33.019 --> 00:43:37.400
in the medical side. So kind of getting back

00:43:37.400 --> 00:43:40.159
to the question I had prior to this, just with

00:43:40.159 --> 00:43:42.360
physicians as well as what they do. And I kind

00:43:42.360 --> 00:43:44.199
of brought up the fact they tend not to spend

00:43:44.199 --> 00:43:46.619
a lot of time with the patient themselves. They

00:43:46.619 --> 00:43:47.980
might be five to 10 minutes a day. They have

00:43:47.980 --> 00:43:50.400
a lot of patients to see. Conversely, nurses,

00:43:50.579 --> 00:43:53.460
which is the role that I do. The orders the physicians

00:43:53.460 --> 00:43:55.300
write, the treatments, we're the ones who actually

00:43:55.300 --> 00:43:58.139
perform those. We provide monitoring of the patients.

00:43:58.300 --> 00:44:01.440
We're often there for emotional support. Nursing

00:44:01.440 --> 00:44:03.420
itself is just kind of hallmarked by compassion

00:44:03.420 --> 00:44:06.860
and empathy. Do you see a day where AI, say,

00:44:06.960 --> 00:44:09.639
becomes autonomous and you have AI performing

00:44:09.639 --> 00:44:15.000
the tasks of humans and you have AI -driven robots

00:44:15.000 --> 00:44:17.599
essentially performing care? Do you ever see

00:44:17.599 --> 00:44:20.460
a day where... they could reach the kind of emotional,

00:44:20.639 --> 00:44:23.219
you know, those characteristics of empathy or

00:44:23.219 --> 00:44:26.079
sympathy. And beyond that, do you think people

00:44:26.079 --> 00:44:28.019
would actually be receptive to being taken care

00:44:28.019 --> 00:44:30.780
of by, you know, fully autonomous robot nurse?

00:44:32.400 --> 00:44:37.360
I think, like, couldn't LLM write a better eye

00:44:37.360 --> 00:44:41.059
chart message than a person? Probably, because

00:44:41.059 --> 00:44:43.880
people are busy. And the LLM could write you

00:44:43.880 --> 00:44:46.949
a big paragraph. explaining all the details,

00:44:47.210 --> 00:44:48.489
and, like, there's very compassionate. And when

00:44:48.489 --> 00:44:50.190
people are rushed, and, like, when you're, I

00:44:50.190 --> 00:44:52.110
don't know about you, but, like, me at my most

00:44:52.110 --> 00:44:55.449
stressed is not my most elaborate to well -thought

00:44:55.449 --> 00:44:57.110
-out email that shows a lot of compassion and

00:44:57.110 --> 00:45:00.510
connection. When it comes to the point of seeing

00:45:00.510 --> 00:45:02.869
the person in the bedside, I have a hard time

00:45:02.869 --> 00:45:06.670
imagining that something kind of hits, like,

00:45:06.730 --> 00:45:11.510
this kind of weird pseudo -synthetic thing. Like,

00:45:11.510 --> 00:45:12.909
I don't know, like, a metal box that's trying

00:45:12.909 --> 00:45:14.389
to look human would be more creepy than just

00:45:14.389 --> 00:45:16.860
a metal box. And there's only so much that can

00:45:16.860 --> 00:45:20.500
comfort you. But over phone, I think there's

00:45:20.500 --> 00:45:22.579
a lot of... It is already true. There's already

00:45:22.579 --> 00:45:24.340
been studies showing this, where people rate

00:45:24.340 --> 00:45:29.539
LLM -generated reports back as more compassionate

00:45:29.539 --> 00:45:32.480
and more patient than the ones written... There's

00:45:32.480 --> 00:45:34.880
been anti -evidence here and there of that, which

00:45:34.880 --> 00:45:37.460
is not surprising. If you have five minutes versus

00:45:37.460 --> 00:45:42.000
infinite GPU flops, you can write poetry. It's

00:45:42.000 --> 00:45:45.130
not that hard. But the actual in -person thing,

00:45:45.250 --> 00:45:47.150
like holding someone's hand, I think there's

00:45:47.150 --> 00:45:50.769
a deeply human thing there. Which is good to

00:45:50.769 --> 00:45:54.429
know. With regard to that, so automation, I think

00:45:54.429 --> 00:45:55.789
a lot of people are kind of fearful of automation.

00:45:57.449 --> 00:46:00.269
It's something that is becoming just more and

00:46:00.269 --> 00:46:02.210
more prevalent. I think AI is getting better

00:46:02.210 --> 00:46:03.969
at kind of helping automation become more like

00:46:03.969 --> 00:46:08.289
humans. Do you ever think that... there'll be

00:46:08.289 --> 00:46:10.489
a day where, you know, most of the medical profession

00:46:10.489 --> 00:46:13.670
is completely just run by automation. Do you

00:46:13.670 --> 00:46:14.969
think that, I know you just said you can never

00:46:14.969 --> 00:46:17.369
really take the human component out, but do you

00:46:17.369 --> 00:46:19.969
think that humans will kind of be reduced in

00:46:19.969 --> 00:46:21.889
their capacity and what they do or have to do?

00:46:21.989 --> 00:46:24.090
Or could they even, you know, that could be beneficial

00:46:24.090 --> 00:46:25.750
at some point because you're not doing the tasks

00:46:25.750 --> 00:46:28.889
that are taking up time away from more direct

00:46:28.889 --> 00:46:31.289
patient care or kind of providing better patient

00:46:31.289 --> 00:46:37.659
care. I, I, The way I think of it is that we

00:46:37.659 --> 00:46:39.840
don't have nearly enough humans to care for people

00:46:39.840 --> 00:46:41.800
at the scale that we want. Like, basically, like,

00:46:41.840 --> 00:46:44.119
in supply and demand, the problem is we don't

00:46:44.119 --> 00:46:46.019
have nearly enough supply. The demand is way,

00:46:46.079 --> 00:46:48.579
way higher. And so we end up with a product that

00:46:48.579 --> 00:46:50.300
we give to patients that is much worse than it

00:46:50.300 --> 00:46:53.800
could be, both in terms of, like, the user experience

00:46:53.800 --> 00:46:55.599
of it and, like, sitting in a waiting room forever

00:46:55.599 --> 00:46:57.219
and the apartment never opens the thing it is.

00:46:57.260 --> 00:47:00.260
Like, it's not the best commercial experience,

00:47:00.340 --> 00:47:06.000
let's just say, on average. I think AI ideally

00:47:06.000 --> 00:47:08.239
will give us a lot more leverage to have the

00:47:08.239 --> 00:47:10.440
existing supply, which is hard, more reasonably

00:47:10.440 --> 00:47:12.639
meets demand and have people that are very skilled

00:47:12.639 --> 00:47:15.539
be able to serve way more people and play where

00:47:15.539 --> 00:47:18.960
they have highest leverage. Now, it's true in

00:47:18.960 --> 00:47:20.940
theory that you could have such good tooling

00:47:20.940 --> 00:47:22.639
that maybe you need less than existing supply.

00:47:23.400 --> 00:47:27.780
I think we're just so far from existing supply.

00:47:29.260 --> 00:47:31.239
It's not like you get an appointment for a specialist

00:47:31.239 --> 00:47:35.110
same day. It's not in that world. We're just

00:47:35.110 --> 00:47:38.469
not. People drive hours and you turn around time

00:47:38.469 --> 00:47:41.949
for a callback, maybe several weeks. We're in

00:47:41.949 --> 00:47:44.329
a very slow world. I think there's not enough

00:47:44.329 --> 00:47:47.989
of almost every specialized profession. There's

00:47:47.989 --> 00:47:49.809
always, how do you get more nurses in the profession?

00:47:49.949 --> 00:47:50.889
How do you get more radiologists? How do you

00:47:50.889 --> 00:47:53.130
get this? I think for the actual demand of people

00:47:53.130 --> 00:47:55.190
wanting to take on these professions that are

00:47:55.190 --> 00:47:58.809
hard and very normal, we can help them be able

00:47:58.809 --> 00:48:01.550
to be more effective. It's a positive view because

00:48:01.550 --> 00:48:03.429
how do you empower people to do more and better?

00:48:04.010 --> 00:48:05.809
But like, could we mess up and make the experience

00:48:05.809 --> 00:48:07.869
much worse and have it even more grinding? And

00:48:07.869 --> 00:48:10.849
for sure, it's also possible. And, you know,

00:48:10.869 --> 00:48:13.329
build it a particular way. It's going to be a

00:48:13.329 --> 00:48:18.409
spicy time. So recently, there's a congressional

00:48:18.409 --> 00:48:20.989
representative named David Schweikert. He introduced

00:48:20.989 --> 00:48:24.389
a bill called H .R. 238. And this proposed that

00:48:24.389 --> 00:48:26.690
AI could actually prescribe medication under

00:48:26.690 --> 00:48:31.559
FDA and state approval. argument towards this

00:48:31.559 --> 00:48:33.860
is that AI could actually help prevent medication

00:48:33.860 --> 00:48:36.699
errors, it improves efficiency, and it can help

00:48:36.699 --> 00:48:39.219
personalize treatment. Do you think that's a

00:48:39.219 --> 00:48:41.019
technology that's actually currently available?

00:48:41.280 --> 00:48:43.239
And do you think it would be capable? Do you

00:48:43.239 --> 00:48:45.519
think that performing that task as far as prescribing

00:48:45.519 --> 00:48:49.780
medication based on, say, a set of lab data or

00:48:49.780 --> 00:48:52.679
an assessment or some other information that's

00:48:52.679 --> 00:48:55.699
fed into a model? I mean, I think if you take

00:48:55.699 --> 00:48:58.079
the word AI out of this, it would make sense.

00:48:58.699 --> 00:49:01.630
Let's say you have a test you have a bacterial

00:49:01.630 --> 00:49:04.610
infection give me a box of antibiotics like why

00:49:04.610 --> 00:49:08.150
do you have to sign an extra form you know like

00:49:08.150 --> 00:49:11.489
uh these things of like it's roads people do

00:49:11.489 --> 00:49:13.530
it over and over again you see a thing you do

00:49:13.530 --> 00:49:16.010
the thing it's not like a fancy ai model you

00:49:16.010 --> 00:49:17.809
have this kind of infection here's what i normally

00:49:17.809 --> 00:49:24.050
give you for it just get out of my way uh makes

00:49:24.050 --> 00:49:29.590
sense you know like uh and So, like, I think

00:49:29.590 --> 00:49:31.650
there's probably low -hanging fruit of that to

00:49:31.650 --> 00:49:33.969
streamline workflows in a way that feels obvious

00:49:33.969 --> 00:49:37.750
and low -risk. Now there's more complicated scenarios

00:49:37.750 --> 00:49:39.829
of, like, you know, you're an oncologist and

00:49:39.829 --> 00:49:41.150
you're figuring out which kind of chemo regimen

00:49:41.150 --> 00:49:42.989
to do and you have all these toxicities and,

00:49:43.010 --> 00:49:46.349
like, is palica going up? Is it because it's

00:49:46.349 --> 00:49:48.030
responding or not responding? These kind of very

00:49:48.030 --> 00:49:51.969
subtle things. It's quite possible that we'll

00:49:51.969 --> 00:49:55.019
kind of create more information. and make an

00:49:55.019 --> 00:49:56.920
easier decision than was possible before, something

00:49:56.920 --> 00:49:58.219
that we're working on, I think is going to be

00:49:58.219 --> 00:50:04.519
very exciting. But in the near -term future to

00:50:04.519 --> 00:50:06.679
mid -term, I don't see that kind of thing being

00:50:06.679 --> 00:50:09.820
fully automated, unless it's a place where the

00:50:09.820 --> 00:50:12.199
standard is so bad that that automation is still

00:50:12.199 --> 00:50:14.219
an improvement. That's the thing about healthcare.

00:50:14.340 --> 00:50:16.920
It's so diverse, and you have really high resource

00:50:16.920 --> 00:50:18.539
settings and such low resource settings. So the

00:50:18.539 --> 00:50:20.420
way I think about automation, fundamentally it

00:50:20.420 --> 00:50:22.179
depends. If you don't have access to an oncologist,

00:50:22.619 --> 00:50:24.099
then yes, this thing that's maybe not as good

00:50:24.099 --> 00:50:27.980
as one is better than none. But if you have one,

00:50:28.059 --> 00:50:29.099
then you should work to help make them better.

00:50:29.360 --> 00:50:31.840
And you shouldn't deliver worse care. So how

00:50:31.840 --> 00:50:33.820
do you help an individual person? And especially

00:50:33.820 --> 00:50:35.119
in the global health scenario, when you have

00:50:35.119 --> 00:50:39.179
one for every X thousand cancer patients, then

00:50:39.179 --> 00:50:41.599
the way to think about the solution has to change.

00:50:44.639 --> 00:50:46.960
So I think public perception, too, is something

00:50:46.960 --> 00:50:50.000
that... there's ethical considerations and public

00:50:50.000 --> 00:50:51.820
perception when it comes to artificial intelligence

00:50:51.820 --> 00:50:55.820
in healthcare. There was a recent JAMA article.

00:50:55.940 --> 00:50:58.039
It was actually a commentary and there was a

00:50:58.039 --> 00:51:00.440
survey and they surveyed over 2000 people. So

00:51:00.440 --> 00:51:02.719
not a huge sample size, but there was a lot of

00:51:02.719 --> 00:51:04.840
mixed trust in artificial intelligence role in

00:51:04.840 --> 00:51:08.119
healthcare. The scores were, you could score

00:51:08.119 --> 00:51:11.619
zero to 12 with zero being like no confidence

00:51:11.619 --> 00:51:14.420
and 12 being confident in artificial intelligence

00:51:14.420 --> 00:51:16.519
role in healthcare. And the mean score was about

00:51:16.519 --> 00:51:19.789
5 .8. So people are kind of split as far as confidence

00:51:19.789 --> 00:51:21.570
level for artificial intelligence to kind of

00:51:21.570 --> 00:51:24.429
be incorporated in healthcare. Do you think there

00:51:24.429 --> 00:51:25.989
are measures that can be taken to just improve

00:51:25.989 --> 00:51:28.949
public trust or acceptance of AI? Do you think

00:51:28.949 --> 00:51:30.469
that's something that is going to take a long

00:51:30.469 --> 00:51:34.550
time to do? One, I think it's a funny scale.

00:51:34.610 --> 00:51:36.469
I've never heard of a zero to 12. Zero to 12.

00:51:36.670 --> 00:51:41.750
Yeah. I don't know. But I think I don't take

00:51:41.750 --> 00:51:44.610
those numbers too seriously. I think people in

00:51:44.610 --> 00:51:47.440
general. are very bad at estimating how they

00:51:47.440 --> 00:51:48.920
will feel about technology that's not there.

00:51:49.900 --> 00:51:52.380
So you ask people in the abstract, would you

00:51:52.380 --> 00:51:55.440
use a language model for this before ChatGPT

00:51:55.440 --> 00:51:58.119
came out? They were terribly predictive of how

00:51:58.119 --> 00:52:01.539
they use it today. Fundamentally, what builds

00:52:01.539 --> 00:52:06.840
trust? Efficacy. If you ask me, I just think

00:52:06.840 --> 00:52:09.360
if the tools really work, building trust is just

00:52:09.360 --> 00:52:10.800
a matter of actually delivering results that

00:52:10.800 --> 00:52:13.110
are actually very good. And like fundamentally

00:52:13.110 --> 00:52:15.510
user, like the experience of using the tool and

00:52:15.510 --> 00:52:17.170
how well it works is what drives it. And if someone

00:52:17.170 --> 00:52:19.969
could tell you, I felt this in my own work. People

00:52:19.969 --> 00:52:22.690
tell me like, oh, I don't know if we can trust

00:52:22.690 --> 00:52:24.150
what it means to have like an A algorithm do

00:52:24.150 --> 00:52:26.510
cancer risk. I want an explanation, blah, blah,

00:52:26.510 --> 00:52:29.809
blah. Hospital validation 10 later, no one asked

00:52:29.809 --> 00:52:33.489
me that question anymore. Because it works. And

00:52:33.489 --> 00:52:35.150
like, I think it's many things are the same way.

00:52:35.210 --> 00:52:37.489
Like many things are scary in the abstract, but

00:52:37.489 --> 00:52:39.989
in practice, if they're useful and you like them.

00:52:41.170 --> 00:52:42.769
then you're useful and you like them. If they're

00:52:42.769 --> 00:52:45.170
not useful, nothing will make you want to use

00:52:45.170 --> 00:52:47.869
it. I think just the utility, if it's not useful,

00:52:47.989 --> 00:52:49.130
people are just going to stop using it anyway.

00:52:49.409 --> 00:52:52.789
It'll probably die off, I would assume. Yeah,

00:52:52.929 --> 00:52:56.090
I think it's hard to have precise conversations

00:52:56.090 --> 00:53:00.409
on this when the tool is abstract. Kind of following

00:53:00.409 --> 00:53:03.239
up with the ethical considerations of it. The

00:53:03.239 --> 00:53:04.579
studies you've done, like I said, you had one

00:53:04.579 --> 00:53:08.159
that had over 150 ,000 scans. And it's kind of

00:53:08.159 --> 00:53:10.480
another crucial topic when talking about AI and

00:53:10.480 --> 00:53:13.619
healthcare. It's just patient privacy is huge.

00:53:14.099 --> 00:53:16.320
It's something that we take very seriously here

00:53:16.320 --> 00:53:19.760
in the United States. When you're using your

00:53:19.760 --> 00:53:22.400
AI models, what kind of safeguards are put into

00:53:22.400 --> 00:53:24.699
place to make sure that the information that

00:53:24.699 --> 00:53:26.619
you're putting into your algorithm to kind of

00:53:26.619 --> 00:53:31.110
come up with just research data? How is that

00:53:31.110 --> 00:53:33.289
safeguarded for people trying to access that

00:53:33.289 --> 00:53:36.730
information or hack into it or just the security

00:53:36.730 --> 00:53:40.110
leaks in general? Yeah. And these settings, I

00:53:40.110 --> 00:53:42.510
mean, like there is like all security is like

00:53:42.510 --> 00:53:44.469
an onion. There's like many layers of security

00:53:44.469 --> 00:53:47.269
that kind of like roll onto each other. So the

00:53:47.269 --> 00:53:49.889
kind of like the first layer is like we, it's

00:53:49.889 --> 00:53:52.550
not like we have like. servers that are sitting

00:53:52.550 --> 00:53:54.949
in an office that you just kind of walk into

00:53:54.949 --> 00:53:57.210
and plug in a flash drive. This is a secure data

00:53:57.210 --> 00:53:58.469
center that's managed essentially by the hospital.

00:53:58.630 --> 00:54:00.130
I couldn't walk in there myself even though I

00:54:00.130 --> 00:54:05.469
bought the server. So there is layers of physical

00:54:05.469 --> 00:54:08.429
and firewall constraints that's like we, even

00:54:08.429 --> 00:54:10.829
though the data we work on is we usually anonymize

00:54:10.829 --> 00:54:12.250
it before actually putting it into the research

00:54:12.250 --> 00:54:15.030
workflow. Even though it is anonymized, we protect

00:54:15.030 --> 00:54:17.929
it as if it wasn't anonymized. And so it lives

00:54:17.929 --> 00:54:20.090
in a place of PHI compliance, all the kind of

00:54:20.090 --> 00:54:21.650
standard IT security constraints accordingly.

00:54:22.210 --> 00:54:24.670
In order to touch any of it, first you have to

00:54:24.670 --> 00:54:27.550
have an IRB to say the risk of privacy leakage

00:54:27.550 --> 00:54:29.909
is low enough for the kind of game that you have.

00:54:30.010 --> 00:54:31.469
There's like minimal risk stuff, so you can actually

00:54:31.469 --> 00:54:32.789
do this kind of large scale stuff. So you have

00:54:32.789 --> 00:54:34.610
to get like ethical approval of the actual concept

00:54:34.610 --> 00:54:37.670
in itself. You have to go through regular trainings,

00:54:37.710 --> 00:54:40.190
both on the IT side and ethical training side.

00:54:40.269 --> 00:54:43.719
There's like a bunch of layers of stuff. And

00:54:43.719 --> 00:54:46.019
even if someone hacked into UCSF and got access

00:54:46.019 --> 00:54:49.739
to our servers, the data is not analyzed. Which

00:54:49.739 --> 00:54:52.699
is good. I know that here in the U .S., we have

00:54:52.699 --> 00:54:54.500
a lot of regulatory oversight. Like you said,

00:54:54.539 --> 00:54:56.460
you have all these layers. Do you know if other

00:54:56.460 --> 00:55:00.219
countries or healthcare systems are kind of protecting

00:55:00.219 --> 00:55:03.960
privacy for patients with regard to not just

00:55:03.960 --> 00:55:05.780
artificial intelligence, but kind of data gathering

00:55:05.780 --> 00:55:07.920
in general? Whereas the United States, are we

00:55:07.920 --> 00:55:10.019
a little more strict? I'm not even sure if you

00:55:10.019 --> 00:55:14.719
know. I'm not. I'm not sure. On the privacy side,

00:55:14.820 --> 00:55:18.400
I usually defer to the privacy offers at the

00:55:18.400 --> 00:55:21.599
hospital system to follow their recommendations

00:55:21.599 --> 00:55:24.320
and rules. I do AI stuff. I have a lot of expertise

00:55:24.320 --> 00:55:28.519
within that world and how the use case is. The

00:55:28.519 --> 00:55:31.440
security world is its own infinite rabbit hole

00:55:31.440 --> 00:55:33.980
of stuff. We have people that are good at it,

00:55:34.019 --> 00:55:35.840
so they can tell me what to do and we do it.

00:55:36.920 --> 00:55:41.360
I think in terms of the capability to build AI

00:55:41.360 --> 00:55:43.949
systems, in health, where you have these privacy

00:55:43.949 --> 00:55:47.969
constraints, there are trade -offs. In Europe,

00:55:48.010 --> 00:55:50.409
you have these national health systems, and in

00:55:50.409 --> 00:55:52.469
theory, you can actually build things across

00:55:52.469 --> 00:55:54.309
the country and get a scale. It's hard to get

00:55:54.309 --> 00:55:56.010
in the U .S. Because in the U .S., we're very

00:55:56.010 --> 00:55:59.190
fragmented. You can build something for all of

00:55:59.190 --> 00:56:01.510
Denmark, for all of the U .K., using the NHS

00:56:01.510 --> 00:56:05.949
data. And that's cool. The downside in that system,

00:56:06.070 --> 00:56:08.289
I'm finding through current collaborations, is

00:56:08.289 --> 00:56:11.730
that... the investments in research is often

00:56:11.730 --> 00:56:14.530
quite a bit smaller. And so like, you know, I

00:56:14.530 --> 00:56:16.070
have a collaborator, they're using the resources

00:56:16.070 --> 00:56:18.530
of Denmark, the GPUs they have access to is really

00:56:18.530 --> 00:56:20.730
quite limited and like quite a bit less than

00:56:20.730 --> 00:56:24.329
just my lab. And so like, you know, would you

00:56:24.329 --> 00:56:26.809
need to build these tools as well? You need data

00:56:26.809 --> 00:56:30.369
and compute to live in the same place. And it's

00:56:30.369 --> 00:56:33.469
hard to find that almost anywhere. And this is

00:56:33.469 --> 00:56:34.730
part of the kind of general conversation of like,

00:56:34.789 --> 00:56:38.039
we ender invest in health AI. So beyond the ethics

00:56:38.039 --> 00:56:40.880
component, too, you kind of just touched on this

00:56:40.880 --> 00:56:42.659
briefly, but there's always kind of that fear

00:56:42.659 --> 00:56:46.599
of litigation and liability. And it's almost

00:56:46.599 --> 00:56:48.460
a question of who would be liable. Like, say

00:56:48.460 --> 00:56:52.820
that, you know, you incorporate AI into the health

00:56:52.820 --> 00:56:54.880
care system and it's used and somebody gets injured.

00:56:55.840 --> 00:56:59.260
Who is actually and this is just kind of a question.

00:56:59.360 --> 00:57:00.880
I don't even know if there's an answer, a correct

00:57:00.880 --> 00:57:03.380
answer for this. Like who would be liable for

00:57:03.380 --> 00:57:06.539
it? Would it be? you know, the provider or the

00:57:06.539 --> 00:57:09.099
technology corporation or even just the programmer

00:57:09.099 --> 00:57:13.000
themselves who created the AI? And do you feel

00:57:13.000 --> 00:57:15.500
that the fear of liability is actually kind of

00:57:15.500 --> 00:57:18.380
a barrier to incorporating AI into more healthcare

00:57:18.380 --> 00:57:23.739
systems? I think it's one of those things like

00:57:23.739 --> 00:57:26.280
it totally depends on the workflow. Let's say,

00:57:26.320 --> 00:57:28.840
like right now, we already use AI in radiology.

00:57:30.000 --> 00:57:32.019
We use it for dictation, right? You speak into

00:57:32.019 --> 00:57:35.579
the microphone, it generates the report. Now,

00:57:35.579 --> 00:57:38.280
if you sign off on a report and the thing misread

00:57:38.280 --> 00:57:41.059
your words and like it put a not where there

00:57:41.059 --> 00:57:44.340
shouldn't be a not, right? And you sign the report,

00:57:44.539 --> 00:57:46.739
that's on you. Dictation software is the same

00:57:46.739 --> 00:57:48.260
way if you like, if you slip and you like hit

00:57:48.260 --> 00:57:50.300
the wrong part of the keyboard, that's on you.

00:57:50.320 --> 00:57:51.840
You sign the thing, you make sure it's there.

00:57:51.980 --> 00:57:53.380
And for many things where like you're signing

00:57:53.380 --> 00:57:57.139
off, it's clear what the responsibility is. Now,

00:57:57.199 --> 00:57:59.579
let's say you have an autonomous system that's

00:57:59.579 --> 00:58:01.679
taking the full decision and it is signing the

00:58:01.679 --> 00:58:04.320
report. then the maker of the system needs to

00:58:04.320 --> 00:58:07.400
be liable, right? Because like, and it should

00:58:07.400 --> 00:58:08.860
be priced accordingly and it should be able to

00:58:08.860 --> 00:58:10.000
put the money in their mouth as it had their

00:58:10.000 --> 00:58:13.219
own, you know, their own medical liability insurance,

00:58:13.579 --> 00:58:15.760
et cetera. So I think like for many cases, it's

00:58:15.760 --> 00:58:22.579
kind of straightforward. Like if, I think the

00:58:22.579 --> 00:58:27.880
ambiguous circumstances are harder to come up

00:58:27.880 --> 00:58:29.079
with. I mean, it's like, you know, let's say

00:58:29.079 --> 00:58:31.960
someone's doing surgery and the scalpel breaks.

00:58:33.589 --> 00:58:36.769
then who's liable in that circumstance? I imagine

00:58:36.769 --> 00:58:38.710
it depends. Like, did it break within warranty?

00:58:39.130 --> 00:58:40.769
You know, or did it break after, like, 15 years

00:58:40.769 --> 00:58:44.949
of not being maintained and cleaned? Right? And,

00:58:44.989 --> 00:58:47.190
like, I think there was, like, things like what

00:58:47.190 --> 00:58:48.590
happens if it breaks in the kind of circumstances,

00:58:48.630 --> 00:58:50.070
but I think a lot of settings that we're imagining

00:58:50.070 --> 00:58:53.090
now where it is a decision aid, like, you know,

00:58:53.110 --> 00:58:54.730
is Google liable if it doesn't give you the best

00:58:54.730 --> 00:58:55.969
search results for the question that you have?

00:58:56.750 --> 00:58:59.750
No. Should it be? Probably not. Like, you don't

00:58:59.750 --> 00:59:03.219
have to use it. But if you're saying that's a

00:59:03.219 --> 00:59:05.900
diagnosis, whatever Google's research is, that's

00:59:05.900 --> 00:59:07.480
a product they're selling you. They should be

00:59:07.480 --> 00:59:12.940
liable for that. So with regard to AI and machine

00:59:12.940 --> 00:59:16.280
learning and future research, I don't know if

00:59:16.280 --> 00:59:19.460
you can share. Are you concentrating everything

00:59:19.460 --> 00:59:21.659
with your lab right now, just working on Mirai?

00:59:21.880 --> 00:59:23.760
Or do you have other projects on the pipeline?

00:59:24.099 --> 00:59:26.059
Are there things you can or can't talk about

00:59:26.059 --> 00:59:29.639
with regard to future research? We're broadly

00:59:29.639 --> 00:59:32.579
focusing on going bigger and broader. So both

00:59:32.579 --> 00:59:35.380
Mira and Sybil are what I think of as highly

00:59:35.380 --> 00:59:39.760
specialized point solutions. We're trying to

00:59:39.760 --> 00:59:42.280
solve one particular problem, problems I deeply

00:59:42.280 --> 00:59:44.199
care about and ones I want to get better at solving.

00:59:46.039 --> 00:59:48.820
And to me, the future of the space is solving

00:59:48.820 --> 00:59:51.380
more and more problems at once and having a broader

00:59:51.380 --> 00:59:54.920
set of capabilities. One, because it makes you

00:59:54.920 --> 00:59:58.329
more nimble. being able to address many potential

00:59:58.329 --> 01:00:00.309
issues within the workflow and kind of like being

01:00:00.309 --> 01:00:02.349
creative and how many types of things can you

01:00:02.349 --> 01:00:04.750
help in at the same time. But two, because it

01:00:04.750 --> 01:00:06.869
also amortizes the cost of translation and like

01:00:06.869 --> 01:00:11.309
regulatory. So like I imagine that the future

01:00:11.309 --> 01:00:13.989
of this commercially is going to be like you

01:00:13.989 --> 01:00:17.170
go for filings for like 400 conditions at the

01:00:17.170 --> 01:00:19.670
same time. Because then it becomes more cost

01:00:19.670 --> 01:00:21.610
effective than doing them one at a time. And

01:00:21.610 --> 01:00:23.590
so like technically, like to give a more concrete

01:00:23.590 --> 01:00:26.329
answer. We're jointly modeling things across

01:00:26.329 --> 01:00:28.329
many cancers, many imaging modalities, trying

01:00:28.329 --> 01:00:30.150
to get a broader set of capabilities, including

01:00:30.150 --> 01:00:32.469
the ones we've already been solving. Because

01:00:32.469 --> 01:00:34.610
one, we think it'll make us better at solving

01:00:34.610 --> 01:00:36.309
every individual one of these things, because

01:00:36.309 --> 01:00:40.710
in every one of the things, by itself, you're

01:00:40.710 --> 01:00:43.570
too data constrained. Together, you have more

01:00:43.570 --> 01:00:44.909
data to play with, so there's more constraints

01:00:44.909 --> 01:00:47.130
on the model for how to learn better. And when

01:00:47.130 --> 01:00:48.750
it comes time to translate it out, translating

01:00:48.750 --> 01:00:50.610
one big model and going through one big filing

01:00:50.610 --> 01:00:53.849
is easier than going through a dozen small models.

01:00:55.219 --> 01:00:58.519
The technology is fantastic because I know that

01:00:58.519 --> 01:01:02.059
I think you had done another research paper that

01:01:02.059 --> 01:01:04.519
you had done or worked on with regard to pancreatic

01:01:04.519 --> 01:01:06.940
cancer as well. And there are certain cancers,

01:01:06.980 --> 01:01:08.719
pancreatic cancer being one of them, as well

01:01:08.719 --> 01:01:10.920
as stomach cancer, that a lot of times you don't

01:01:10.920 --> 01:01:14.420
find malignancies until it's far too late. Pancreatic

01:01:14.420 --> 01:01:16.019
cancer especially because it's got such a high

01:01:16.019 --> 01:01:19.500
mortality rate. Even if you catch it early, it's

01:01:19.500 --> 01:01:22.320
still just super deadly. But like stomach cancers

01:01:22.320 --> 01:01:24.019
as well, because people don't present with symptoms

01:01:24.019 --> 01:01:27.739
until you have that malignancy. And I don't know

01:01:27.739 --> 01:01:30.420
if the same, you know, with the research you've

01:01:30.420 --> 01:01:33.199
done on Sybil and with Mirai, if that same type

01:01:33.199 --> 01:01:36.079
of technology can be used as well to help detect

01:01:36.079 --> 01:01:40.760
these other cancers. Yeah, so there's the same

01:01:40.760 --> 01:01:42.719
type. We're working on two things for that problem.

01:01:44.340 --> 01:01:46.599
It's all the same flavor. You know, like once

01:01:46.599 --> 01:01:48.420
you have imaging, you know so much. You get to

01:01:48.420 --> 01:01:51.250
see. You get to atomically see what's going on.

01:01:51.309 --> 01:01:52.670
You should be able to get better risk models.

01:01:53.329 --> 01:01:55.349
And so there's one flavor of work that we're

01:01:55.349 --> 01:01:56.809
working on. Like if you have a lot of incidental

01:01:56.809 --> 01:01:59.110
imaging, you got an abdomen CT for an unrelated

01:01:59.110 --> 01:02:01.849
reason, kind of figure out your high risk now

01:02:01.849 --> 01:02:04.030
so that like for those people who happen to have

01:02:04.030 --> 01:02:05.130
that imaging, we can do something useful for

01:02:05.130 --> 01:02:07.489
them. The other bucket, when you don't have imaging,

01:02:07.670 --> 01:02:08.869
how do I give you a better baseline prediction?

01:02:09.090 --> 01:02:10.690
And that's all this like EHR modeling stuff,

01:02:10.889 --> 01:02:12.849
which we already talked about before, has to

01:02:12.849 --> 01:02:14.590
always be evolving because the EHR is like this

01:02:14.590 --> 01:02:18.039
dynamic beast. And the future... I think it's

01:02:18.039 --> 01:02:20.420
both. We should model everything. I think there's

01:02:20.420 --> 01:02:24.099
been a lot of efforts that are more comp bio

01:02:24.099 --> 01:02:25.460
plays where you say, like, I'm going to invent

01:02:25.460 --> 01:02:28.139
a new panel, see a new blood test, maybe I'll

01:02:28.139 --> 01:02:30.300
look for circular tumor DNA or something else.

01:02:31.079 --> 01:02:34.260
And those are great efforts, and I think they

01:02:34.260 --> 01:02:37.619
can be complementary. But the type of thing I'm

01:02:37.619 --> 01:02:39.019
most interested in is, like, what can we do if

01:02:39.019 --> 01:02:41.360
it's already widely collected? Because there

01:02:41.360 --> 01:02:43.980
you have scale. Then it's an AI play. if you're

01:02:43.980 --> 01:02:45.860
doing a new blood test and you have a new kind

01:02:45.860 --> 01:02:47.239
of panel, you're going to read these four things

01:02:47.239 --> 01:02:49.179
out or these 10 things out. You're going to have

01:02:49.179 --> 01:02:51.400
maybe across as many years, 100 samples or 300

01:02:51.400 --> 01:02:55.960
samples. It's too small. I mean, it can be really

01:02:55.960 --> 01:02:58.420
valuable and that's great, but it's too small

01:02:58.420 --> 01:03:00.420
for my skill set and what the top stuff in my

01:03:00.420 --> 01:03:02.500
lab does to make the big difference. So we're

01:03:02.500 --> 01:03:04.440
very much focused like where can we operate at

01:03:04.440 --> 01:03:06.619
scale where the type of technologies that we're

01:03:06.619 --> 01:03:08.199
really good at building can make the most difference.

01:03:08.599 --> 01:03:10.599
So it leads you to like imaging and EHR stuff

01:03:10.599 --> 01:03:12.679
that we've been collecting for decades. less

01:03:12.679 --> 01:03:14.079
than the kind of like new novel experimental

01:03:14.079 --> 01:03:18.059
platform that is by its nature new and small

01:03:18.059 --> 01:03:20.860
and limited adoption. There is room to make AI

01:03:20.860 --> 01:03:22.519
innovations there, but it's like a different

01:03:22.519 --> 01:03:25.380
flavor of work and it's quite a bit harder. How

01:03:25.380 --> 01:03:27.000
hard has it been for you when you're kind of

01:03:27.000 --> 01:03:29.699
procuring data just to get, because you needed

01:03:29.699 --> 01:03:33.329
so much data for these models? have you had difficulty

01:03:33.329 --> 01:03:36.989
finding uh just facilities or just medical rec

01:03:36.989 --> 01:03:39.650
just obtaining the medical records and do you

01:03:39.650 --> 01:03:41.349
see that as kind of being another roadblock in

01:03:41.349 --> 01:03:43.449
the future when you're trying to do these other

01:03:43.449 --> 01:03:45.429
predictive models is just actually getting access

01:03:45.429 --> 01:03:49.369
to the data that you need it is a major challenge

01:03:49.369 --> 01:03:51.809
i mean i think that uh we're not bad at it i

01:03:51.809 --> 01:03:54.250
spent a lot of time on it i don't know how uh

01:03:54.250 --> 01:03:57.469
you know how hard is as i say it's hard enough

01:03:57.469 --> 01:03:58.989
to spend a lot of my time on it i think about

01:03:58.989 --> 01:04:02.699
it a lot i And it's so hard, I don't know how

01:04:02.699 --> 01:04:05.960
to compare it to anything else. So it's that

01:04:05.960 --> 01:04:09.599
level of hard. It's like tier one hard. And it's

01:04:09.599 --> 01:04:11.480
not just about finding other centers. It's like

01:04:11.480 --> 01:04:13.739
the kind of stuff that we're building and the

01:04:13.739 --> 01:04:15.619
level of which we're trying to pull data is like,

01:04:15.659 --> 01:04:17.539
I want everything in the health system that's

01:04:17.539 --> 01:04:21.019
ever happened. I want everything. That's hard.

01:04:21.159 --> 01:04:23.199
There's not a button. There's not people have

01:04:23.199 --> 01:04:25.039
been asking for that for many years. And it's

01:04:25.039 --> 01:04:26.800
very easy to export everything in an easily available

01:04:26.800 --> 01:04:29.360
format. It takes a lot of time to pull it. It's

01:04:29.360 --> 01:04:32.059
a basic, even internally. The basic piping to

01:04:32.059 --> 01:04:35.739
download CTs at UCSF is not that fast. It takes

01:04:35.739 --> 01:04:37.219
a lot of time. And we're spending a lot of time

01:04:37.219 --> 01:04:38.300
kind of like, okay, well, what's the best way

01:04:38.300 --> 01:04:39.619
to use the system as it is to kind of get the

01:04:39.619 --> 01:04:42.539
volume that we need? It's challenging internally,

01:04:42.780 --> 01:04:45.760
it's challenging externally, other places. And

01:04:45.760 --> 01:04:48.079
fundamentally, because it's so much, it's so

01:04:48.079 --> 01:04:49.760
laborious to get the system working, it's based

01:04:49.760 --> 01:04:52.639
on relationships. Like not some random stranger

01:04:52.639 --> 01:04:54.940
who like does not care about the project or like

01:04:54.940 --> 01:04:56.639
what we're trying to achieve. It's going to go

01:04:56.639 --> 01:04:58.179
through all the work to curate this data set

01:04:58.179 --> 01:05:00.389
for us. You have to find partners that believe

01:05:00.389 --> 01:05:01.670
in the mission, that you're aligned with. You've

01:05:01.670 --> 01:05:03.050
got to build something together. They're willing

01:05:03.050 --> 01:05:04.889
to go through the work with you to make it happen.

01:05:06.090 --> 01:05:09.090
And it's a burden of love. It's a burden of love.

01:05:10.590 --> 01:05:13.469
Another question. A lot of people don't realize

01:05:13.469 --> 01:05:17.489
that not everybody has digital records as well.

01:05:17.630 --> 01:05:19.570
So we still get a lot of outside hospitals that

01:05:19.570 --> 01:05:22.190
will send us images that are captured on DVDs

01:05:22.190 --> 01:05:25.849
or CDs. And I don't know how it is because you

01:05:25.849 --> 01:05:28.110
had collaborations just with other parts of the

01:05:28.110 --> 01:05:31.250
world. And I'm not sure as far as like digital

01:05:31.250 --> 01:05:33.389
radiographs, like a lot of places can't upload

01:05:33.389 --> 01:05:34.550
them pretty quickly. You were just saying it

01:05:34.550 --> 01:05:36.730
takes a lot of time just to kind of actually

01:05:36.730 --> 01:05:38.989
get the information and then like download it

01:05:38.989 --> 01:05:41.630
into your system. Have you had trouble just getting

01:05:41.630 --> 01:05:45.110
just the type of media that the imaging comes

01:05:45.110 --> 01:05:47.469
on? Has that been a problem too, as far as integrating

01:05:47.469 --> 01:05:50.610
that? Not even integrating, but just getting

01:05:50.610 --> 01:05:54.320
the data and being able to upload that? it just

01:05:54.320 --> 01:05:56.380
happens to think about the scale like moving

01:05:56.380 --> 01:06:00.420
500 terabytes of images one place to another

01:06:00.420 --> 01:06:03.179
it's just not it's not that easy because like

01:06:03.179 --> 01:06:04.559
fundamentally you pull from the clinical system

01:06:04.559 --> 01:06:05.760
well clinical systems they do clinical stuff

01:06:05.760 --> 01:06:08.380
all day so it has to slow down so you don't mess

01:06:08.380 --> 01:06:11.860
up clinical operations and like moving that much

01:06:11.860 --> 01:06:13.559
thing when you don't have is like you know fast

01:06:13.559 --> 01:06:15.500
links between places just like it's just slow

01:06:15.500 --> 01:06:19.079
and like and if people haven't done that kind

01:06:19.079 --> 01:06:20.659
of thing before beating the pipes is challenging

01:06:20.659 --> 01:06:22.579
not at the patient level when someone wants to

01:06:22.579 --> 01:06:27.050
like get a risk score, or they're curious about

01:06:27.050 --> 01:06:30.110
the research, they want to try a model. Fundamentally,

01:06:30.130 --> 01:06:32.650
as a patient, you don't have easy query access.

01:06:32.809 --> 01:06:37.849
People get CDs. People sometimes mail me CDs.

01:06:40.170 --> 01:06:43.190
It's too challenging as a patient to get access

01:06:43.190 --> 01:06:47.070
to your data in a convenient format. At the institutional

01:06:47.070 --> 01:06:50.170
level, that's not really the thing. PAC systems

01:06:50.170 --> 01:06:52.590
have APIs, but depending on the hospital that

01:06:52.590 --> 01:06:56.010
you're in, You don't have the bulk export enabled

01:06:56.010 --> 01:06:57.829
in your PACS license. They're going to squeeze

01:06:57.829 --> 01:07:00.329
you for more money. Or maybe you have it, but

01:07:00.329 --> 01:07:02.610
you don't have an intermediate place to store

01:07:02.610 --> 01:07:06.230
500 terabytes of storage. So it's not trivial.

01:07:06.710 --> 01:07:10.829
It's not trivial because of the scale. With regard

01:07:10.829 --> 01:07:15.530
to funding, has it been difficult to get funding

01:07:15.530 --> 01:07:19.570
just as far as governmental, publicly, privately,

01:07:19.789 --> 01:07:22.309
however you procure funding for your research?

01:07:23.159 --> 01:07:25.519
And do you think that there's a way that you

01:07:25.519 --> 01:07:28.500
could, I don't know, hopefully this will help

01:07:28.500 --> 01:07:30.519
this podcast, depending on who takes a listen

01:07:30.519 --> 01:07:32.579
to it, kind of spread the word about what you're

01:07:32.579 --> 01:07:35.199
doing and how beneficial it can be and probably

01:07:35.199 --> 01:07:40.719
will be. Have you found it hard to actually get

01:07:40.719 --> 01:07:44.280
funding for what you're doing? Yeah, I think

01:07:44.280 --> 01:07:45.800
that's one of the things. You never have a baseline.

01:07:46.239 --> 01:07:47.800
You spend a lot of time on it. You work hard

01:07:47.800 --> 01:07:53.460
on it. It is always challenging, and the cycle

01:07:53.460 --> 01:07:55.019
time that we have in the system is quite slow.

01:07:55.679 --> 01:08:00.739
So just today, I got my notice of award for the

01:08:00.739 --> 01:08:03.860
R01 slash R37 that I wrote. Very excited for

01:08:03.860 --> 01:08:05.500
that. That's going to hopefully give us quite

01:08:05.500 --> 01:08:09.000
a bit of runway. From the time of writing that,

01:08:09.199 --> 01:08:13.519
the first time I wrote it was June of 20 -something,

01:08:13.699 --> 01:08:18.600
more than a year ago. I think it was, yeah, it

01:08:18.600 --> 01:08:21.859
must have been June of 23. They had some comments.

01:08:21.880 --> 01:08:23.659
We revised it. The first time we could resubmit,

01:08:23.760 --> 01:08:29.439
given the revision, was February of 24. And we

01:08:29.439 --> 01:08:33.279
did not receive that funding until today. Now,

01:08:33.380 --> 01:08:35.760
I'm super, like, I mean, the fact we have the

01:08:35.760 --> 01:08:38.180
NIH, there's nothing quite like it. It's an incredible

01:08:38.180 --> 01:08:40.800
resource. But, like, practically the cycle time

01:08:40.800 --> 01:08:43.060
of, like, having an idea, having the resource

01:08:43.060 --> 01:08:46.619
to pursue that idea, median time is, like, 18

01:08:46.619 --> 01:08:49.640
months. That's very slow in the pace of science.

01:08:50.730 --> 01:08:52.449
Because by the time that comes over, you have

01:08:52.449 --> 01:08:53.909
all these kind of new ideas. And you just can't

01:08:53.909 --> 01:08:59.189
be very nimble. And that means it's hard to move

01:08:59.189 --> 01:09:02.529
quickly. And practically right now for our work,

01:09:03.149 --> 01:09:05.310
I can say for a fact, the rate limiting factor

01:09:05.310 --> 01:09:07.489
is not ideas. It's not students building really

01:09:07.489 --> 01:09:10.050
great stuff. It's compute. And it's going to

01:09:10.050 --> 01:09:11.649
remain that way for the foreseeable future. And

01:09:11.649 --> 01:09:13.869
so I think as a lab leader, what is my responsibility?

01:09:14.149 --> 01:09:16.449
I'm trying to both create things in the world

01:09:16.449 --> 01:09:19.560
of cancer. and develop useful things for patients.

01:09:20.319 --> 01:09:22.060
And the best way for me to do that is to empower

01:09:22.060 --> 01:09:25.180
my team to do that. And resourcing and getting

01:09:25.180 --> 01:09:28.180
computers is a huge part of that. And when anything

01:09:28.180 --> 01:09:31.000
that you do has a minimum year plus cycle time

01:09:31.000 --> 01:09:33.720
to the normal pathways, it's hard to be nimble.

01:09:34.039 --> 01:09:35.779
And I think there's a better system to be had.

01:09:37.140 --> 01:09:38.840
So it's not easy. It's not easy for anyone. I

01:09:38.840 --> 01:09:43.800
think things have gone relatively well. Relatively.

01:09:44.020 --> 01:09:45.640
But compared to one's ambitions, compared to

01:09:45.640 --> 01:09:47.840
my ambitions, not enough. Though that's more

01:09:47.840 --> 01:09:51.260
dispositional than anything. I think I'm always

01:09:51.260 --> 01:09:52.899
trying to think of ways of how can we move faster?

01:09:53.020 --> 01:09:55.800
How can we deliver faster? To me, the biggest

01:09:55.800 --> 01:09:57.760
risk is in 10 years, care is still the same.

01:09:58.460 --> 01:10:02.939
That is failure. And the normal time for translating

01:10:02.939 --> 01:10:05.560
anything is super slow. And so we need to find

01:10:05.560 --> 01:10:08.039
ways to be faster, more aggressive. And for now,

01:10:08.220 --> 01:10:10.720
compute is a limiting factor. And resourcing

01:10:10.720 --> 01:10:15.970
is a big part of the game. A few kind of just

01:10:15.970 --> 01:10:19.470
AI questions in general. So I know that quantum

01:10:19.470 --> 01:10:22.489
computing is something I'm not a super technically

01:10:22.489 --> 01:10:26.489
savvy person, but I've read a little bit just

01:10:26.489 --> 01:10:29.069
about quantum computing and how it might be a

01:10:29.069 --> 01:10:31.710
game changer as far as giving you more compute

01:10:31.710 --> 01:10:34.310
power and just kind of speeding everything up.

01:10:34.449 --> 01:10:37.020
I know that there... there are chips being processed

01:10:37.020 --> 01:10:38.899
right now, but how far in the future do you see

01:10:38.899 --> 01:10:41.020
like quantum computing being able to be incorporated

01:10:41.020 --> 01:10:44.079
into artificial intelligence? And is that something

01:10:44.079 --> 01:10:46.000
that could help speed up the process of a lot

01:10:46.000 --> 01:10:50.039
of what you're doing? I'm not sure. I have some

01:10:50.039 --> 01:10:52.520
friends that work in that space. And as far as

01:10:52.520 --> 01:10:54.460
I understand, I haven't seen any kind of like

01:10:54.460 --> 01:11:00.300
AI breakthroughs made conceivably possible by

01:11:00.300 --> 01:11:02.239
that. As far as I know, the type of stuff that

01:11:02.239 --> 01:11:05.409
quantum is really great for is like shor's algorithm

01:11:05.409 --> 01:11:07.810
and stuff to kind of like you know factor primes

01:11:07.810 --> 01:11:12.090
and stuff and uh and there's already post quantum

01:11:12.090 --> 01:11:15.010
crypto algorithms that are out there so i personally

01:11:15.010 --> 01:11:16.710
and i know that like if you're doing quantum

01:11:16.710 --> 01:11:18.510
simulations and some type of molecular modeling

01:11:18.510 --> 01:11:21.090
it's very useful uh but in the kind of stuff

01:11:21.090 --> 01:11:24.869
that i do it's hard it's it's unknown uh maybe

01:11:24.869 --> 01:11:26.250
this country comes around and it's great and

01:11:26.250 --> 01:11:28.890
it's awesome that'd be cool uh but i don't uh

01:11:28.890 --> 01:11:31.229
i don't see a light at the end of the tunnel

01:11:31.229 --> 01:11:35.710
of that kind i think uh More and more, we're

01:11:35.710 --> 01:11:38.069
getting the NVIDIA GPUs keep getting better and

01:11:38.069 --> 01:11:40.210
better. And really, it's not a matter of technologies

01:11:40.210 --> 01:11:44.090
not exist. It's having the money to buy it and

01:11:44.090 --> 01:11:50.189
then doing stuff with it. This is kind of a ridiculous

01:11:50.189 --> 01:11:55.289
question, but are you familiar with the simulation

01:11:55.289 --> 01:11:58.890
theory? We're just a part of a simulation. And

01:11:58.890 --> 01:12:01.090
if you are familiar with that, do you buy into

01:12:01.090 --> 01:12:06.159
the simulation theory? I mean, like, there's

01:12:06.159 --> 01:12:07.899
some things I think are, like, I don't even think

01:12:07.899 --> 01:12:10.260
about because they're not actionable. You know?

01:12:10.300 --> 01:12:13.899
So, like, from a probability standpoint, so if

01:12:13.899 --> 01:12:16.079
you believe this is true, like, I mean, it's

01:12:16.079 --> 01:12:18.760
plausible. If you create a simulation, sure,

01:12:18.899 --> 01:12:22.100
I guess. Why not? Then, like, if you take that

01:12:22.100 --> 01:12:24.100
premise and the odds of you being in a simulation

01:12:24.100 --> 01:12:26.800
environment, it's not higher because you don't

01:12:26.800 --> 01:12:29.720
know the probability density distribution. but

01:12:29.720 --> 01:12:32.060
like it would be like believing you're in the

01:12:32.060 --> 01:12:33.680
center of the universe to say that like you are

01:12:33.680 --> 01:12:36.619
in the primal base reality or whatever but like

01:12:36.619 --> 01:12:40.020
functionally I like whatever because if it is

01:12:40.020 --> 01:12:41.979
or it's not true it changes nothing in life so

01:12:41.979 --> 01:12:44.140
it's like there's many things of that kind like

01:12:44.140 --> 01:12:48.199
like another version of this is like if there

01:12:48.199 --> 01:12:50.680
is a if there is a god his favorite number is

01:12:50.680 --> 01:12:52.939
some real number there is an incontestable infinite

01:12:52.939 --> 01:12:54.819
number of those real numbers so no matter which

01:12:54.819 --> 01:12:56.319
of them you believe in you're probably wrong

01:12:56.319 --> 01:13:00.060
because what are the odds that it's true You

01:13:00.060 --> 01:13:03.880
can play these games. There's one of infinite

01:13:03.880 --> 01:13:05.500
possibilities. Because infinite possibilities,

01:13:05.979 --> 01:13:08.680
it would be hubris to say that you're the one.

01:13:08.899 --> 01:13:10.039
You have no reason to believe that your infinite

01:13:10.039 --> 01:13:12.399
possibility is the one. But that's like saying

01:13:12.399 --> 01:13:13.640
I could be wrong about something. You can always

01:13:13.640 --> 01:13:17.300
be wrong about something. Whatever. It's actually

01:13:17.300 --> 01:13:20.439
a great point of view. Another question I had,

01:13:20.579 --> 01:13:22.699
and this is something that a lot of people theorize,

01:13:22.760 --> 01:13:26.279
that eventually when AI becomes fully sentient,

01:13:26.319 --> 01:13:28.359
that there's something called the singularity.

01:13:28.720 --> 01:13:30.460
that is going to happen i i don't know if you

01:13:30.460 --> 01:13:33.039
could like what your thoughts are on the singularity

01:13:33.039 --> 01:13:35.380
and you could even tell like our listeners what

01:13:35.380 --> 01:13:42.920
that means i this is concepts of like once you

01:13:42.920 --> 01:13:44.500
get to a particular level capability then the

01:13:44.500 --> 01:13:46.500
ai can prove itself at some exponential rate

01:13:46.500 --> 01:13:49.020
and it's going to solve all things it's almost

01:13:49.020 --> 01:13:52.619
like a messianic kind of argument of like you

01:13:52.619 --> 01:13:57.060
know hockey stick goes up kind of thing i i think

01:13:57.060 --> 01:14:02.920
that it assumes something about cost. It basically

01:14:02.920 --> 01:14:04.560
assumes that there is no cost improvement, that

01:14:04.560 --> 01:14:06.520
cost is not improving, is not increasing as well.

01:14:06.779 --> 01:14:08.680
And these things are like unbounded and restricted,

01:14:08.899 --> 01:14:10.960
which I think is probably an oversimplification.

01:14:11.779 --> 01:14:14.239
Like, for example, like to build new chips, the

01:14:14.239 --> 01:14:16.760
cost of a fab costs like 2x more or something

01:14:16.760 --> 01:14:22.760
every single time. There's many, like, I'm skeptical

01:14:22.760 --> 01:14:24.840
of the concept that there is no fundamental trade

01:14:24.840 --> 01:14:26.899
-off that you can actually see given existing

01:14:26.899 --> 01:14:32.060
resources. even if you had systems that were

01:14:32.060 --> 01:14:35.720
able to like buy new resources it takes time

01:14:35.720 --> 01:14:38.460
to build a new power plant and to create a new

01:14:38.460 --> 01:14:40.640
data center there's time to like you know lay

01:14:40.640 --> 01:14:43.300
more bricks and like manufacture more trips like

01:14:43.300 --> 01:14:44.520
there's a bunch of physical time stuff there

01:14:44.520 --> 01:14:46.779
and so the notion that you can get exponential

01:14:46.779 --> 01:14:48.659
growth you have in fixed constraints and fixed

01:14:48.659 --> 01:14:51.979
resources forever right so that's where you get

01:14:51.979 --> 01:14:53.439
the singularity thing and now like you solve

01:14:53.439 --> 01:14:56.319
all things it like seems to be a bit bold Like,

01:14:56.439 --> 01:14:57.600
if you want to invent new drugs, you can put

01:14:57.600 --> 01:15:01.319
that shit in mice. Mice take time to grow. You

01:15:01.319 --> 01:15:03.039
know, like, I just think, like, the over -signification

01:15:03.039 --> 01:15:04.859
there is, like, to me, feels like a stretch.

01:15:06.619 --> 01:15:09.020
Will we get systems that will change the nature

01:15:09.020 --> 01:15:12.060
of labor? You would hope so. I mean, the internet

01:15:12.060 --> 01:15:14.199
did that. But then it sounds much less magical.

01:15:14.659 --> 01:15:18.300
So, like, will we have things that will disrupt,

01:15:18.439 --> 01:15:20.439
like, you know, disrupt the way that we work

01:15:20.439 --> 01:15:22.819
and cause a magical transformation that will

01:15:22.819 --> 01:15:24.260
be the biggest thing we remember of this century?

01:15:25.260 --> 01:15:29.220
I think so. Does it look like a sci -fi movie?

01:15:30.119 --> 01:15:34.180
I mean, what are the odds that the way we imagine

01:15:34.180 --> 01:15:36.239
it happens to be what's going to happen? It's

01:15:36.239 --> 01:15:41.279
continuous odds, so probably not. Do you ever

01:15:41.279 --> 01:15:44.819
have any fears that with AI that it's going to

01:15:44.819 --> 01:15:46.579
get to a point where it could become detrimental

01:15:46.579 --> 01:15:49.699
to humanity? Do you think that we're, you know,

01:15:49.720 --> 01:15:53.140
I would say smart enough? or cautious enough

01:15:53.140 --> 01:15:56.199
that we'll take precautions developing ai that

01:15:56.199 --> 01:15:59.180
we don't have to worry about it you know realizing

01:15:59.180 --> 01:16:02.079
that humans are actually the cause of all terrible

01:16:02.079 --> 01:16:03.859
things in this world so it decides to destroy

01:16:03.859 --> 01:16:07.760
all humans do you think that uh are you more

01:16:07.760 --> 01:16:10.319
optimistic or pessimistic towards the future

01:16:10.319 --> 01:16:15.699
uh i'm i think we'll both do harm and good like

01:16:15.699 --> 01:16:17.819
we do with basically everything i think we're

01:16:17.819 --> 01:16:19.399
going to get more volatile with the good we can

01:16:19.399 --> 01:16:21.079
achieve it's going to be more ambitious than

01:16:21.079 --> 01:16:23.279
we currently thought And the bad we can achieve

01:16:23.279 --> 01:16:24.760
is going to be worse than we can currently think.

01:16:25.640 --> 01:16:29.420
And it's going to happen through both. It's going

01:16:29.420 --> 01:16:31.699
to be this kind of running balance between the

01:16:31.699 --> 01:16:33.460
two, and it's hard to predict what that one will

01:16:33.460 --> 01:16:35.640
be. The internet has done a huge amount of good.

01:16:36.319 --> 01:16:39.840
It has also done some harm. Social media has

01:16:39.840 --> 01:16:41.600
done some good, and it's done some harm. And

01:16:41.600 --> 01:16:43.520
I think AI is what I would call a broad empowering

01:16:43.520 --> 01:16:46.399
technology that lets you do a lot of both. And

01:16:46.399 --> 01:16:49.619
I don't like to say that we would do, to take

01:16:49.619 --> 01:16:53.029
a purely optimistic view. would be utopian that

01:16:53.029 --> 01:16:56.609
like we've we've exited human conflict that seems

01:16:56.609 --> 01:16:59.390
unlikely uh and to take a broadly pessimistic

01:16:59.390 --> 01:17:04.050
view would would be counter to the arc of like

01:17:04.050 --> 01:17:05.729
psychological progress across the last like 80

01:17:05.729 --> 01:17:10.029
years uh but still i view these things like i

01:17:10.029 --> 01:17:12.710
very much of the mode of like i have a particular

01:17:12.710 --> 01:17:14.710
problem trying to solve there's a classic they're

01:17:14.710 --> 01:17:16.970
trying to do better and if we execute poorly

01:17:16.970 --> 01:17:18.579
we could do some harm We're going to try very

01:17:18.579 --> 01:17:19.520
hard not to. We're going to try to build all

01:17:19.520 --> 01:17:20.779
the right systems to do so. We're also going

01:17:20.779 --> 01:17:23.520
to be driving towards better care. That's why

01:17:23.520 --> 01:17:25.119
you have to run trials, because not every trial

01:17:25.119 --> 01:17:28.800
succeeds. That's the kind of name of the game.

01:17:28.960 --> 01:17:31.600
On the broader societal level, there's lots of

01:17:31.600 --> 01:17:37.760
weird stuff. I have no claim on the wisdom to

01:17:37.760 --> 01:17:40.460
predict how this will interact with geopolitics.

01:17:40.560 --> 01:17:44.659
All this kind of stuff is out of my lane. So

01:17:44.659 --> 01:17:46.680
for anyone who's listening who might be curious

01:17:46.680 --> 01:17:50.359
about going into the field, just of programming

01:17:50.359 --> 01:17:55.380
or coding or just in AI in general, who has just

01:17:55.380 --> 01:17:58.000
kind of a fascination with it, what advice could

01:17:58.000 --> 01:18:02.520
you give them with pursuing that as far as where

01:18:02.520 --> 01:18:05.979
they should go to school, what kind of material

01:18:05.979 --> 01:18:08.439
they should read to get involved or just to learn

01:18:08.439 --> 01:18:11.279
more about artificial intelligence and the type

01:18:11.279 --> 01:18:16.600
of work that you do? Like anything, you learn

01:18:16.600 --> 01:18:19.699
by doing, you learn by building. And I would

01:18:19.699 --> 01:18:21.560
say it's a better time to build doubt than ever

01:18:21.560 --> 01:18:24.359
before. As an individual person trying to build

01:18:24.359 --> 01:18:25.680
something new, you have more leverage now than

01:18:25.680 --> 01:18:28.119
ever before. The tools are better, the resources

01:18:28.119 --> 01:18:31.539
are better, the compute, the raw hardware is

01:18:31.539 --> 01:18:35.819
better than ever before. And so, like, less reading,

01:18:35.880 --> 01:18:38.159
more building, you learn by doing. Now, if you

01:18:38.159 --> 01:18:39.899
want to do the kind of stuff that I do, then

01:18:39.899 --> 01:18:43.420
practically you need access to... collaborators

01:18:43.420 --> 01:18:45.279
in a lab that has the right resources to kind

01:18:45.279 --> 01:18:47.020
of do that. And so like, you know, reach out

01:18:47.020 --> 01:18:48.399
to the faculty interested in working with and

01:18:48.399 --> 01:18:50.279
like try to find opportunities. There's opportunities

01:18:50.279 --> 01:18:52.800
in industry in addition to academia. But like

01:18:52.800 --> 01:18:55.100
fundamentally someone to get into AI, build stuff.

01:18:56.699 --> 01:18:59.840
It is easier than ever. And because it's easier

01:18:59.840 --> 01:19:01.380
than ever to do something small, it means you

01:19:01.380 --> 01:19:02.699
can be more ambitious than ever to do something

01:19:02.699 --> 01:19:04.640
bigger. Like what two people working together

01:19:04.640 --> 01:19:07.539
can build today is like, you know, nine and eight

01:19:07.539 --> 01:19:08.960
came out five years ago. And it's going to continue

01:19:08.960 --> 01:19:10.079
to feel that way for the foreseeable future.

01:19:10.140 --> 01:19:14.529
And it's really cool. It's cool to see. It's

01:19:14.529 --> 01:19:16.630
cool to see. Yeah, I'm fascinated by it. Do you

01:19:16.630 --> 01:19:19.750
have just a large language model that you prefer

01:19:19.750 --> 01:19:23.770
to use? Do you prefer to use ChatGPT or Perplexity

01:19:23.770 --> 01:19:25.670
or Grok? Or is there one that people might not

01:19:25.670 --> 01:19:27.550
know about that is actually more useful than

01:19:27.550 --> 01:19:30.710
any of those? It depends on what you're doing.

01:19:30.850 --> 01:19:33.170
I mean, I use LMS a lot in like prototype encoding

01:19:33.170 --> 01:19:36.090
stuff. Not so much for research. I mean, like

01:19:36.090 --> 01:19:37.550
not so much research for cool stuff when I'm

01:19:37.550 --> 01:19:39.430
like trying to figure out new systems, data infer

01:19:39.430 --> 01:19:43.560
thing. Both ChatGPT and Cloud are pretty good

01:19:43.560 --> 01:19:47.439
at that kind of thing. Practically, as an IDE,

01:19:47.600 --> 01:19:50.600
I use Cursor. Cursor's pretty great. I only use

01:19:50.600 --> 01:19:54.380
it for stuff where I can use it given security

01:19:54.380 --> 01:19:56.899
compliance issues or whatever. But that's...

01:19:56.899 --> 01:20:00.579
I think as a whole, this is a different kind

01:20:00.579 --> 01:20:04.659
of coding than was possible before. And if you're

01:20:04.659 --> 01:20:06.279
doing something where you don't... For example,

01:20:06.300 --> 01:20:07.880
I don't build web apps anymore. I used to when

01:20:07.880 --> 01:20:09.239
I was an undergrad a long time ago. I don't really

01:20:09.239 --> 01:20:11.659
do it anymore for a variety of reasons. i needed

01:20:11.659 --> 01:20:13.520
one for small prototype to test out a particular

01:20:13.520 --> 01:20:18.539
kind of workflow and like five years ago that

01:20:18.539 --> 01:20:20.500
have taken so long you have to like learn react

01:20:20.500 --> 01:20:23.640
and learn how to do npm packagement like all

01:20:23.640 --> 01:20:25.859
this like little stuff but like you know just

01:20:25.859 --> 01:20:30.939
like uh it is much easier to kind of like bootstrap

01:20:30.939 --> 01:20:33.920
to minimum capability very quickly especially

01:20:33.920 --> 01:20:35.840
when you don't know the library so it's cool

01:20:35.840 --> 01:20:38.939
i encourage more people to get in the game yeah

01:20:38.939 --> 01:20:41.640
i've never like i i wouldn't know how to code.

01:20:42.300 --> 01:20:44.560
Is there, so somebody like me who's never coded

01:20:44.560 --> 01:20:46.340
at all in their life, could they use one of these

01:20:46.340 --> 01:20:50.199
LLMs to actually, would it help in teaching them

01:20:50.199 --> 01:20:51.819
or would it just, are you actually able to feed

01:20:51.819 --> 01:20:54.380
in kind of what you're looking for? I'm just

01:20:54.380 --> 01:20:58.000
kind of curious as the process of coding, I understand

01:20:58.000 --> 01:21:01.579
the basis for it, but how much could I, as someone

01:21:01.579 --> 01:21:03.340
who has no experience with it, use one of these

01:21:03.340 --> 01:21:06.420
LLMs to actually develop, say, a webpage or,

01:21:06.520 --> 01:21:08.899
I don't know, I think you said it was for app

01:21:08.899 --> 01:21:11.220
development. For things that are well covered

01:21:11.220 --> 01:21:13.159
by the internet, there's a billion tutorials

01:21:13.159 --> 01:21:15.079
on how to build a web page out there. For that

01:21:15.079 --> 01:21:17.140
kind of thing, it really can. You can literally

01:21:17.140 --> 01:21:18.680
copy and paste the error message back into the

01:21:18.680 --> 01:21:20.220
chat and have it tell it to fix it over and over

01:21:20.220 --> 01:21:22.500
again so it works. For things that are easy.

01:21:23.279 --> 01:21:25.800
A web page is so well covered by the internet

01:21:25.800 --> 01:21:30.720
that it's pretty straightforward. But it is not

01:21:30.720 --> 01:21:33.760
a panacea. It's not being good at coding. What

01:21:33.760 --> 01:21:35.460
it means, if you're good, you just have so much

01:21:35.460 --> 01:21:38.600
more leverage. You can go so much further. But

01:21:38.600 --> 01:21:40.340
if you don't know what you're doing, then at

01:21:40.340 --> 01:21:43.479
some point, it's going to stop fixing it. And

01:21:43.479 --> 01:21:45.479
then you'll be stuck. And even guiding it the

01:21:45.479 --> 01:21:46.760
right thing, understanding what the error is.

01:21:46.859 --> 01:21:51.300
As times get more competitive, the only defense

01:21:51.300 --> 01:21:54.039
is always to be good. That's always what it is.

01:21:54.180 --> 01:21:55.560
When times are good, you want to be excellent.

01:21:55.680 --> 01:21:56.699
When times are bad, you want to be excellent.

01:21:57.260 --> 01:22:00.119
Because resources are narrow or good. That's

01:22:00.119 --> 01:22:03.800
the only path that's out. And so there's no kind

01:22:03.800 --> 01:22:07.279
of panacea, but like... it does mean minimal

01:22:07.279 --> 01:22:09.119
resourcing to build something is smaller than

01:22:09.119 --> 01:22:10.640
ever before. So you can be more ambitious. I

01:22:10.640 --> 01:22:12.880
think that's very cool. I have to jump the next

01:22:12.880 --> 01:22:15.140
two minutes, unfortunately. Oh, sorry, man. Yeah,

01:22:15.260 --> 01:22:17.920
I will let you go. I just want to let you know,

01:22:17.960 --> 01:22:20.079
thank you so much for coming on. I want to let

01:22:20.079 --> 01:22:23.340
our listeners know too. Looking at Adam's research,

01:22:23.439 --> 01:22:26.060
it's excellent. I appreciate you and what you

01:22:26.060 --> 01:22:27.600
do. I've always thought people who especially

01:22:27.600 --> 01:22:30.899
have kind of... Just your drive, your ambition,

01:22:31.039 --> 01:22:33.319
especially with, you know, your technical background

01:22:33.319 --> 01:22:35.619
going to MIT and now you see Berkeley. The fact

01:22:35.619 --> 01:22:38.199
that you use that to try and help other people

01:22:38.199 --> 01:22:41.560
is fantastic. I think it's a noble, noble endeavor.

01:22:42.220 --> 01:22:43.859
There's a lot of people with the same kind of

01:22:43.859 --> 01:22:45.539
background who might, you know, try to be more

01:22:45.539 --> 01:22:48.100
profit seeking. But what you're doing, I find

01:22:48.100 --> 01:22:50.779
is excellent. It's so helpful. And I have a lot

01:22:50.779 --> 01:22:52.659
of faith in your efforts that it's going to continue

01:22:52.659 --> 01:22:55.680
just to grow and just be completely successful.

01:22:56.460 --> 01:22:59.819
So thank you very, very much for coming on. One

01:22:59.819 --> 01:23:01.520
last question. Have you ever seen the television

01:23:01.520 --> 01:23:06.899
show Silicon Valley? I've heard of it. I have

01:23:06.899 --> 01:23:10.680
not seen it. It's a great show. It's a comedy.

01:23:10.800 --> 01:23:13.359
It's on HBO. It aired from, I don't know, it

01:23:13.359 --> 01:23:17.560
goes like 2012 to 2014. I don't know. It's about

01:23:17.560 --> 01:23:20.880
a decade ago. But it involves, it's all about

01:23:20.880 --> 01:23:23.039
living in Silicon Valley. And just, this is more

01:23:23.039 --> 01:23:25.819
about. like app developers trying to get known.

01:23:25.920 --> 01:23:29.340
It's extremely funny though. So if you have a

01:23:29.340 --> 01:23:32.180
chance, check it out. Anyway, Adam, thank you

01:23:32.180 --> 01:23:34.500
so much for joining us. Did you have anything

01:23:34.500 --> 01:23:36.560
else you want to tell our listeners? Anybody

01:23:36.560 --> 01:23:39.640
you want to kind of, where people can find your

01:23:39.640 --> 01:23:46.279
information? Yeah. My, like most faculty, I maintain

01:23:46.279 --> 01:23:48.220
a website. I don't update it enough. Like also

01:23:48.220 --> 01:23:49.939
all other faculty, you know, like in 20 years,

01:23:49.979 --> 01:23:51.779
I'll have the exact same photo I have now. But

01:23:51.779 --> 01:23:53.619
like addemail .org is my website. And if someone

01:23:53.619 --> 01:23:55.460
wants to reach me, then like my email's there.

01:23:56.079 --> 01:23:58.140
And if someone wants to either as a house hunter

01:23:58.140 --> 01:23:59.479
get involved in your own research in some way,

01:23:59.560 --> 01:24:03.340
you know, hit me up. Awesome. Thank you so much.

01:24:03.399 --> 01:24:06.520
I appreciate to just all the time you took. Thank

01:24:06.520 --> 01:24:08.260
you everybody for listening. As always, that

01:24:08.260 --> 01:24:12.380
can be reached through Gmail at makemesickpod.

01:24:12.920 --> 01:24:17.579
And I can also be found on Twitter or X at makemesickpod.

01:24:18.989 --> 01:24:20.770
Thank you again, Adam, very much for coming on.

01:24:20.810 --> 01:24:23.350
I appreciate the time. Remember, everybody, to

01:24:23.350 --> 01:24:51.100
wash your hands. Data and dreams in his steady

01:24:51.100 --> 01:24:58.920
hands Building a world where hope expands Dr.

01:24:59.079 --> 01:25:01.899
Yala's got the spark, the vision so bold He's

01:25:01.899 --> 01:25:04.140
reading the answers and the patterns they hold

01:25:04.140 --> 01:25:07.800
A glow in the cold, a map to uncover Guiding

01:25:07.800 --> 01:25:38.220
us closer, a life to recover The week after y

01:25:38.220 --> 01:26:03.409
'all has got the spark He's weaving a fight,

01:26:03.529 --> 01:26:07.170
a beacon of science in the darkest night Connecting

01:26:07.170 --> 01:26:10.510
the pieces, defying the shroud Lifting the silent,

01:26:10.829 --> 01:26:16.770
making us loud Oh -oh -oh -oh -oh -oh -oh -oh

01:26:16.770 --> 01:26:21.949
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:21.949 --> 01:26:22.829
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:22.829 --> 01:26:23.590
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:23.590 --> 01:26:23.670
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:23.670 --> 01:26:23.689
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:23.689 --> 01:26:23.930
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:23.930 --> 01:26:23.930
-oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh -oh

01:26:23.930 --> 01:26:33.920
-oh -oh -oh -oh -oh Breast cancer secrets he

01:26:33.920 --> 01:26:38.979
dares to seek Giving strength to the scared,

01:26:39.260 --> 01:26:44.720
the lost, the weak Dr. Yala's got the spark,

01:26:44.899 --> 01:26:47.760
a vision so bold He's reading the answers and

01:26:47.760 --> 01:26:50.659
the patterns they hold A glow in the cold, a

01:26:50.659 --> 01:26:54.039
map to uncover Guiding us closer, one life to

01:26:54.039 --> 01:26:56.819
recover Dr. Yala's got the spark, a vision so

01:26:56.819 --> 01:27:44.539
bold Quick Cover! Connecting the pieces, defying

01:27:44.539 --> 01:27:48.779
the shroud, lifting the silent, making us loud.

01:28:11.949 --> 01:28:19.130
With steady hands Crafting hope From lines of

01:28:19.130 --> 01:28:26.510
commands Nearer I see Where shadows hide A beacon

01:28:26.510 --> 01:28:49.560
of hope A guiding light your spark promise me

01:28:49.560 --> 01:28:52.600
mapping risks with data's grace