We're going to talk about artificial intelligence
and we have Carlos Colonga from where again?
Hospital Regional Materno Infantil de Especialidades
de Monterrey. He's from Mexico. We have Em Gootee,
who came as my medical media fellow from Turkey
after graduating medical school, and I was fortunate
to recruit her to stay on as one of my colleagues
now at Cincinnati Children's, helping us in innovation.
And Rami Shahaban, who I was introduced to years
ago, who is an ENT surgeon who is now a Ph .D.
and a professor in instructional technology at
Utah State University. Rock on. All right. This
is the team. They're going to update us on artificial
intelligence. Take it away. So we are going to
talk about AI, specifically regenerative AI use
cases in medicine and medical education. and
we have three categories of use cases research
productivity and medical education So let's start
with that poll question here. So which of the
following is false? Is false. Okay. So that's
tricky a little bit. ChatGBT can help to organize
literature review. ChatGBT can analyze data within
an Excel sheet. ChatGBT can run statistical analyses.
ChatGBT can help in the research design. Or none
of the above. So what do you think? Any thoughts
here in the room? Do you think any of these are
false or all of them are true? What about the
unpaid version? It still doesn't. Yeah. Mike.
Yeah. Oh, right. Yeah. We're talking about both
of them, yes. Let's keep going. All of them are
true. We are researching how ChatGPT can help
us in the research process. I personally use
ChatGPT in systematic review and meta -analysis.
When you do systematic review, you screen thousands
of articles, and then you have a title and abstract
screening, and then the full text screen. And
this process is intimidating. You have to have
many people involved in that process. So we tried
ChatGPT to do the title and abstract screening,
as well as the full text screening, and it does
a great job. Do you want to explain how to do
it? Yeah, sure. I think we have the next slide
here. Talking about that so we have punch of
articles to go into title abstract screen And
then we asked chat GVT this prompt. I'm conducting
a systematic review of the use of AI in organ
transplantation I need you to act as a data analyst
Please analyze the attached excel sheet and screen
the included articles for title abstract by the
way that The AI in organ transplantation paper
was published already so we used chatgbt in that
use the following inclusion criteria we have
some inclusion criteria here and then after screening
the title and abstract so now we want chatgbt
also to manipulate the document itself so it's
not only just giving me text in a answer through
chatgbt but also go inside the excel sheet and
then read the data from excel sheet add columns
to the Excel sheet and say whether this article
is relevant or not relevant or maybe relevant.
Because we don't trust ChatGPT a lot, I asked
ChatGPT also to add a column to explain why ChatGPT
decided that decision. So you have a better understanding
about this data and if you don't agree with ChatGPT,
you can manipulate the prompt, engineer the prompt
to make it more accurate. Okay, and we get an
excellent result with that. We had to do the
human title and abstract screening also because
we want to have a reliable answer, but we did
inter -rater reliability between ChagVT and the
human reliability, and the inter -rater reliability
was excellent. So ChagVT could be used in that.
part yeah so what is what is the first cycle
and second so yeah okay so uh first we wanted
to use chat gbt uh to give us yes or no question
yes or no relevant or not relevant and then we
just redesigned the prompt a little bit to make
chat gbt say whether it's relevant not relevant
and maybe relevant so we decided to consider
all of these versions of the prompt and then
make the inter -rater reliability between all
of that columns. Question? I've done this, not
for the exact same purpose, but I don't get the
same answer each time. So my question is about
reliability. I know humans are not reliable either,
but it sounds like every time you offer this
question, at least to ChatGPT, I haven't tried
other ones, you get a slightly different answer.
It approaches the problem like... for the first
time every time? I think so. Yeah, so if you
use ChatGPT in the general search, you get different
answers, but we created custom GPT for that.
So you can walk ChatGPT in serial of prompts
until you get the most accurate results, which
is just not that difference between if you run
it multiple times, you get a slight difference.
And we had to have human reliability here also
to just make sure. that it is reliable. But that's
a good potential for the future iterations for
. I think also it might be one of the questions
that, what is the AI limitations when you use
it for an actual paper? and go for publishing,
what is the disclosures that you do for the journals,
do they have any limitations, do you have any
extra information on that? Yeah, so we use ChatGPT
here as a part of the methodology and part of
the reliability process. We don't use it to,
for example, as an act of plagiarism or writing,
yes. So, yeah, our journals are very, yeah, of
course, yeah. And we frankly say that this cycle
of screening is done by . The second cycle of
screening done by . Third cycle of screening
is done by humans. So, yeah. You always have
to have a human in the equations. So we have
to take in mind that these AI models, they help
us increase our efficiency. And sometimes we
equate wrongly efficiency to productivity, but
sometimes you get productivity out of the efficiency.
So it helps us be more efficient. by accelerating
how we do the work. But we always, at least in
the medical and the science part, we have to
have a human in the equation. Because we have
to remember that even if we use the prompts that
help you get the same results, because ChatGPT
is a black box model that we don't know what
it is trained about, and they're always changing
the weights, it'll change. So it needs us to
tell it what to do and to understand and read
the answers of ChatGPT. Yeah, you want to move
on to the next one? Yeah. So let's go to talk
about Jenny AI, whose motto is use AI to supercharge
your research paper. So what does it do? You
open the Jenny AI, which is one of different
academic models that has been fine -tuned with
research papers. And you literally put your prompt.
I want to write about this, this, and that. In
this example, let's talk about pediatric interception.
You put the prompt. And the first thing that
you'll get is a suggestion. You can start writing
this. You, because you're the surgeon that has
the hand on the wheel, can say, yeah, I like
it. I want to modify it. I want to change it
in this and that way. And then, the next slide,
please. You can also talk with its in -house
chatbot. and say, hey, I really want to give
it this pin. I want it to research specifically
in this. And the chatbot will give you the answers
that you're searching for. It will cite papers
to you, and it can help you cite on your preferred
type of citation specifications into your paper.
Next, please. So here. You've decided that, yes,
I like what it is saying. I also want you to
add this, this, and that on my academic paper.
And it gives you a little suggestion of the summary
of the papers that you want to introduce. So
we can see how this accelerates your paper writing,
but you're still at the command of it. You're
still in the helm. Next, please. So let's do
an audience poll to see how you're feeling about
this. So how recent are the articles suggested
from the use of these AI models? So they have
a cut -off date of 2023. They're older than five
years, older than 10 years, and they're on par
with the latest published research. What do you
think? Any ideas? Can we see the answers from
the international audience, please? I don't know
if the poll is up. Oh. Can we see the poll results?
Okay. How do people know what's up? Yeah. So,
mostly, nowadays, they're mostly on par. We just
have to research into what model we're using
and what library they're using, but it is on
par. We just have to remember that they're searching.
for those latest articles in this specific type
of library, which we'll talk about it at a later
moment. Well, that was the moment, yeah. So...
Also, there's another tool with the Gen -E AI.
It's called Open Evidence, and I think you said
this is the difference that this is not necessarily
helping you write it, but it helps you do literature
search. Exactly, which is amazing. This has been...
This was part of the program accelerator over
by Harvard team and the Magic Clinic. And they
especially focus on the Elsevier Library. So
you can ask any type of question. It will reply
in a conversational manner, trying to answer
your question while summarizing the papers, the
latest papers that it's citing. So it gives you
a conversational answer. And then at the end
of it, It gives you the papers that you can personally
check out to confirm what it's saying. If you
want to press the details, it gives you a bigger
summary. And then it suggests, hey, so you're
talking about these things. It's like you just
cited Dr. Russell's paper in the second part.
Yeah, yeah. And if you wanted to go... Accurate.
Yeah. And it also gives you suggestions. So I
know you're talking about the appendectomy. Hey,
you could also talk about this and that, which
is up in the next slide. And you can ask follow
-up questions. Yep, yep, yeah. So it's pretty
great. So what would you say the personal practice
changes for your part? For my part? Okay. So
we are going to talk about the medical education
part. Is that right? Medical education is coming
next. Okay. We're going to first talk about the
productivity part, but I think it kind of summarizes
that large language models can enhance the efficiency
and productivity when you do research by analyzing
the large data sets for you, or you can use those
tools to help you write. This is especially very
important for the international audience that
have, once you have something on hand and work
on it, it's way easier. and writing something
by yourself from scratch. So these large language
models also can be an editor and help you write
these papers. And we're gonna move on. Yeah,
Jose. Dr. Holcomb, what's the JPS policy about
these AI tools? I think we wanted to submit something.
They asked if we used it. Yeah, so you can use
it for background information, but you can't
use generative AI, that is, to have AI write
you the paper. And are you making sure it's not
being used, or you have to disclose it somewhere?
Well, first of all, if you used it and you disclosed
it, it's probably not going to get accepted.
But we have, we being Elsevier, has tools to...
to figure out if it's been used or if it's being
used. So that's sort of the policy is you can't
use generative AI to create your paper, to write
your paper. I think it's okay to use it. You
could, for language, it's probably appropriate.
And for literature, you know, background information,
it's certainly appropriate. I think it all started
a couple years ago when ChatGPT started getting
used in different fields. The first couple papers
were written and ChatGPT was among the authors.
And I think it's not been permitted anymore.
You cannot do that. But I don't know. Probably
it's the same for all the journals. You cannot
ask ChatGPT to write a section of it and show
it in the authors. Yeah, most of the journals
I see is just asking you to, if you want to use
ChatGPT, you use it to just enhance the writing
style, not create the ideas. So, yeah. And our
next question, because we're going to talk about
ChatGPT and other AI tools and how you use to
enhance your daily productivity, not necessarily
for research. And we asked the audience, how
often do you use ChatGPT or similar AI tools?
It could be Cloud, Gemini, anything. It's a spectrum
from never to always. And one third is never.
Whenever Todd sees those numbers. He kind of
has a heart attack every time he sees people
that never use chat, GPT, or AI. So if you first
start with productivity part with MeetGeek, this
is one of the tools that helps you. You basically
synchronize this tool with your calendar. It
jumps in. Meeting that you attach to your calendar
and it takes notes for you and at the end of
it It sends you a mail of the transcription notes
and the summary And it basically tells like who
was in the Hoover in the meeting what was the
summary and what you need to do it goes with
the recording and it assigns action items per
person who are in the meaning and Fireflies is
a similar one with the Miki if you go look in
the internet to see meeting summarizer AI tools.
They're like a bunch of them. Most of them does
the same job. They're on different price points.
Most of the time, it's just personal preference.
They mostly do the same thing. I know you started
using MeetGeek and then you transferred to Fireflies
because we... Influenced you? Yeah, I think Firefly,
yeah. So I use mid -gig, I use the paid version
of mid -gig, and then Firefly is the free version,
and then the paid version. And when you invite
Rami to a meeting, shows up as four people. Rami,
AI note -taker one, Rami, AI note -taker two.
The human version is not there, is that right?
So I think Firefly is... My personal preference
is because it has also action items that So after
the meeting it it gives you the action items
for what's next so it's just to keep you Informed
it is great. I use fireflies because of you guys.
I started using Carlos also comes as a two people
every meeting The issue that we've run into,
because I was addicted to it, I needed it because
it gives me takeaways and all the key points.
Here's what you need to do. But then our hospital,
people got worried, especially with some of the
meetings I was having, that it was auto -recording
everything. So you just have to figure out what's
the safe way. At Cincinnati Children's, they
now have... co -pilot approved by co -pilot that's
approved by the hospital but even then our lawyers
asked me to stop using it on half of my meetings
because they're legally sensitive well you talk
about confidential things yeah confidential things
so um it's it's We're going to make progress
with that because this is such a valuable tool,
and we have to get past the security part of
it, but it is game -changing. Especially if you're
using an institution level, it needs to be a
lot of preventative, like preventions be taking
place for security purposes, especially like
you and Dr. Winallman's level executive meetings,
but like you and me in a meeting. Fireflies can
record that. There's not going to be that one.
We've actually. Added it to our platform. So
we use a co -pilot and that's the only thing
that's basically approved You know for Carl stores
to use in regards to note -taking or getting
information Recording stuff and then you know
analyzing it afterwards I think it's the same
at Cincinnati Children's and they keeps that
they Generally send us like weekly AI guidelines
of how should we use it? And what kind of information
should not be used when you use co -pilot or
ask for it? meeting summaries and everything.
And I think that's a good step to endorse people
to use with not giving away the secrets of the
company, basically. Yeah, we have to have these
talks. We have to learn how to use it because
it's here to stay. We can't close our eyes and
don't admit that everyone's using it to increase
their efficiency and improve the way they manage
their day -to -day lives. Zoom AI basically does
the same thing because some companies have Zoom
instead of Microsoft. But the plus I saw in the
Zoom, if you're late to a meeting and you have
Zoom AI companion, you can ask if your name is
mentioned, what people ask you to do, and recap
the meeting just before you join what happened.
So I think that's a very cool thing. And if you're
generally late to the meetings, I would definitely
recommend. One minute? Okay, sorry. Okay, this
is Todd's favorite. Read aloud GPT. We created
for it. You can use this QR code and you can
use it by yourself. It basically helps you listen
to your long emails or any text that you need
to listen to instead of you read. We know Todd
talks, like have to not talk, sorry. He has to
read a lot of things, the emails and everything.
And sometimes, what do you call walk and talk?
T 'walk. T 'walk. So it's hard to read when you
walk. So you listen to it and you use the ChatGPT
to have conversations. Instead of an audio book,
I will drive. I'm about to drive to Cincinnati
after this. And I will talk to ChatGPT for an
hour about a book. So I'll say, summarize this
book. And I will go back like a book club. I'll
say, tell me now, what does it say about this?
Tell me, let's go into chapter two. I want to
know this. I don't get this. So I will have a
conversation while I work out, while I am driving.
And I think this is one of the most important
ones for this audience here. Todd said he's getting
a lot of requests for letter of recommendations.
And we created this GPT for him. And you can
use the QR code if you can't put the slides up
here. But they keep saying we created GPTs. It's
very easy to do. So if you're using the same
prompts over and over and it's a series of prompts,
you can create it and just keep reusing it. So
when I have a letter that I have to write, it
goes through M -Media for me. So when you use
this QR code, you're going to go to this letter
of recommendation GPT. All you need to do is
upload the CV cover letter. Why are you writing
this letter of recommendation? It's going to
give you a draft. You can start working on it
and personalize it for the candidate. We do not
recommend the first thing you get from Chachapit
to use it, but it's a great start for... It will
sometimes be over... This person is from heaven.
I actually almost always come back and say, I
really don't know this person. I'm giving a reference,
so please tone it down. And then it's like, this
person is very good, blah, blah, blah. So you
have to just tell it what you want. Like with
Halpern, I'm going to say, please really tone
this down. He's not that good. Do you want to
do the medical education real fast? Yeah. Sure.
Just one minute. We'll just talk quickly about
that. The medical education part, if you want
to have a personal trainer, we use also ChatGPT
and other generative AI in that. You can, for
example, ask ChatGPT to upload a guideline to
ChatGPT and then ask ChatGPT to create a training
session, an interactive training session between
you and ChatGPT. Can we put the slides up? Thank
you. Yeah, so you can ask ChatGPT to create a
training session out of a guideline so it doesn't
have to go into hallucinations or fabricating
information. It takes the guideline and then
it creates a series of interactive questions
where you can interact with ChatGPT and get trained
on specific things. So, for example, that's an
appendicitis case that we ask ChatGPT to create
problem -solving activities. And also we ask
ChatGPT... to score these activities and give
us feedback at the end. So that's the feedback
that it gets at the end. On a similar note, you
could also make it do any type of case, a step
one, step two type of case, any type of case.
You can ask for it. You do the correct prompt,
and with that, it'll help you do the medical
case in a step -by -step manner, and at the end,
it will give you feedback. All right, this was
awesome. I let us go way over. That's my fault.
I'm sorry. AI is cool. Keep using it. This is
phenomenal. These guys are great. We're going
to keep doing series of courses throughout the
year for these kind of questions. But guys, thank
you so much. That was awesome.