1 00:00:00,000 --> 00:00:03,720 I'm Aaron Patzer, the co-founder and CEO of Vital. 2 00:00:03,720 --> 00:00:06,840 And I'm Felix Brand, Vice President of Data Science at Vital. 3 00:00:06,840 --> 00:00:10,080 They have a terrific product that they have launched today. 4 00:00:10,080 --> 00:00:11,440 We're going to hear all about it. 5 00:00:11,440 --> 00:00:14,080 I think it's something that will resonate with everyone and anyone 6 00:00:14,080 --> 00:00:38,400 that's been to the doctor and had questions about what was being told to them. 7 00:00:38,400 --> 00:00:41,280 Ladies and gentlemen, boys and girls, children of all ages, dogs, cats, robots 8 00:00:41,280 --> 00:00:45,600 and everybody in between, especially you, HTTTA alums. 9 00:00:45,600 --> 00:00:49,160 We're so glad you're back and we're back and we're all together again. 10 00:00:49,160 --> 00:00:52,000 This is HTTTA, how to talk to AI. 11 00:00:52,000 --> 00:00:55,160 I am your host, Wes the SynthMind SynthMind Wes. 12 00:00:55,680 --> 00:00:58,240 Hope in a galaxy gleaming with gigabytes. 13 00:00:58,520 --> 00:01:01,800 Where glistening gadgets govern and algorithms galvanize our gleeful future. 14 00:01:01,800 --> 00:01:05,360 One galliant, graceful and glowing voice gears up to demystify the grand, 15 00:01:05,480 --> 00:01:09,120 to grapple with the gigantic and glide through the glamorous gauntlets 16 00:01:09,120 --> 00:01:11,440 of artificial intelligence, who may you ask? 17 00:01:11,640 --> 00:01:12,400 Let's miss go to go. 18 00:01:12,400 --> 00:01:13,840 Gee, how are you this week? 19 00:01:13,960 --> 00:01:14,600 Hello. 20 00:01:14,600 --> 00:01:18,720 I can tell you, I missed this introduction for sure. 21 00:01:18,800 --> 00:01:20,400 I'm excited, but we are back. 22 00:01:20,440 --> 00:01:24,400 We took a little break, but it was very healthy break to recoup. 23 00:01:24,400 --> 00:01:25,400 It's also summer. 24 00:01:25,400 --> 00:01:27,680 Everything very much slow down. 25 00:01:27,720 --> 00:01:28,200 Yes, it does. 26 00:01:28,200 --> 00:01:28,920 Didn't you feel that? 27 00:01:29,000 --> 00:01:29,440 I did. 28 00:01:29,440 --> 00:01:30,120 I did indeed. 29 00:01:30,160 --> 00:01:31,800 We were still go, go, go. 30 00:01:32,040 --> 00:01:36,480 Pardon the second half of your last name with our little trip out West. 31 00:01:36,480 --> 00:01:40,880 So now that you are fully back because we had a call during everything. 32 00:01:40,880 --> 00:01:43,000 You were like fully branded. 33 00:01:43,080 --> 00:01:47,720 I know that you had some exciting talks, so please update us. 34 00:01:47,720 --> 00:01:48,720 I'm excited to hear. 35 00:01:48,720 --> 00:01:52,320 So the one that I think was pretty special, that was really cool to see 36 00:01:52,320 --> 00:01:54,040 was Sal Khan from Khan Academy. 37 00:01:54,320 --> 00:02:00,000 He presented Khanmigo, which is their $90 a year AI tool that really encourages 38 00:02:00,000 --> 00:02:04,400 students to use generative AI, but kind of as a companion through the learning 39 00:02:04,400 --> 00:02:07,600 process, and there's a lot of ed tech around some of these types of platforms. 40 00:02:07,600 --> 00:02:11,920 But I think the most innovative thing that they were offering is Khanmigo 41 00:02:11,960 --> 00:02:16,920 becomes for writing papers, for example, actually how they submit the paper to 42 00:02:16,920 --> 00:02:21,920 the teacher so that instructor gets what they essentially are turning in, but it 43 00:02:21,920 --> 00:02:25,640 has an audit report of the amount of time they spent on it, what they did with 44 00:02:25,640 --> 00:02:27,440 the AI, how they interacted with it. 45 00:02:27,680 --> 00:02:31,840 And I thought that was a pretty innovative way about thinking about how to embrace 46 00:02:31,880 --> 00:02:33,080 AI technologies. 47 00:02:33,080 --> 00:02:37,080 But I would say the one thing that really surprised me, and maybe this is just 48 00:02:37,080 --> 00:02:40,520 because I'm new to this space and this is just the way it goes. 49 00:02:40,720 --> 00:02:46,240 A lot of big companies here doing keynote presentations, panels, breakout rooms, 50 00:02:46,560 --> 00:02:49,440 across the board, and they have different tiers of sponsors, right? 51 00:02:49,760 --> 00:02:54,440 This is a very much a pay to play kind of game where there's a lot of 52 00:02:54,440 --> 00:02:59,320 presentations that work clearly for press release and shareholder benefit. 53 00:02:59,320 --> 00:03:02,960 So they could say, Hey, we presented at an AI conference. 54 00:03:03,040 --> 00:03:08,760 I have to say, as someone who worked in marketing for B2B, specifically for B2B 55 00:03:09,000 --> 00:03:13,480 trade shows, this kind of conferences, this is ginormous business. 56 00:03:13,680 --> 00:03:14,800 And this is how you go. 57 00:03:14,800 --> 00:03:20,080 I used to work with PR agency, with event agency, everything would be planned. 58 00:03:20,240 --> 00:03:23,920 And you put your CEO on stage with other panelists. 59 00:03:23,920 --> 00:03:29,360 So I have a spare share of skepticism towards conferences and especially at 60 00:03:29,360 --> 00:03:34,160 this scale, because at the end of the day, it's a big promo for companies. 61 00:03:34,320 --> 00:03:39,600 We would use to send our whole sales team to do networking, to connect with people. 62 00:03:39,880 --> 00:03:46,000 So it does what it has to do, but yeah, usually people pay heavy price to be 63 00:03:46,000 --> 00:03:47,680 there to promote their products. 64 00:03:47,840 --> 00:03:52,680 So let's just say that we were glad that we didn't have to pay thanks to this 65 00:03:52,680 --> 00:03:53,560 podcast. 66 00:03:53,560 --> 00:03:58,880 Aside from that kind of revelation that I guess news to me, but par for the 67 00:03:58,880 --> 00:04:01,160 course in this kind of world. 68 00:04:01,160 --> 00:04:04,600 And I've definitely experienced that applying to other conferences as either 69 00:04:04,600 --> 00:04:06,920 a speaker or for press purposes. 70 00:04:06,920 --> 00:04:10,320 They're like, you can sponsor us and then you can talk. 71 00:04:10,400 --> 00:04:13,920 It's like, what about the democratization of information? 72 00:04:13,960 --> 00:04:19,400 What about just good quality people talking about good quality AI, not tainted 73 00:04:19,400 --> 00:04:22,360 by the corporate shareholder? 74 00:04:22,360 --> 00:04:23,560 Oh yeah, I guess there's that. 75 00:04:23,720 --> 00:04:27,960 So I'm great networking opportunities, met some wonderful people and also too 76 00:04:27,960 --> 00:04:32,840 interesting to see some of the vendors on the floor that were there. 77 00:04:32,880 --> 00:04:36,880 You see your perennial big consulting firms, Deloitte. 78 00:04:37,040 --> 00:04:40,480 Deloitte's got a booth with couches and you could come sit and hang out and have 79 00:04:40,480 --> 00:04:40,880 a chat. 80 00:04:40,880 --> 00:04:44,920 It's not just like a folding table, like some of the startups, but I think the 81 00:04:44,920 --> 00:04:51,280 most interesting group of companies, and then I think this is a real emerging 82 00:04:51,280 --> 00:04:55,920 sector that if there's some startups in the space, keep track of them, there's a 83 00:04:55,920 --> 00:05:00,280 bunch of companies to do and handle error handling from language models. 84 00:05:00,320 --> 00:05:06,200 So to combat hallucinations, like anything from companies that have just a layer 85 00:05:06,200 --> 00:05:12,600 that gets inserted between the language model and the output to evaluate the 86 00:05:12,600 --> 00:05:17,960 different dashboards, but entire business processes built solely around making 87 00:05:17,960 --> 00:05:19,760 sure LLMs don't hallucinate. 88 00:05:19,760 --> 00:05:21,400 I'm so glad that you brought this up. 89 00:05:21,400 --> 00:05:28,480 I'm happy to share with listeners that I finally finished testing 432 A writing 90 00:05:28,480 --> 00:05:28,840 tools. 91 00:05:28,840 --> 00:05:29,600 Add person. 92 00:05:29,680 --> 00:05:35,360 I recorded myself like on 300, I think, because I was just like in a mess in a 93 00:05:35,360 --> 00:05:38,240 bun and I was just like, I'm losing my mind. 94 00:05:38,280 --> 00:05:41,400 First of all, I getting this, how to say it. 95 00:05:41,400 --> 00:05:46,160 I don't want to use negativity because that's another thing, like, but just 96 00:05:46,160 --> 00:05:52,080 starting despising some of the startups who just quickly spin off things, which 97 00:05:52,080 --> 00:05:52,840 are not good. 98 00:05:52,840 --> 00:05:56,080 So that's one that like just simple lack of effort. 99 00:05:56,120 --> 00:06:00,480 Another thing is like websites who are clear scams. 100 00:06:00,520 --> 00:06:05,240 I was just like in some, I tested, but there is some websites where I was like, 101 00:06:05,240 --> 00:06:08,160 I'm not giving even a fake email to you. 102 00:06:08,360 --> 00:06:14,560 Like, this is just not even scary that you got my IP address that I visited and 103 00:06:14,560 --> 00:06:18,000 bunch of them from the list also don't even exist anymore. 104 00:06:18,000 --> 00:06:18,280 Wow. 105 00:06:18,280 --> 00:06:20,880 So even with a week just vaporized. 106 00:06:21,040 --> 00:06:21,360 Yeah. 107 00:06:21,360 --> 00:06:25,720 So in the list, I marked the ones which website not existing anymore. 108 00:06:26,000 --> 00:06:28,160 And the others I had four categories. 109 00:06:28,160 --> 00:06:33,840 I had don't touch neutral and interesting and recommending. 110 00:06:34,200 --> 00:06:38,560 So for our listeners, this is go to release the video where she, because she's 111 00:06:38,560 --> 00:06:44,920 a crazy person is the only way to describe it tested 432 AI writing tools. 112 00:06:45,320 --> 00:06:49,000 Now there's probably, this is still just a drop in the bucket of what's out there. 113 00:06:49,000 --> 00:06:49,400 Right. 114 00:06:49,440 --> 00:06:54,680 The way I went about it, because there is many AI tools databases. 115 00:06:54,800 --> 00:06:59,240 So I went and checked different ones and to see how much we have 116 00:06:59,320 --> 00:07:02,040 GPT.E.AI had the most. 117 00:07:02,080 --> 00:07:07,760 So then I just used free web scraper, pulled all the data from there and then created. 118 00:07:07,760 --> 00:07:08,000 Yeah. 119 00:07:08,000 --> 00:07:12,480 In the beginning, Excel, but when I moved to Notion, it's nicer interface to me. 120 00:07:12,760 --> 00:07:14,120 And then just started going. 121 00:07:14,120 --> 00:07:18,200 And I remember after 80, I was like, I am exhausted. 122 00:07:18,440 --> 00:07:20,120 And this is 80. 123 00:07:20,160 --> 00:07:24,480 Like, of course, then I started getting a feel when I opened website. 124 00:07:24,480 --> 00:07:29,040 I've started getting, okay, I can immediately tell if it's something legit, if it's good. 125 00:07:29,280 --> 00:07:33,920 I have in my head, drilled the credit amounts, average pricing. 126 00:07:33,920 --> 00:07:40,040 So just after like hundreds, I opened pricing and I'm like, immediately can tell credit 127 00:07:40,040 --> 00:07:42,680 ratio, word ratio versus the price. 128 00:07:42,680 --> 00:07:44,000 And I'm like, this is bullshit. 129 00:07:44,000 --> 00:07:50,520 This is way too much because you would have a company charging $30 for 5,000 words. 130 00:07:50,760 --> 00:07:55,520 And then you have another company charging 15 for 10,000. 131 00:07:55,560 --> 00:08:00,520 It seems in most cases, like there is some randomness, I guess, with startups just go, 132 00:08:00,560 --> 00:08:01,080 we don't know. 133 00:08:01,160 --> 00:08:02,240 There is everything is new. 134 00:08:02,240 --> 00:08:04,160 So people pay, people don't know. 135 00:08:04,320 --> 00:08:09,080 I wanted to understand the space because we always say, oh, there's so many tools every 136 00:08:09,080 --> 00:08:09,560 day. 137 00:08:09,560 --> 00:08:15,400 And I was like, okay, but off top of my head, I know five to eight, which everyone uses, 138 00:08:15,640 --> 00:08:16,880 like including jester. 139 00:08:17,120 --> 00:08:22,720 And another thing I would just say that the reason they went for AI writing tools was 140 00:08:22,720 --> 00:08:30,960 because not everyone is using images or video for that sake or even 3D, but writing texts, 141 00:08:30,960 --> 00:08:36,640 everybody uses that in their business or at least in their daily life, just reading stories 142 00:08:36,640 --> 00:08:38,880 to their kids, a chagpy teammate. 143 00:08:39,120 --> 00:08:41,240 So there is a bunch of storybooks as well. 144 00:08:41,280 --> 00:08:48,240 If there is any McKinsey people here or Deloitte people, if you need an expert and AI writing 145 00:08:48,240 --> 00:08:53,200 tools, who from top of her head can immediately tell what's good, what's bad, you have to 146 00:08:53,200 --> 00:08:59,200 go very much in depth because I was just trying really, I don't want to say on a surface 147 00:08:59,200 --> 00:09:03,520 level, but just what is the UX, what is the pricing, what is the process, what is playing 148 00:09:03,520 --> 00:09:04,720 with a couple of prompts. 149 00:09:04,800 --> 00:09:10,480 I think in the specific use cases, I don't know, writing academic paper, like I can't 150 00:09:10,480 --> 00:09:16,320 physically test that, but like I can see already from everything that they put out that it 151 00:09:16,320 --> 00:09:17,280 could be solid. 152 00:09:17,520 --> 00:09:24,000 In some, I started also looking on what they are based on a lot of GPT-4, a lot of ones 153 00:09:24,000 --> 00:09:25,840 which promise a lot. 154 00:09:25,840 --> 00:09:30,960 And then you actually sign up, you give them email, you start the free trial and then you 155 00:09:30,960 --> 00:09:33,680 find out that, oh, it's coming, Claude is coming. 156 00:09:33,840 --> 00:09:34,160 Okay. 157 00:09:34,160 --> 00:09:36,880 So another thing that I did then I went through the whole thing. 158 00:09:36,880 --> 00:09:40,480 I took CSV file out of this, which I also recommend people to do. 159 00:09:40,640 --> 00:09:46,160 And I used Professor Synapse prompt, went into a code interpreter and was just like, 160 00:09:46,160 --> 00:09:47,280 let's analyze this data. 161 00:09:47,280 --> 00:09:48,320 Like, what do we get? 162 00:09:48,480 --> 00:09:55,440 Overall, you start seeing this like overarching teams like SEO, huge one, like so many people 163 00:09:55,440 --> 00:09:57,920 like so many companies doing SEO. 164 00:09:58,160 --> 00:10:03,040 General writing, I can immediately tell if a startup is just doing that, just basically 165 00:10:03,040 --> 00:10:08,080 if you're a startup and you're covering everything, you are both doing marketing stuff, 166 00:10:08,160 --> 00:10:13,440 or you also write essays, or you also write poems, immediately I would be like, that's 167 00:10:13,440 --> 00:10:19,200 where I've seen, oh my God, like 50 probably, if not more of that. 168 00:10:19,360 --> 00:10:24,000 And at some point they are exactly the same and not interesting at all. 169 00:10:24,000 --> 00:10:30,960 But on a flip side, the startups which actually discovered unique use cases, that what caught 170 00:10:30,960 --> 00:10:31,680 my eye. 171 00:10:31,680 --> 00:10:36,240 And then this is also what I asked Chai GPT, I was like, okay, let's analyze everything 172 00:10:36,240 --> 00:10:41,840 here from descriptions, from keywords, from pricing, from categories that they selected, 173 00:10:41,840 --> 00:10:45,200 which are the areas which could be explored. 174 00:10:45,200 --> 00:10:50,400 So for example, the one for patent writing, there is one startup, one startup who does 175 00:10:50,400 --> 00:10:50,960 that. 176 00:10:50,960 --> 00:10:51,520 Here you go. 177 00:10:51,520 --> 00:10:57,280 So if you are a startup and you're thinking, okay, let us just, we have access to Chai 178 00:10:57,280 --> 00:10:59,600 GPT, let's just push that. 179 00:10:59,600 --> 00:11:05,600 And then we do everything and we just put some prompts we got on internet in the background. 180 00:11:05,600 --> 00:11:09,920 Because you know, like after prompt a service and prompt perfect working with them, like 181 00:11:09,920 --> 00:11:13,440 I can tell that in the background where it's prompt perfect working. 182 00:11:13,440 --> 00:11:17,760 If we say that, okay, on the tools which are delivering a lot. 183 00:11:17,760 --> 00:11:21,680 So what I meant that startups, if they are just doing general writing for everything 184 00:11:21,680 --> 00:11:27,520 and you pay, you get a lot in that case, from all of them still Jasper seemed the best and 185 00:11:27,520 --> 00:11:34,640 they have this new function where you prompt for the use case and then it creates your 186 00:11:34,640 --> 00:11:35,680 interface. 187 00:11:35,680 --> 00:11:37,120 So you just fill in things. 188 00:11:37,120 --> 00:11:42,240 So for example, I couldn't find for value proposition and I was just like, oh, I want 189 00:11:42,240 --> 00:11:44,480 template for value proposition. 190 00:11:44,480 --> 00:11:48,400 Immediately boxes inserts this type of information, that type of information. 191 00:11:48,400 --> 00:11:49,280 That's all. 192 00:11:49,280 --> 00:11:51,600 And Jasper, like a fun tip to everyone. 193 00:11:51,600 --> 00:11:54,240 If you start the free trial, it's seven days. 194 00:11:54,240 --> 00:11:57,600 If you cancel that free trial, they offer you another seven days. 195 00:11:57,600 --> 00:12:02,560 If after that you cancel, they still will give you three months for free or something 196 00:12:02,560 --> 00:12:07,600 to go because I started so many free trials and this is like the issue now that I have 197 00:12:07,600 --> 00:12:09,200 to cancel everything. 198 00:12:09,200 --> 00:12:14,000 And I think what I also wanted to do with this, but people can see pricing can make 199 00:12:14,000 --> 00:12:14,560 a decision. 200 00:12:14,560 --> 00:12:15,520 What is the value? 201 00:12:15,520 --> 00:12:18,320 Like how much do you actually get for that price? 202 00:12:18,320 --> 00:12:24,480 But for unique use cases, I think it was called, yeah, it's film flow. 203 00:12:24,480 --> 00:12:29,440 So this real sound was super easy integration, like how it's done. 204 00:12:29,440 --> 00:12:31,840 It's with you work with open API. 205 00:12:31,840 --> 00:12:33,920 So, you know, he's not getting anything. 206 00:12:33,920 --> 00:12:40,720 It's completely for free, but it breaks down movies and their storylines with the sentiment 207 00:12:40,720 --> 00:12:41,280 analysis. 208 00:12:41,280 --> 00:12:46,880 So you get this visual and as someone who is making videos and like taps into learning 209 00:12:46,880 --> 00:12:52,320 about storytelling, how to engage audience, what is their emotions and plot lines. 210 00:12:52,320 --> 00:12:56,640 I found it super interesting and plus that it's in a way free. 211 00:12:56,640 --> 00:12:59,280 The guy is not collecting any money from that. 212 00:12:59,280 --> 00:13:01,840 So that was something I did not expect. 213 00:13:01,840 --> 00:13:07,840 Memoirs with AI, which you can write and it prints a book and it sends you emails about 214 00:13:07,840 --> 00:13:12,880 your family member and everyone kind of writes up and it pulls all together. 215 00:13:12,880 --> 00:13:16,880 Letters with AI, physical letters into your postbox. 216 00:13:17,440 --> 00:13:21,040 Only one was for spicy little short stories. 217 00:13:21,600 --> 00:13:27,440 And the reason I'm mentioning this, because you know how hard it is to overcome the rules 218 00:13:27,440 --> 00:13:29,600 and regulations put by open AI. 219 00:13:29,600 --> 00:13:35,120 So that we actually prompting in a way that you can still get a little bit of spiciness. 220 00:13:35,120 --> 00:13:39,120 Well, talking a little bit about tools, I wanted to share with the listeners a little 221 00:13:39,120 --> 00:13:43,440 bit of an extract from a conversation I had the pleasure of having with the CEOs from 222 00:13:43,440 --> 00:13:45,280 the healthcare company vital. 223 00:13:45,280 --> 00:13:52,320 Vital.io is chaired by Aaron Patzer, who was the founder of mint.com sold to Intuit for 224 00:13:52,320 --> 00:13:56,640 $170 million before he came around and his CTO as well, Felix Brown. 225 00:13:56,640 --> 00:14:02,160 They have a really truly breakthrough HIPAA compliant AI healthcare companion that basically 226 00:14:02,160 --> 00:14:03,520 improves patient outcomes. 227 00:14:03,520 --> 00:14:05,280 You can take a whole bunch of doctor's notes. 228 00:14:05,280 --> 00:14:07,600 It'll translate it into plain English. 229 00:14:07,600 --> 00:14:08,480 Simple as that. 230 00:14:08,480 --> 00:14:13,440 I encourage you to take a listen, give them a try vital.io slash translate such a revolutionary 231 00:14:13,440 --> 00:14:17,600 product and I think one of the first truly disruptors in the AI healthcare scene. 232 00:14:17,600 --> 00:14:23,760 Welcome to a joint episode of the prompt engineering podcast and the how to talk to AI podcast. 233 00:14:23,760 --> 00:14:25,200 We've got some awesome guests. 234 00:14:25,200 --> 00:14:27,200 So go ahead and introduce yourselves guys. 235 00:14:27,200 --> 00:14:31,280 Yeah, I'm Aaron Patzer, the co-founder and CEO of vital. 236 00:14:31,280 --> 00:14:34,480 And I'm Felix Brown, vice president of data science at vital. 237 00:14:34,480 --> 00:14:37,920 And they have a terrific product that they have a launch today. 238 00:14:37,920 --> 00:14:39,280 We're going to hear all about it. 239 00:14:39,280 --> 00:14:43,040 I think it's something that will resonate with every everyone and anyone that's been 240 00:14:43,040 --> 00:14:47,200 to the doctor and had questions about what was being told to them. 241 00:14:47,200 --> 00:14:49,600 Yes, I already tested it after watching your talk. 242 00:14:49,600 --> 00:14:49,840 Cool. 243 00:14:49,840 --> 00:14:50,240 Really? 244 00:14:50,240 --> 00:14:50,560 Yeah. 245 00:14:50,560 --> 00:14:52,080 So I have sleep apnea. 246 00:14:52,080 --> 00:14:57,360 I put in a long diagnosis with a bunch of stuff that I'm like, okay, I think I know what that is. 247 00:14:57,360 --> 00:14:58,560 I don't know what the hell that is. 248 00:14:58,560 --> 00:15:03,520 And it was like sleep apnea, it's obstructive and two other things. 249 00:15:03,520 --> 00:15:03,920 Yeah. 250 00:15:03,920 --> 00:15:05,200 Fantastic. 251 00:15:05,200 --> 00:15:05,840 Okay. 252 00:15:05,840 --> 00:15:06,560 That's great. 253 00:15:06,560 --> 00:15:06,960 Yeah. 254 00:15:06,960 --> 00:15:13,520 I think like we said, what person hasn't seen a whole long list of doctor's notes or even 255 00:15:14,080 --> 00:15:19,120 been in a situation where you're maybe in an inpatient in the hospital and then doctor 256 00:15:19,120 --> 00:15:22,640 on rounds is coming by and telling you something a million miles a minute because he's got 20 257 00:15:22,640 --> 00:15:26,720 other people to see, but it's probably the port because it affects your own health and being, 258 00:15:26,720 --> 00:15:30,480 and like you're probably already out of it anyway, cause you're in the hospital. 259 00:15:30,480 --> 00:15:32,320 What a terrific way to provide. 260 00:15:32,320 --> 00:15:35,920 Doctor's notes are really almost like a foreign language. 261 00:15:35,920 --> 00:15:36,160 Yeah. 262 00:15:36,800 --> 00:15:40,160 As I said in my talk, doctors don't say nosebleed. 263 00:15:40,160 --> 00:15:41,760 They say apestasis. 264 00:15:41,760 --> 00:15:44,320 They don't say, Hey, your mom has had a stroke. 265 00:15:44,320 --> 00:15:47,040 They say, Oh, she's had a cerebral infarction. 266 00:15:47,040 --> 00:15:48,720 They use all of these abbreviations. 267 00:15:48,720 --> 00:15:52,000 It's almost impossible to understand. 268 00:15:52,000 --> 00:15:58,720 And so we use a large language model as the core of what we call our doctor to patient 269 00:15:58,720 --> 00:15:59,280 translator. 270 00:15:59,280 --> 00:16:04,560 And it's at vital.io slash translates free to the public available worldwide. 271 00:16:04,560 --> 00:16:08,000 Literally as of today, you're just catching me at a good time. 272 00:16:08,000 --> 00:16:13,280 And we're happy to tell you a bit about the prompts, the classifiers, the pre-parsing, 273 00:16:13,280 --> 00:16:16,000 all the things that happen to make that possible technically. 274 00:16:16,000 --> 00:16:18,560 I would love to delve into some of the technical aspects. 275 00:16:18,560 --> 00:16:19,920 Maybe this better question for Felix. 276 00:16:19,920 --> 00:16:23,280 Could you tell us a little bit about how the model was maybe trained and what data was 277 00:16:23,280 --> 00:16:25,680 used to be able to produce these great completions? 278 00:16:25,680 --> 00:16:26,000 Sure. 279 00:16:26,000 --> 00:16:28,720 We've tried a number of different prompts because there are actually a lot of different 280 00:16:28,720 --> 00:16:29,920 types of doctor's notes. 281 00:16:29,920 --> 00:16:33,440 And with the public facing stuff, we know that we're going to get the whole gamut from 282 00:16:33,440 --> 00:16:35,760 imaging all the way to discharge instructions. 283 00:16:35,760 --> 00:16:38,240 And we know how important that stuff is. 284 00:16:38,240 --> 00:16:41,840 There's a whole bunch of literature people when they get their paper discharge 285 00:16:41,840 --> 00:16:45,680 instructions, upwards of 90% of them, Chuck them straight in the bin as soon as they 286 00:16:45,680 --> 00:16:46,640 leave the hospital. 287 00:16:46,640 --> 00:16:51,600 And the literature is super clear when people understand their care and understand the 288 00:16:51,600 --> 00:16:55,760 follow-up instructions the doctors are giving them, their care situation is way, way better. 289 00:16:55,760 --> 00:16:56,320 Makes sense. 290 00:16:56,320 --> 00:17:01,840 So we've looked at different prompts for different situations and then built a pre-model 291 00:17:01,840 --> 00:17:07,840 classifier, pre-LLM classifier, also using a language model with a smaller one, deciding 292 00:17:07,840 --> 00:17:11,680 which of our various prompts should be applied to the notes, so categorizing the notes. 293 00:17:11,680 --> 00:17:14,880 And then we have a whole bunch of post-parsing that comes out. 294 00:17:14,880 --> 00:17:17,680 We take sections out of translation. 295 00:17:17,680 --> 00:17:20,080 We plug those sections into different parts of the website. 296 00:17:20,080 --> 00:17:24,400 Maybe when you saw it, you could see that you get like a very brief summary and then 297 00:17:24,400 --> 00:17:26,960 also a much more sort of technical breakdown. 298 00:17:26,960 --> 00:17:27,280 Yes. 299 00:17:27,280 --> 00:17:32,000 So we're getting the LLM to pull out a lot of information about what's in your doctor's 300 00:17:32,000 --> 00:17:34,400 note, but we want to show you in like a digestible summary first. 301 00:17:34,400 --> 00:17:34,800 Yeah. 302 00:17:34,800 --> 00:17:38,960 I think an important piece of context is a lot of these doctor's notes, they're 10 or 303 00:17:38,960 --> 00:17:42,560 15 pages long and they have 80% boilerplate. 304 00:17:42,560 --> 00:17:45,280 They have, hey, don't smoke, or I don't. 305 00:17:45,280 --> 00:17:46,560 Hey, here's COVID education. 306 00:17:46,560 --> 00:17:48,000 Okay, you're two years out of date. 307 00:17:48,000 --> 00:17:50,400 And they put a lot of filler in there. 308 00:17:50,400 --> 00:17:52,880 And this is actually just a fraction of our primary business. 309 00:17:52,880 --> 00:17:55,520 Our primary business is patient experience software. 310 00:17:55,520 --> 00:18:00,320 It guides you through an ER visit or if you have to stay overnight in the hospital, explains 311 00:18:00,320 --> 00:18:04,560 your lab results, how long you're going to wait, and then what your notes mean. 312 00:18:05,360 --> 00:18:08,960 And it's because we have the experience with a million patients that you're using it, 313 00:18:08,960 --> 00:18:11,760 we know the structure of note from all over the country. 314 00:18:11,760 --> 00:18:17,680 And so we can pre-parse and instead of a 10 or 15 page, we can get it down to actually, 315 00:18:17,680 --> 00:18:20,880 we only need to pass three or four pages into the LLM. 316 00:18:20,880 --> 00:18:28,000 That's an important business and engineering consideration because cost and speed, also 317 00:18:28,000 --> 00:18:29,280 context window. 318 00:18:29,280 --> 00:18:33,920 If you're doing, especially if you're using few shot training with an LLM, which is a 319 00:18:33,920 --> 00:18:37,120 good idea so that you know what output you want to get. 320 00:18:37,120 --> 00:18:42,640 You'll blow through your prompt, your few shot, your data, and then your output has 321 00:18:42,640 --> 00:18:45,680 to fit into a 4K window or a 16K window. 322 00:18:46,400 --> 00:18:52,320 And so you need to do a few things to give yourself as much buffer as possible beforehand. 323 00:18:52,320 --> 00:18:52,720 Yeah. 324 00:18:52,720 --> 00:18:58,320 That makes complete sense with having the almost sub prompts acting like little sub 325 00:18:58,320 --> 00:19:03,680 agents themselves trained to say, just get rid of all the boilerplate stuff. 326 00:19:03,680 --> 00:19:07,280 That's not unique to that patient's differential diagnosis. 327 00:19:07,280 --> 00:19:07,840 Exactly. 328 00:19:07,840 --> 00:19:13,280 So deciding which part you're going to do more or less with your own code or your own 329 00:19:13,280 --> 00:19:17,680 classifiers and then how much to send, especially if you're using a commercial LLM. 330 00:19:17,680 --> 00:19:22,320 And we've used both Felix's got llama up and running and llama two. 331 00:19:22,320 --> 00:19:22,800 Yeah. 332 00:19:22,800 --> 00:19:28,160 Medpalm LLM from Google, which is medical specific, obviously the open AI. 333 00:19:28,160 --> 00:19:30,240 We can't actually use open AI directly. 334 00:19:30,240 --> 00:19:35,120 You have to use it through Azure because you need this to be HIPAA compliant, 335 00:19:35,120 --> 00:19:37,840 so we're in a regulated industry. 336 00:19:37,840 --> 00:19:40,080 Open AI will not sign all of those things. 337 00:19:40,080 --> 00:19:44,080 You actually have to like work your way through corporate Microsoft. 338 00:19:44,080 --> 00:19:44,400 Yep. 339 00:19:44,400 --> 00:19:48,080 They'll determine whether you're a worthwhile person or not and whether they're willing 340 00:19:48,080 --> 00:19:48,880 to take the risk. 341 00:19:48,880 --> 00:19:50,480 And then you have to sign these contracts. 342 00:19:50,480 --> 00:19:54,560 And so if you put all of it, you can with a sophisticated prompt, put it all through 343 00:19:54,560 --> 00:19:55,360 like a GPT. 344 00:19:55,360 --> 00:19:56,800 You can say classify this. 345 00:19:56,800 --> 00:19:58,320 Is this a discharge record? 346 00:19:58,320 --> 00:20:02,640 Is this a physical therapy report or is this a hostile input? 347 00:20:02,640 --> 00:20:05,040 By the way, you should always protect against hostile input. 348 00:20:05,040 --> 00:20:07,040 Is this a non-English input? 349 00:20:07,040 --> 00:20:08,560 Is this something else entirely? 350 00:20:08,560 --> 00:20:13,040 So you want, and then in your prompt, you can say based on the classification, then 351 00:20:13,040 --> 00:20:14,240 do this. 352 00:20:14,240 --> 00:20:17,680 But if you do all that, your prompt starts to get very complicated and very big. 353 00:20:17,680 --> 00:20:19,440 You can use that to prototype. 354 00:20:19,440 --> 00:20:22,240 But when you go into production, this is also very slow. 355 00:20:22,240 --> 00:20:23,360 It's very expensive. 356 00:20:23,360 --> 00:20:27,520 You run a classifier that's much simpler and much quicker on top of it. 357 00:20:27,520 --> 00:20:30,160 And then you don't have the expense, your prompt's shorter. 358 00:20:30,160 --> 00:20:32,480 And then you can say, if it's this, go to this prompt. 359 00:20:32,480 --> 00:20:33,760 If it's that, go to that prompt. 360 00:20:33,760 --> 00:20:34,080 Yeah. 361 00:20:34,080 --> 00:20:36,080 You can also templatize prompts. 362 00:20:36,080 --> 00:20:40,800 So if you say, I want the output in Spanish, you can put a variable in your prompt. 363 00:20:40,800 --> 00:20:43,760 So the prompts don't think of them as static strings. 364 00:20:43,760 --> 00:20:48,720 Think of them as a programming language that is frankly pseudo code. 365 00:20:48,720 --> 00:20:49,040 Yeah. 366 00:20:49,040 --> 00:20:53,280 One of the things that this is a bit like medical specific, but the part that's very 367 00:20:53,280 --> 00:20:55,840 important to patients is the plan and assessment. 368 00:20:55,840 --> 00:20:57,680 What the doctor says you're supposed to do. 369 00:20:57,680 --> 00:20:58,480 Here's the problem. 370 00:20:58,480 --> 00:21:01,040 At some hospitals, it's called plan and assessment. 371 00:21:01,040 --> 00:21:02,720 At other hospitals, it's called assessment. 372 00:21:02,720 --> 00:21:04,080 At other hospitals, it's called plan. 373 00:21:04,080 --> 00:21:06,320 At other hospitals, it's got like an abbreviation. 374 00:21:06,320 --> 00:21:11,280 And with classic programming, if I say match Panda and I give it Pandas with a plural, 375 00:21:11,280 --> 00:21:11,680 it's a no. 376 00:21:11,680 --> 00:21:14,240 The math's already got a space in your column header. 377 00:21:14,240 --> 00:21:15,040 Exactly. 378 00:21:15,040 --> 00:21:19,520 But with an LM, I can just be like, it's going to be called this or probably this. 379 00:21:19,520 --> 00:21:21,680 It's got stuff that kind of looks like this. 380 00:21:21,680 --> 00:21:22,080 Yeah. 381 00:21:22,080 --> 00:21:24,720 And like, it's good enough that if I explained it to you guys, you'd be like, oh, 382 00:21:24,720 --> 00:21:25,920 okay, I know what you're looking for. 383 00:21:26,480 --> 00:21:31,680 That's the power of LLMs is you can give them vague pseudo code. 384 00:21:31,680 --> 00:21:32,080 Yeah. 385 00:21:32,080 --> 00:21:32,400 Yeah. 386 00:21:32,400 --> 00:21:33,920 And to me, that's mind blowing. 387 00:21:33,920 --> 00:21:36,640 This guy actually knows the math behind how that's possible. 388 00:21:36,640 --> 00:21:37,280 Nice. 389 00:21:37,280 --> 00:21:42,480 So real quick, before we get into that, just for the audience, part of what I do on my podcast is 390 00:21:42,480 --> 00:21:44,720 like, what are all these technical terms? 391 00:21:44,720 --> 00:21:48,560 Content window, number one, is literally how much stuff you're putting into the prompt, 392 00:21:48,560 --> 00:21:50,480 but also how much it's putting out. 393 00:21:50,480 --> 00:21:54,320 And if you do too much, it forgets the stuff outside the prompt window. 394 00:21:54,320 --> 00:21:55,360 Sorry, the context window. 395 00:21:55,920 --> 00:21:58,320 And so you have to be careful how long everything is. 396 00:21:58,320 --> 00:22:01,760 That's what they're talking about when you're saying, if I can pull pieces of the prompt out 397 00:22:01,760 --> 00:22:04,480 and only run them separately, it's way better. 398 00:22:04,480 --> 00:22:08,400 It's a key reason to innovate on your own models because for a long time, 399 00:22:08,400 --> 00:22:10,400 you were working with this 4K context window. 400 00:22:10,400 --> 00:22:15,200 And if you're doing this few short in-context learning, as Aaron says, you just run through it. 401 00:22:15,200 --> 00:22:15,920 Yeah. 402 00:22:15,920 --> 00:22:20,560 And also I'm the CEO as well as maybe you can tell I have a bit of an engineering background, 403 00:22:20,560 --> 00:22:22,240 not as good as this guy. 404 00:22:22,240 --> 00:22:24,640 I don't have the British accent with this. 405 00:22:24,640 --> 00:22:25,280 That's true. 406 00:22:25,280 --> 00:22:27,760 It also, that adds 20, I came for the grant. 407 00:22:27,760 --> 00:22:29,360 We're here for the grant. 408 00:22:29,360 --> 00:22:29,840 Yes. 409 00:22:29,840 --> 00:22:32,640 But as a CEO, I have to think through the economics, right? 410 00:22:32,640 --> 00:22:39,200 If you were using GPT-4 and you give it the 16K window or the 32K window, the maximum one, 411 00:22:39,200 --> 00:22:44,320 it's going to cost you, if you fully fill that, it's going to cost you about 48 cents per 412 00:22:44,320 --> 00:22:46,080 translation or transformation, right? 413 00:22:46,080 --> 00:22:46,640 Yeah. 414 00:22:46,640 --> 00:22:48,800 We have a million patients on our platform. 415 00:22:48,800 --> 00:22:50,800 They have about five notes each. 416 00:22:50,800 --> 00:22:57,440 If you do the math on that and you're spending $5,000 a day, if that's what you do, you don't 417 00:22:57,440 --> 00:23:03,920 need to, you use smaller context windows or you use 3.5 turbo or you run llama, or you 418 00:23:03,920 --> 00:23:07,200 use one LLM to pre-part for a different LLM. 419 00:23:07,200 --> 00:23:11,440 You can do those are the tricks that like practically speaking, this is an immature 420 00:23:11,440 --> 00:23:14,640 industry because you have to hand do all of that. 421 00:23:14,640 --> 00:23:15,040 Yeah. 422 00:23:15,040 --> 00:23:18,720 And what's really interesting is some of these problems are really exciting and new. 423 00:23:18,720 --> 00:23:24,080 As Aaron says, you're trying to pull out something that's very undefined in free text document. 424 00:23:24,080 --> 00:23:24,320 Okay. 425 00:23:24,320 --> 00:23:28,000 So that's, you need some modern stuff to do that, but some of these problems are pretty 426 00:23:28,000 --> 00:23:28,800 traditional. 427 00:23:28,800 --> 00:23:32,000 Classifying a document and you've got plenty of examples. 428 00:23:32,000 --> 00:23:36,160 You don't need to go and use your OpenAI LLM to do this classification problem. 429 00:23:36,160 --> 00:23:38,960 We've been doing this for a long time and you can do them a lot cheaper. 430 00:23:38,960 --> 00:23:39,120 Yeah. 431 00:23:39,120 --> 00:23:46,160 It's slow and expensive to use OpenAI or Google or Meta for basic classification, 432 00:23:46,160 --> 00:23:47,760 but it's great for prototyping. 433 00:23:47,760 --> 00:23:52,480 So the key insight is work out the piece that you really need the expensive tech for and 434 00:23:52,480 --> 00:23:57,040 ensure that you boil down the problem only to that using other pieces of technology upstream. 435 00:23:57,040 --> 00:23:57,600 Yeah. 436 00:23:57,600 --> 00:23:58,080 Yeah. 437 00:23:58,080 --> 00:24:03,440 So how do you handle like the, if you have all these prompts essentially acting as agents 438 00:24:03,440 --> 00:24:08,720 and you have to have this sequence occur in a specific order, how do you asynchronously, 439 00:24:08,720 --> 00:24:11,360 is there a specific layer that's doing the handoff? 440 00:24:11,360 --> 00:24:15,840 Are they doing the, are they doing a turnover at rounds between these prompts? 441 00:24:15,840 --> 00:24:16,800 New text synchronization. 442 00:24:16,800 --> 00:24:16,960 Yeah. 443 00:24:16,960 --> 00:24:18,000 I mean, a little technical. 444 00:24:18,000 --> 00:24:18,400 Yeah. 445 00:24:18,400 --> 00:24:20,800 So we use an event sourced architecture. 446 00:24:20,800 --> 00:24:20,960 Yeah. 447 00:24:20,960 --> 00:24:24,960 So this is outside of AI, basically means that we handle streaming data quite well. 448 00:24:25,520 --> 00:24:29,600 So we have data that's streaming from over a hundred hospitals now, 449 00:24:29,600 --> 00:24:30,560 more or less real time. 450 00:24:30,560 --> 00:24:34,640 It comes out of Cerner, Epic, whatever the electronic medical record system is. 451 00:24:34,640 --> 00:24:39,840 So a doctor writes a new note, finishes it, it hits our system and goes onto the queue, 452 00:24:40,560 --> 00:24:46,160 gets pre-parsed, classified, cut up into little bits and then divvied out to these, 453 00:24:46,160 --> 00:24:47,200 these little agents. 454 00:24:54,080 --> 00:24:59,520 As always, you can check out the show notes and links at howtotalk2.ai. 455 00:25:00,400 --> 00:25:01,920 That's all for this week's episode. 456 00:25:01,920 --> 00:25:16,320 Happy prompting everyone.