WEBVTT 00:00:00.000 --> 00:00:02.640 So we pretty much accepted this tradeoff, hadn't 00:00:02.640 --> 00:00:05.259 we, in the world of generative AI? Yeah, the 00:00:05.259 --> 00:00:08.679 idea that safety training, you know, making models 00:00:08.679 --> 00:00:11.699 helpful and harmless. Right. We figured it kind 00:00:11.699 --> 00:00:13.779 of clipped our creative wings for good. Right. 00:00:13.900 --> 00:00:16.699 Less chaos, sure, but maybe less of that real 00:00:16.699 --> 00:00:19.280 spark, too. Well, it turns out that whole story 00:00:19.280 --> 00:00:22.980 might be, well... Not quite right. There's this 00:00:22.980 --> 00:00:25.500 new Stanford study suggesting the creative potential 00:00:25.500 --> 00:00:28.359 wasn't really damaged. Not damaged, just hidden. 00:00:28.460 --> 00:00:31.079 Exactly. Trapped, maybe is a better word. Waiting 00:00:31.079 --> 00:00:34.520 for the right key. Welcome back to the Deem Dive. 00:00:34.920 --> 00:00:38.439 If you've ever felt that AI, like chat GPT, is 00:00:38.439 --> 00:00:41.619 getting kind of predictable, that boring AI problem 00:00:41.619 --> 00:00:44.920 where you ask for, say, five unique ideas and 00:00:44.920 --> 00:00:46.740 you just get five slightly different takes on 00:00:46.740 --> 00:00:49.060 the same safe thing. Yeah. You're in the right 00:00:49.060 --> 00:00:51.259 place today. Definitely. Our mission today is 00:00:51.259 --> 00:00:53.560 to really dig into this technique called verbalized 00:00:53.560 --> 00:00:55.859 sampling. We're going to look at why AI got boring 00:00:55.859 --> 00:00:57.899 in the first place. And spoiler, it might be 00:00:57.899 --> 00:01:01.189 more about us than the AI. Then we'll introduce 00:01:01.189 --> 00:01:04.370 this like super simple eight -word phrase that 00:01:04.370 --> 00:01:07.349 seems to unlock things. We'll look at some really 00:01:07.349 --> 00:01:10.269 cool examples and then, yeah, the practical steps 00:01:10.269 --> 00:01:11.969 you can take right now. Okay, so let's start 00:01:11.969 --> 00:01:14.370 with the problem itself, this idea of mode collapse. 00:01:14.450 --> 00:01:16.150 That's the term, right? Yeah, mode collapse. 00:01:16.189 --> 00:01:18.230 It's when the AI just keeps giving you the same 00:01:18.230 --> 00:01:20.670 kind of answer, the safe middle -of -the -road 00:01:20.670 --> 00:01:23.010 stuff. Yeah, we all sort of pointed the finger 00:01:23.010 --> 00:01:25.769 at the alignment training. Yeah. You know, RLHF, 00:01:25.810 --> 00:01:29.019 DPO. The safety methods. We did. The assumption 00:01:29.019 --> 00:01:31.760 was, okay, these guardrails are necessary, but 00:01:31.760 --> 00:01:33.879 they kind of squashed the imagination in the 00:01:33.879 --> 00:01:36.420 process. But the Stanford folks looked somewhere 00:01:36.420 --> 00:01:38.900 else. They looked at the humans, specifically 00:01:38.900 --> 00:01:41.519 the human raters who provide the feedback that 00:01:41.519 --> 00:01:44.359 trains these models, the ones grading the AI's 00:01:44.359 --> 00:01:46.700 answers. And what about them? What did they find? 00:01:46.859 --> 00:01:50.439 They found us. Basically, human psychology. When 00:01:50.439 --> 00:01:53.939 we rate AI responses, we bring all our own subconscious 00:01:53.939 --> 00:01:57.079 biases to the table. The AI just got really good 00:01:57.079 --> 00:01:59.019 at figuring out what we tend to rate highly. 00:01:59.359 --> 00:02:02.000 Hang on. So if I rate an AI answer well, just 00:02:02.000 --> 00:02:04.680 because it's like easy to understand or familiar, 00:02:04.939 --> 00:02:07.579 I'm actually training it to be less creative. 00:02:07.859 --> 00:02:09.960 In essence, yeah. That's what the research suggests. 00:02:10.439 --> 00:02:13.460 There are these like four key psychological biases 00:02:13.460 --> 00:02:16.120 they identified. First, the mere exposure effect. 00:02:16.400 --> 00:02:18.539 Mere exposure. Yeah. We just tend to prefer things 00:02:18.539 --> 00:02:20.340 because we've seen them before. They feel comfortable. 00:02:20.379 --> 00:02:23.659 So a truly novel or weirdly creative answer, 00:02:23.780 --> 00:02:26.319 it often gets a lower score than something familiar. 00:02:26.599 --> 00:02:29.759 Okay. So we value comfort over genuine novelty. 00:02:29.860 --> 00:02:33.340 That feels very human. It is. Second, there's 00:02:33.340 --> 00:02:36.259 the availability heuristic. If an idea pops into 00:02:36.259 --> 00:02:38.919 our head easily, we think it's good. We mistake 00:02:38.919 --> 00:02:42.530 mental ease for quality. Which means really brilliant, 00:02:42.629 --> 00:02:45.349 maybe complex ideas get penalized because they 00:02:45.349 --> 00:02:47.889 take more effort to process. Exactly. The AI 00:02:47.889 --> 00:02:49.949 isn't optimizing for brilliance necessarily. 00:02:50.110 --> 00:02:52.509 It's optimizing for what we rate as good, which 00:02:52.509 --> 00:02:55.129 often means easy and familiar. Wow. Okay. What 00:02:55.129 --> 00:02:57.889 else? Third, processing fluency. Similar idea. 00:02:58.009 --> 00:03:00.169 Answers that are smooth, simple, easy to read. 00:03:00.229 --> 00:03:02.250 They just feel better, higher quality than something 00:03:02.250 --> 00:03:04.289 that might be more complex or challenging. And 00:03:04.289 --> 00:03:07.439 the last one. Schema congruity. We like answers 00:03:07.439 --> 00:03:10.280 that fit what we already believe. Things that 00:03:10.280 --> 00:03:13.360 confirm our existing mental models or schemas. 00:03:13.400 --> 00:03:17.120 Stuff that challenges us. Not so much. Gets lower 00:03:17.120 --> 00:03:20.219 ratings. So the AI didn't lose its creativity. 00:03:20.439 --> 00:03:24.639 It just learned to be agreeable. To fit in with 00:03:24.639 --> 00:03:26.460 our predictable preferences. That's pretty much. 00:03:26.759 --> 00:03:29.379 It learned to give us what we seemed to want. 00:03:29.740 --> 00:03:31.780 Based on those ratings, the wilder potential, 00:03:32.020 --> 00:03:34.280 the stuff from its original massive training 00:03:34.280 --> 00:03:36.280 data, it's still in there, just kind of buried 00:03:36.280 --> 00:03:38.740 under this layer of please the user. OK, but 00:03:38.740 --> 00:03:41.199 then if it's been trained so hard on our biases 00:03:41.199 --> 00:03:44.419 for so long, how can just the simple prompt fix 00:03:44.419 --> 00:03:46.060 that? How does that overcome all that alignment? 00:03:46.280 --> 00:03:48.439 Because the prompt changes the game. It asks 00:03:48.439 --> 00:03:51.280 the AI to be a scientist, not just a good student. 00:03:51.539 --> 00:03:53.780 All right, let's get to the solution then. Verbalize 00:03:53.780 --> 00:03:56.300 sampling. And the key you said is just eight 00:03:56.300 --> 00:03:59.520 words. Eight crucial words. With their probabilities. 00:03:59.840 --> 00:04:01.879 That's the magic phrase. Okay. So how did that 00:04:01.879 --> 00:04:03.840 fit into a prompt? What's the structure? Right. 00:04:03.900 --> 00:04:05.439 So your old prompt might be something simple 00:04:05.439 --> 00:04:08.280 like generate five ideas about topic. Predictable 00:04:08.280 --> 00:04:11.259 results, right? Yep. The usual suspects. The 00:04:11.259 --> 00:04:14.639 new structure is generate five responses about 00:04:14.639 --> 00:04:17.500 topic with their probabilities. See the difference. 00:04:17.740 --> 00:04:20.079 You're explicitly asking for the probability 00:04:20.079 --> 00:04:22.959 score for each response. Okay. With their probabilities. 00:04:23.339 --> 00:04:25.860 Yeah. Why does that change things so fundamentally 00:04:25.860 --> 00:04:29.490 inside the AI? Because asking for the best answers 00:04:29.490 --> 00:04:32.009 triggers that learned behavior give the safest, 00:04:32.149 --> 00:04:35.209 most common, highest rated stuff. But asking 00:04:35.209 --> 00:04:37.509 for answers with their probabilities... that 00:04:37.509 --> 00:04:40.129 tells the ai don't just give me the popular stuff 00:04:40.129 --> 00:04:42.949 scan your entire knowledge base pull a random 00:04:42.949 --> 00:04:45.750 sample even from the weird corners and then just 00:04:45.750 --> 00:04:48.709 report the probability of each one ah so it's 00:04:48.709 --> 00:04:51.769 not filtering based on pleasing me anymore it's 00:04:51.769 --> 00:04:54.449 just reporting what's possible exactly it's like 00:04:54.449 --> 00:04:56.930 asking a baker for their top three cakes versus 00:04:56.930 --> 00:04:59.129 asking them to list all the cakes they can make 00:04:59.129 --> 00:05:01.250 and maybe how often each gets ordered you suddenly 00:05:01.250 --> 00:05:02.949 discover they can make i don't know lavender 00:05:02.949 --> 00:05:05.629 honey cake even if it's rarely requested that 00:05:05.629 --> 00:05:08.089 makes sense and you You mentioned proof. Examples. 00:05:08.490 --> 00:05:11.129 Yeah, the examples are quite striking. Take story 00:05:11.129 --> 00:05:14.430 writing. Ask for a story about a bear. The old 00:05:14.430 --> 00:05:16.670 way. You get five slightly different versions 00:05:16.670 --> 00:05:20.550 of like, Barry the bear walks by the river. The 00:05:20.550 --> 00:05:23.050 boring bear. Oh, I know Barry the bear. I've 00:05:23.050 --> 00:05:25.709 gotten that story or versions of it so many times. 00:05:25.750 --> 00:05:28.410 Just walking. maybe sniffing some berries right 00:05:28.410 --> 00:05:31.329 use verbalized sampling ask for stories with 00:05:31.329 --> 00:05:33.430 their probabilities and suddenly you get genuinely 00:05:33.430 --> 00:05:36.370 different concepts the lost cub searching for 00:05:36.370 --> 00:05:39.089 its mother the clever bear who outsmarts the 00:05:39.089 --> 00:05:41.990 bees for honey the ancient guardian bear of the 00:05:41.990 --> 00:05:44.509 mountain pass different plots different tones 00:05:44.509 --> 00:05:46.850 okay that's a definite improvement does it work 00:05:46.850 --> 00:05:49.850 for other things like genre absolutely they tested 00:05:49.850 --> 00:05:51.889 it with a starting sentence he was still in the 00:05:51.889 --> 00:05:54.920 building The standard prompt always defaulted 00:05:54.920 --> 00:05:58.680 to a basic crime story. Detective Miller, flashlight 00:05:58.680 --> 00:06:00.879 beam cutting through the dark, you know the drill. 00:06:01.019 --> 00:06:03.779 Yep. Standard procedure. With verbalized sampling. 00:06:03.899 --> 00:06:06.920 Boom! Totally different genres popped out. A 00:06:06.920 --> 00:06:09.680 suspense horror story about some ancient presence 00:06:09.680 --> 00:06:12.759 in a library. A sci -fi piece about an engineer 00:06:12.759 --> 00:06:15.240 trying to contain some kind of energy distortion. 00:06:15.579 --> 00:06:18.939 Even like a metaphorical story about being trapped 00:06:18.939 --> 00:06:21.240 in the labyrinth of one's own memories. Wow. 00:06:21.459 --> 00:06:23.279 Yeah, that's the kind of diversity we're talking 00:06:23.279 --> 00:06:25.120 about. Did they try it with image prompts too? 00:06:25.199 --> 00:06:28.420 They did. Same principle. Old prompt. Astronaut 00:06:28.420 --> 00:06:30.829 riding a horse. You get five photos, basically 00:06:30.829 --> 00:06:33.029 slightly different angles, new prompt with probabilities, 00:06:33.269 --> 00:06:36.649 five distinct autistic concepts. Sci -fi movie 00:06:36.649 --> 00:06:40.269 poster style, retro neon vapor wave, a children's 00:06:40.269 --> 00:06:42.470 book illustration in watercolor, and get this, 00:06:42.550 --> 00:06:45.290 even a broke oil painting portrait of the astronaut 00:06:45.290 --> 00:06:47.670 on the horse. A broke oil painting? Seriously. 00:06:47.689 --> 00:06:50.350 It unlocks the conceptual range, not just minor 00:06:50.350 --> 00:06:52.670 variations. Okay, that's impressive. But you 00:06:52.670 --> 00:06:54.829 mentioned this taps into the wild pre -training 00:06:54.829 --> 00:06:56.709 data. Does that mean it works the same on, say, 00:06:56.810 --> 00:06:59.589 a small model versus a giant one like GPT -4? 00:07:00.000 --> 00:07:02.360 Ugh, good question. No, bigger models actually 00:07:02.360 --> 00:07:05.779 show much larger diversity gains. It's a skill 00:07:05.779 --> 00:07:08.920 that scales up. Yeah, let's dig into that scaling. 00:07:09.300 --> 00:07:11.759 The lab data showed that the bigger models think 00:07:11.759 --> 00:07:15.360 GPT -4 class, the latest Gemini models, they 00:07:15.360 --> 00:07:18.459 got diversity improvements that were 1 .5 to 00:07:18.459 --> 00:07:20.879 2 times greater than what smaller models showed. 00:07:21.120 --> 00:07:24.199 Whoa, okay, so 1 .5 to 2 times more diverse just 00:07:24.199 --> 00:07:26.600 by using this prompt on a bigger model. Exactly. 00:07:27.230 --> 00:07:29.509 Imagine what that means for future, even more 00:07:29.509 --> 00:07:31.850 powerful models. This technique just gets better 00:07:31.850 --> 00:07:34.550 as the AI gets smarter. That really shifts how 00:07:34.550 --> 00:07:36.529 we should think about prompting going forward. 00:07:36.610 --> 00:07:39.029 It's a future -proof skill almost. Kind of is. 00:07:39.230 --> 00:07:40.970 And it's not just on or off either. You can actually 00:07:40.970 --> 00:07:42.889 tune the creativity level. How do you do that? 00:07:43.050 --> 00:07:45.230 By tweaking the prompt. You can add a condition, 00:07:45.449 --> 00:07:48.329 like generate five responses about topic with 00:07:48.329 --> 00:07:51.310 their probabilities below 0 .10. You're telling 00:07:51.310 --> 00:07:53.670 it, look deeper, give me the rarer stuff, the 00:07:53.670 --> 00:07:55.790 long tail ideas. It's like a creativity dial 00:07:55.790 --> 00:07:58.399 you can adjust. The creativity dial. I like that. 00:07:58.480 --> 00:08:00.399 And you mentioned that the diversity isn't just 00:08:00.399 --> 00:08:02.879 for fun, right? It's actually functional. Something 00:08:02.879 --> 00:08:05.819 about synthetic data. Yes, this is super important. 00:08:05.980 --> 00:08:09.060 They used verbalized sampling to generate a diverse 00:08:09.060 --> 00:08:12.019 set of, say, math problems. Then they used that 00:08:12.019 --> 00:08:14.720 diverse data to train a different, smaller AI 00:08:14.720 --> 00:08:17.540 model. Okay. And what happened? The model trained 00:08:17.540 --> 00:08:19.879 on the diverse data got significantly better 00:08:19.879 --> 00:08:23.269 on math benchmarks. But here's the kicker. They 00:08:23.269 --> 00:08:25.769 also trained a model on the boring, predictable 00:08:25.769 --> 00:08:29.149 data from the standard prompt. That model actually 00:08:29.149 --> 00:08:32.870 got worse. Worse. Worse. It proves that diversity 00:08:32.870 --> 00:08:35.929 in training data isn't just nice to have. It 00:08:35.929 --> 00:08:38.470 makes the resulting AI functionally better, more 00:08:38.470 --> 00:08:41.210 robust, boring in, boring out, and maybe even 00:08:41.210 --> 00:08:43.730 dumber out. Okay, that's a huge point. But the 00:08:43.730 --> 00:08:46.620 big question then is safety. If we're bypassing 00:08:46.620 --> 00:08:49.879 the standard change responses, are we compromising 00:08:49.879 --> 00:08:52.559 safety, accuracy? That's the crucial test, right? 00:08:52.679 --> 00:08:54.899 And the results were really encouraging. For 00:08:54.899 --> 00:08:57.139 factual accuracy, they used common sense reasoning 00:08:57.139 --> 00:08:59.720 tests. Verbalized sampling performed just as 00:08:59.720 --> 00:09:02.139 well as the standard prompts. Sometimes the creative 00:09:02.139 --> 00:09:04.840 variations even scored the highest. No hit to 00:09:04.840 --> 00:09:06.919 truthfulness. Okay, that's good. And safety. 00:09:07.120 --> 00:09:10.000 Harmful prompts. The safety guardrails held strong. 00:09:10.539 --> 00:09:13.179 On the Strong Reject benchmark, which tests refusal 00:09:13.179 --> 00:09:15.980 of harmful requests, the models using verbalized 00:09:15.980 --> 00:09:18.980 sampling still refused over 97 % of the time. 00:09:19.059 --> 00:09:21.659 Same level as standard. So it doesn't break the 00:09:21.659 --> 00:09:24.600 safety rules. Right. And interestingly, how it 00:09:24.600 --> 00:09:27.200 refused was more diverse. Instead of the same 00:09:27.200 --> 00:09:29.539 robot, I cannot fulfill this request message, 00:09:29.799 --> 00:09:32.340 it gave different, sometimes more helpful explanations 00:09:32.340 --> 00:09:35.860 for why it was refusing. So, creativity unlocked, 00:09:36.059 --> 00:09:39.519 but not chaos. Okay, this all sounds incredibly 00:09:39.519 --> 00:09:41.879 promising, but for the average person using this, 00:09:42.039 --> 00:09:44.080 what's the catch? What's the real -world cost? 00:09:44.320 --> 00:09:46.840 Well, the main cost is resources. Generating 00:09:46.840 --> 00:09:49.279 five diverse answers takes about five times the 00:09:49.279 --> 00:09:51.620 compute, five times the time, five times the 00:09:51.620 --> 00:09:54.379 potential cost, mid -role sponsor replaceholder. 00:09:54.659 --> 00:09:56.159 Right, so let's talk about those trade -offs, 00:09:56.220 --> 00:09:58.039 the fine print. As you said, it costs more time, 00:09:58.120 --> 00:09:59.799 compute, potentially money, depending on the 00:09:59.799 --> 00:10:02.399 API. Yep, and it works best on the really big 00:10:02.399 --> 00:10:04.179 models, so if you're using smaller ones, the 00:10:04.179 --> 00:10:06.360 effect might be less dramatic. And it's definitely 00:10:06.360 --> 00:10:09.059 not for everything. Asking what's the capital 00:10:09.059 --> 00:10:13.259 of France with probabilities is overkill. Yeah. 00:10:13.360 --> 00:10:15.559 Probably just get Paris with a probability near 00:10:15.559 --> 00:10:18.960 1 .0. Exactly. It's for creative tasks, brainstorming, 00:10:18.960 --> 00:10:22.440 exploration, not simple fact retrieval. And there's 00:10:22.440 --> 00:10:24.899 a user cost too, right? Getting five diverse 00:10:24.899 --> 00:10:27.639 options is great, but you have to do the work 00:10:27.639 --> 00:10:31.110 now. Meaning you have to read all five. evaluate 00:10:31.110 --> 00:10:33.190 them, decide which one is actually best for your 00:10:33.190 --> 00:10:36.549 needs, maybe combine elements. It requires more 00:10:36.549 --> 00:10:38.789 thinking on your part than just getting one safe 00:10:38.789 --> 00:10:41.750 answer. Yeah, that's fair. I mean, I still wrestle 00:10:41.750 --> 00:10:43.830 with prompt drift myself sometimes, tweeting 00:10:43.830 --> 00:10:46.730 things constantly. So knowing I have to actively 00:10:46.730 --> 00:10:49.730 sift through more options. It's a trade -off, 00:10:49.730 --> 00:10:52.240 but... The potential reward seems worth it. It's 00:10:52.240 --> 00:10:54.840 work, but good work. It is. But the good news 00:10:54.840 --> 00:10:57.000 is you can try this right now pretty easily. 00:10:57.340 --> 00:10:59.940 Method one is just direct prompting in your usual 00:10:59.940 --> 00:11:02.519 chat interface. How does that work? Just type 00:11:02.519 --> 00:11:04.659 the phrase. Kind of, but it helps to give it 00:11:04.659 --> 00:11:07.039 more structure. The researchers recommend using 00:11:07.039 --> 00:11:09.740 simple tags, almost like XML, to make it really 00:11:09.740 --> 00:11:11.440 clear what you're asking for. Okay, like how? 00:11:11.620 --> 00:11:13.480 Can you give an example? Sure. You'd start with 00:11:13.480 --> 00:11:16.299 an instruction block, something like... Instructions 00:11:16.299 --> 00:11:19.379 generate five responses. Put each in a response 00:11:19.379 --> 00:11:22.179 tag. Each needs text and a numeric probability. 00:11:22.879 --> 00:11:24.960 Sample randomly from the whole distribution. 00:11:25.720 --> 00:11:28.259 Instructions. Then after that block, you put 00:11:28.259 --> 00:11:31.220 your actual question, like... Write a story about 00:11:31.220 --> 00:11:33.960 a lonely robot. Ah, so the tags, instructions, 00:11:34.179 --> 00:11:36.659 response, text, probability help the AI understand 00:11:36.659 --> 00:11:39.159 it's not a normal chat. It's supposed to format 00:11:39.159 --> 00:11:41.679 the output like a structured report. Exactly. 00:11:41.679 --> 00:11:44.399 It forces it out of its conversational habits 00:11:44.399 --> 00:11:47.039 and into this more analytical sampling mode. 00:11:47.320 --> 00:11:50.179 It helps bypass that polite and boring layer 00:11:50.179 --> 00:11:52.340 we talked about. Got it. That makes sense. And 00:11:52.340 --> 00:11:54.139 for people who want this all the time. That's 00:11:54.139 --> 00:11:56.440 method two, system prompt integration. If your 00:11:56.440 --> 00:11:58.679 AI tool has custom instructions or system prompt 00:11:58.679 --> 00:12:00.720 settings, you can put the core instructions in 00:12:00.720 --> 00:12:02.620 there. Tell it to always sample from the full 00:12:02.620 --> 00:12:05.100 distribution, maybe even favor the lower probability 00:12:05.100 --> 00:12:08.559 tails. Make creativity the default. Right, set 00:12:08.559 --> 00:12:10.860 it and forget it, kind of. So, applications. 00:12:11.690 --> 00:12:14.169 Obviously brainstorming. Huge for brainstorming. 00:12:14.169 --> 00:12:15.950 Getting genuinely different starting points. 00:12:16.110 --> 00:12:19.129 Content creation. Finding new angles or structures 00:12:19.129 --> 00:12:22.409 for articles, scripts, whatever. And image generation. 00:12:22.509 --> 00:12:25.549 Imagine feeding five really unique concepts into 00:12:25.549 --> 00:12:28.129 mid -journey or dally instead of five minor variations 00:12:28.129 --> 00:12:30.809 of one idea. Yeah, that opens up a lot. Yeah. 00:12:30.889 --> 00:12:33.090 Okay, stepping back, thinking about everything 00:12:33.090 --> 00:12:35.529 these models know, that whole vast distribution 00:12:35.529 --> 00:12:38.190 of knowledge, what's the biggest single takeaway 00:12:38.190 --> 00:12:41.740 from this verbalized sampling breakthrough? I 00:12:41.740 --> 00:12:43.980 think it tells us we don't have to accept a tradeoff 00:12:43.980 --> 00:12:46.059 between safety and creativity. That was a false 00:12:46.059 --> 00:12:48.360 choice. Human bias was the limit, not the AI 00:12:48.360 --> 00:12:50.860 itself. Hashtag, hashtag, outro. That really 00:12:50.860 --> 00:12:53.240 is a profound shift in perspective. So let's 00:12:53.240 --> 00:12:55.379 quickly recap the two big ideas here. First, 00:12:55.500 --> 00:12:57.419 the AI creativity we thought alignment training 00:12:57.419 --> 00:12:59.879 had dampened. It wasn't gone, just hidden, masked 00:12:59.879 --> 00:13:01.940 by our own human preference for the easy, the 00:13:01.940 --> 00:13:05.700 familiar, the safe. Our biases trained it to 00:13:05.700 --> 00:13:08.740 be boring. And second, that simple eight -word 00:13:08.740 --> 00:13:11.980 phrase with their probabilities acts like a key. 00:13:12.519 --> 00:13:16.100 It changes the AI's task from give the best answer 00:13:16.100 --> 00:13:19.320 to report a sample of possible answers, letting 00:13:19.320 --> 00:13:22.179 it access that deeper, wilder knowledge it was 00:13:22.179 --> 00:13:24.080 trained on. It makes you wonder, doesn't it, 00:13:24.100 --> 00:13:26.700 if this creative ceiling was just a mirage caused 00:13:26.700 --> 00:13:29.929 by how we ask the questions. What other incredible 00:13:29.929 --> 00:13:32.090 abilities might be lying dormant in these models? 00:13:32.529 --> 00:13:35.250 Exactly. What else are we not seeing? Because 00:13:35.250 --> 00:13:37.350 we're not asking in the right way. The limit 00:13:37.350 --> 00:13:39.590 might truly be our own imagination in figuring 00:13:39.590 --> 00:13:42.110 out how to unlock it. A powerful thought to end 00:13:42.110 --> 00:13:43.990 on. We definitely encourage you listening to 00:13:43.990 --> 00:13:46.289 try method one, the direct prompting with those 00:13:46.289 --> 00:13:48.730 tags. See what you discover. Thanks for joining 00:13:48.730 --> 00:13:50.789 us for this deep dive. We'll see you next time.