WEBVTT 00:00:00.000 --> 00:00:03.279 OK, so have you ever found yourself building 00:00:03.279 --> 00:00:06.080 an AI agent, maybe for your business, maybe for 00:00:06.080 --> 00:00:08.619 content, or perhaps like a customer chatbot, 00:00:08.939 --> 00:00:11.580 and you just realize its responses? They aren't 00:00:11.580 --> 00:00:13.880 quite hitting the mark. Maybe it repeats itself 00:00:13.880 --> 00:00:16.760 or gets a bit stuck. Doesn't always follow instructions. 00:00:16.820 --> 00:00:19.019 It's super common, honestly. Almost like you've 00:00:19.019 --> 00:00:21.539 got this incredibly powerful engine, but you 00:00:21.539 --> 00:00:24.140 just can't seem to get it out of second gear. 00:00:24.440 --> 00:00:27.460 It's frustrating. And what a lot of us are discovering, 00:00:27.640 --> 00:00:30.670 sometimes much to our surprise, is the real fix 00:00:30.670 --> 00:00:33.009 isn't always about writing some super complex 00:00:33.009 --> 00:00:35.530 prompt or even shelling out for a bigger, pricier 00:00:35.530 --> 00:00:38.070 model. Often it's actually tucked away in these 00:00:38.070 --> 00:00:40.170 powerful but pretty simple settings that, well, 00:00:40.369 --> 00:00:42.409 many people just don't know about or maybe overlook. 00:00:42.850 --> 00:00:45.509 So today we're doing a deep dive. And this is 00:00:45.509 --> 00:00:47.710 really custom tailored to help you unlock that 00:00:47.710 --> 00:00:50.250 true potential hiding in your AI agents. We're 00:00:50.250 --> 00:00:52.649 going to demystify eight crucial, let's call 00:00:52.649 --> 00:00:55.210 them knobs and dials that can seriously boost 00:00:55.210 --> 00:00:57.689 your AI's performance, its reliability, and frankly, 00:00:57.710 --> 00:00:59.780 how smart it seems. Okay, let's get into it. 00:00:59.960 --> 00:01:02.380 It's so true. It really is Yeah, what often gets 00:01:02.380 --> 00:01:05.280 missed is that real control, you know genuine 00:01:05.280 --> 00:01:08.700 control over an AI agent It goes way beyond just 00:01:08.700 --> 00:01:11.840 what you type in the prompt box. It's about understanding 00:01:11.840 --> 00:01:15.359 those Fundamental mechanics that shape the output. 00:01:15.359 --> 00:01:17.540 Yeah, essentially gives you the power to sculpt 00:01:17.540 --> 00:01:20.769 its behavior with well surprising precision, 00:01:20.890 --> 00:01:23.629 moving from guesswork to really granular control. 00:01:23.870 --> 00:01:26.409 Exactly. When we first start with AI, yeah, all 00:01:26.409 --> 00:01:28.109 our focus goes straight to the prompt. Or maybe 00:01:28.109 --> 00:01:30.209 we spend ages debating, you know, should I use 00:01:30.209 --> 00:01:33.069 GPT -4 or Claude 3? And look, those things are 00:01:33.069 --> 00:01:34.769 absolutely important. Don't get me wrong. But 00:01:34.769 --> 00:01:37.030 there's this deeper layer, this really powerful 00:01:37.030 --> 00:01:39.310 control layer working behind the scenes. These 00:01:39.310 --> 00:01:41.430 are what we call model parameters. And these 00:01:41.430 --> 00:01:43.750 are the settings that genuinely shape how an 00:01:43.750 --> 00:01:46.250 AI model generates its entire response. It really 00:01:46.250 --> 00:01:48.390 does remind me of like stepping beyond the auto 00:01:48.390 --> 00:01:52.049 mode on on a fancy camera. For quick snaps, auto 00:01:52.049 --> 00:01:54.890 is fine. But a real photographer, they know that 00:01:54.890 --> 00:01:57.390 to get that perfect shot, that nuanced image, 00:01:57.629 --> 00:01:59.709 they have to manually adjust things like aperture, 00:02:00.030 --> 00:02:03.090 ISO, shutter speed. It's the same idea here. 00:02:03.469 --> 00:02:05.810 These AI settings let you fine tune everything. 00:02:06.170 --> 00:02:09.009 Creativity, randomness, even the length of the 00:02:09.009 --> 00:02:11.270 response or what topics it focuses on. And this 00:02:11.270 --> 00:02:12.789 is where it gets really interesting. You gain 00:02:12.789 --> 00:02:15.889 that sort of artistic control. Right. And the 00:02:15.889 --> 00:02:18.189 great thing is how standardized these parameters 00:02:18.189 --> 00:02:20.449 mostly are. So whether you're hitting an API 00:02:20.449 --> 00:02:24.030 directly or maybe using an aggregator like OpenRadar 00:02:24.030 --> 00:02:26.729 or even building workflows and tools like, say, 00:02:26.930 --> 00:02:29.590 N8n, these skills are totally transferable. What 00:02:29.590 --> 00:02:31.090 you learn here, you can pretty much apply it 00:02:31.090 --> 00:02:33.110 across all the major AI models and platforms 00:02:33.110 --> 00:02:35.849 out there. All right, let's jump right into these 00:02:35.849 --> 00:02:37.569 essential settings, the ones that are really 00:02:37.569 --> 00:02:39.509 going to transform your AI agents. First one 00:02:39.509 --> 00:02:42.550 up, frequency penalty. Have you ever noticed 00:02:42.550 --> 00:02:45.349 your AI agent just repeating the same words or 00:02:45.349 --> 00:02:47.550 phrases? making the output sound a bit, well, 00:02:47.689 --> 00:02:50.330 like a broken record, or just unnatural. Yep, 00:02:50.449 --> 00:02:52.789 seen that many times. That's exactly what frequency 00:02:52.789 --> 00:02:55.590 penalty tackles. It basically discourages the 00:02:55.590 --> 00:02:58.389 AI from reusing words and phrases it's already 00:02:58.389 --> 00:03:00.789 put out in that specific response. So you get 00:03:00.789 --> 00:03:02.909 much more diverse, less robotic -sounding text. 00:03:03.289 --> 00:03:05.569 The scale usually goes from about negative 2 00:03:05.569 --> 00:03:08.689 .0 to 2 .0. Positive values, say somewhere between 00:03:08.689 --> 00:03:11.629 plus 0 .5 and plus 1 .5, they penalize that repetition, 00:03:12.069 --> 00:03:13.909 pushing the AI to find different ways of saying 00:03:13.909 --> 00:03:15.949 things. But here's a kind of interesting twist. 00:03:16.669 --> 00:03:18.810 Negative values, like negative 0 .5 down to negative 00:03:18.810 --> 00:03:21.409 1 .5, they actually encourage repetition. Now 00:03:21.409 --> 00:03:23.310 that seems counterintuitive, right? Encouraging 00:03:23.310 --> 00:03:24.789 repetition. It does at first, Claire. Can you 00:03:24.789 --> 00:03:26.449 give us a scenario where you'd actually want 00:03:26.449 --> 00:03:28.590 the AI to be repetitive? Why would that be crucial? 00:03:29.009 --> 00:03:31.009 That's an excellent question because it really 00:03:31.009 --> 00:03:32.990 highlights how different outputs have different 00:03:32.990 --> 00:03:36.250 needs, you know. So, while you absolutely want 00:03:36.250 --> 00:03:38.629 that linguistic diversity for say, marketing 00:03:38.629 --> 00:03:41.370 copy or a blog post, keep things fresh and engaging. 00:03:41.909 --> 00:03:43.870 Imagine you're generating something like medical 00:03:43.870 --> 00:03:46.509 disclaimers, or maybe legal terms and conditions. 00:03:46.949 --> 00:03:50.389 In those cases, exact, consistent wording is 00:03:50.389 --> 00:03:53.330 absolutely critical. You want the AI to repeat 00:03:53.330 --> 00:03:57.129 key phrases precisely to assure accuracy, compliance, 00:03:57.330 --> 00:04:00.099 clarity. So yeah, a slightly negative frequency 00:04:00.099 --> 00:04:02.060 penalty could actually be incredibly useful there. 00:04:02.139 --> 00:04:04.379 Keeps it consistent. Ah, okay, that makes perfect 00:04:04.379 --> 00:04:07.460 sense. Consistency versus creativity. So if frequency 00:04:07.460 --> 00:04:09.800 penalty handles the quality or diversity of the 00:04:09.800 --> 00:04:13.099 words, what of the quantity? And, maybe more 00:04:13.099 --> 00:04:15.639 importantly for many people, the cost. That brings 00:04:15.639 --> 00:04:18.439 us neatly to max tokens. Now, a token is basically 00:04:18.439 --> 00:04:21.050 a piece of a word. Roughly speaking, about 100 00:04:21.050 --> 00:04:24.170 tokens is like 75 words. This setting puts a 00:04:24.170 --> 00:04:26.569 hard stop, a limit on how long the AI's response 00:04:26.569 --> 00:04:28.629 can actually be. And believe me, this is a critical 00:04:28.629 --> 00:04:30.829 knob for controlling both your output and your 00:04:30.829 --> 00:04:33.610 API costs. Absolutely crucial for costs. Right. 00:04:33.649 --> 00:04:35.769 By default, or if you set it to a mass one, the 00:04:35.769 --> 00:04:38.769 AI will just keep generating text up to its model's 00:04:38.769 --> 00:04:41.389 maximum limit, which can be huge, thousands of 00:04:41.389 --> 00:04:43.470 tokens sometimes. But if you give it a positive 00:04:43.470 --> 00:04:46.350 number, like say 150, the AI just stops dead 00:04:46.350 --> 00:04:49.350 once it hits that 150 token mark. This is invaluable 00:04:49.350 --> 00:04:52.029 for things where you need short, concise outputs, 00:04:52.490 --> 00:04:55.410 like headlines, tweets, SMS messages, maybe little 00:04:55.410 --> 00:04:57.930 bits of text for user interfaces. I definitely 00:04:57.930 --> 00:04:59.670 remember an early project where I forgot to set 00:04:59.670 --> 00:05:01.709 this. The AI tried to write a five -page essay 00:05:01.709 --> 00:05:03.870 when all I needed was a tweet. Oh dear. Yeah, 00:05:03.949 --> 00:05:05.870 my API bill that month. Definitely a memorable 00:05:05.870 --> 00:05:08.970 lesson learned. So beyond just fitting content, 00:05:09.009 --> 00:05:11.370 what's the bigger picture, the so what, of managing 00:05:11.370 --> 00:05:15.129 max tokens from, say, an operational view? Exactly. 00:05:15.310 --> 00:05:17.769 And the real sting often isn't just the per token 00:05:17.769 --> 00:05:20.709 charge itself, but the cumulative waste over 00:05:20.709 --> 00:05:24.430 time. I've literally seen teams unknowingly generate 00:05:24.430 --> 00:05:27.430 gigabytes, gigabytes of unnecessary text and 00:05:27.430 --> 00:05:30.490 background jobs or automated workflows. It turns 00:05:30.490 --> 00:05:33.220 what seems like a cheap API call into a really 00:05:33.220 --> 00:05:35.420 shocking monthly bill when you add it all up. 00:05:36.000 --> 00:05:38.040 So Max Tokens isn't just about fitting content 00:05:38.040 --> 00:05:40.620 neatly into boxes. It's a direct way to stop 00:05:40.620 --> 00:05:43.139 hidden costs from spiraling. It ensures your 00:05:43.139 --> 00:05:46.029 AI budget is spent efficiently, not just on Well, 00:05:46.170 --> 00:05:48.269 extra words nobody needed. That's such a crucial 00:05:48.269 --> 00:05:50.769 point. Those hidden costs can really, really 00:05:50.769 --> 00:05:52.850 sneak up on you. OK, but what if you don't just 00:05:52.850 --> 00:05:55.550 want the text itself? What if you need the AI's 00:05:55.550 --> 00:05:57.569 output to actually do something in your workflow? 00:05:58.029 --> 00:05:59.970 That's where response format becomes an absolute 00:05:59.970 --> 00:06:02.670 game changer. This setting basically forces the 00:06:02.670 --> 00:06:05.009 AI to output its response in a specific structured 00:06:05.009 --> 00:06:08.009 format. Most often, that's JSON instead of just 00:06:08.009 --> 00:06:11.149 plain old text. Huge difference. Yeah. By default, 00:06:11.230 --> 00:06:13.889 the AI usually just gives you raw text. But if 00:06:13.889 --> 00:06:16.720 you flip the setting to JSON, the AI is then 00:06:16.720 --> 00:06:19.019 constrained. It has to generate a valid JSON 00:06:19.019 --> 00:06:22.060 object. Now the key thing here is you still need 00:06:22.060 --> 00:06:24.100 to clearly describe the JSON structure you want 00:06:24.100 --> 00:06:25.879 within your prompt. You have to tell it what 00:06:25.879 --> 00:06:28.480 the JSON should look like. This is so vital when 00:06:28.480 --> 00:06:30.540 you're passing that AI output along to another 00:06:30.540 --> 00:06:33.360 application or another API, right? Or when you 00:06:33.360 --> 00:06:35.959 need to avoid messy text parsing. Or maybe you're 00:06:35.959 --> 00:06:38.079 extracting specific bits of info like names, 00:06:38.319 --> 00:06:40.759 dates, sentiment scores. Imagine analyzing customer 00:06:40.759 --> 00:06:42.899 feedback, for instance. You could prompt the 00:06:42.899 --> 00:06:45.220 AI. Give me the sentiment. the key issues, and 00:06:45.220 --> 00:06:47.439 a suggested action item, all neatly formatted 00:06:47.439 --> 00:06:49.860 in JSON, ready to go straight into your CRM or 00:06:49.860 --> 00:06:52.339 project tool. You've absolutely nailed the core 00:06:52.339 --> 00:06:54.939 value there. The importance of structured data, 00:06:55.259 --> 00:06:57.300 especially for building robust AI workflows, 00:06:57.899 --> 00:07:00.459 it just can't be overstated. When you're integrating 00:07:00.459 --> 00:07:03.279 AI into a bigger system, predictability in that 00:07:03.279 --> 00:07:06.720 output format is everything. JSON ensures your 00:07:06.720 --> 00:07:09.399 downstream apps get clean, easily parsable data. 00:07:09.500 --> 00:07:12.319 It prevents errors cascading down the line, and 00:07:12.319 --> 00:07:14.980 it significantly boosts the reliability and, 00:07:14.980 --> 00:07:17.319 frankly, the scalability of your whole automation. 00:07:17.959 --> 00:07:19.480 It's really the difference between getting a 00:07:19.480 --> 00:07:22.079 messy text file you have to somehow decode and 00:07:22.079 --> 00:07:24.000 getting a perfectly organized database entry 00:07:24.000 --> 00:07:27.339 ready to use. Fascinating how just a simple format 00:07:27.339 --> 00:07:29.220 change can ripple through the whole system like 00:07:29.220 --> 00:07:31.339 that. Okay, let's move on to presence penalty. 00:07:31.709 --> 00:07:33.870 Now, while frequency penalty discourages repeating 00:07:33.870 --> 00:07:36.290 words, presence penalty discourages repeating 00:07:36.290 --> 00:07:38.709 poppics. It's all about encouraging the AI to 00:07:38.709 --> 00:07:41.769 bring in genuinely new concepts and ideas, stopping 00:07:41.769 --> 00:07:43.709 it from getting stuck on just one track, you 00:07:43.709 --> 00:07:45.990 know, going around in circles conversationally. 00:07:46.110 --> 00:07:48.290 Keeping the conversation moving forward. Exactly. 00:07:48.410 --> 00:07:51.250 It uses the same Neginesh 2 .0 to 2 .0 scale. 00:07:51.370 --> 00:07:54.230 positive values, maybe plus 0 .5 to plus 1 .5, 00:07:54.730 --> 00:07:56.930 strongly encourage the AI to introduce new topics. 00:07:57.370 --> 00:07:59.509 This is fantastic for things like brainstorming, 00:07:59.850 --> 00:08:01.790 like maybe you want diverse suggestions for a 00:08:01.790 --> 00:08:03.709 travel itinerary. You want it to cover historical 00:08:03.709 --> 00:08:05.829 sites, restaurants, and outdoor activities, making 00:08:05.829 --> 00:08:07.509 sure it hits different types of suggestions. 00:08:08.370 --> 00:08:11.029 Conversely, negative values, say negative 0 .5 00:08:11.029 --> 00:08:13.810 down to 1 .5, make the AI stay really focused 00:08:13.810 --> 00:08:15.490 on the topics already on the table, drilling 00:08:15.490 --> 00:08:18.430 down deeper into those. So imagine brainstorming 00:08:18.430 --> 00:08:20.670 marketing angles for a new app, a high presence 00:08:20.680 --> 00:08:23.600 Penalty ensures it suggests ideas across various 00:08:23.600 --> 00:08:25.699 channels, you know, social media, content marketing, 00:08:25.879 --> 00:08:27.939 maybe influencer outreach, instead of just giving 00:08:27.939 --> 00:08:30.660 you 10 slight variations of a Facebook ad concept. 00:08:30.939 --> 00:08:32.820 How does this setting really drive that broader 00:08:32.820 --> 00:08:35.799 ideation compared to just, say, varying the vocabulary 00:08:35.799 --> 00:08:38.159 like frequency penalty does? Well, it's a fundamental 00:08:38.159 --> 00:08:40.799 shift in how the AI generates the content, not 00:08:40.799 --> 00:08:43.980 just what words it uses. While frequency penalty 00:08:43.980 --> 00:08:46.539 ensures, let's say, linguistic variety within 00:08:46.539 --> 00:08:49.580 a given topic, presence penalty literally pushes 00:08:49.580 --> 00:08:53.500 the AI to explore new conceptual ground. It stops 00:08:53.500 --> 00:08:56.379 the model from just endlessly rephrasing or slightly 00:08:56.379 --> 00:08:58.700 expanding on themes it's already mentioned. It 00:08:58.700 --> 00:09:00.799 forces it to branch out, to think differently. 00:09:01.200 --> 00:09:03.940 So for creative tasks or maybe exploratory queries 00:09:03.940 --> 00:09:06.399 where you want novel ideas, this means the AI 00:09:06.399 --> 00:09:08.559 doesn't just give you more of the same slightly 00:09:08.559 --> 00:09:11.419 rewarded, it genuinely offers up new avenues 00:09:11.419 --> 00:09:13.120 of thought. It's really the difference between 00:09:13.120 --> 00:09:16.100 getting, say, 10 variations on one theme versus 00:09:16.100 --> 00:09:18.600 getting 10 genuinely different themes to consider. 00:09:18.799 --> 00:09:20.639 Right, different themes altogether. And now for 00:09:20.639 --> 00:09:22.919 one that often feels a bit like magic, the one 00:09:22.919 --> 00:09:25.360 that truly seems to change the AI's personality, 00:09:25.799 --> 00:09:28.700 temperature. This is arguably the most important 00:09:28.700 --> 00:09:30.740 setting for shaping the creative output, the 00:09:30.740 --> 00:09:33.940 feel of your AI. It directly adjusts the randomness 00:09:33.940 --> 00:09:36.019 and creativity in the response. It's like tuning 00:09:36.019 --> 00:09:38.659 the AI's personality dial. The creativity dial, 00:09:38.659 --> 00:09:41.389 I like that. Yeah. The scale usually goes from 00:09:41.389 --> 00:09:43.490 zero up to one, though sometimes you see it go 00:09:43.490 --> 00:09:45.629 up to two, depending on the model provider. A 00:09:45.629 --> 00:09:48.230 low temperature, maybe down between 0 .1 and 00:09:48.230 --> 00:09:51.850 0 .3, makes the AI extremely predictable. Very 00:09:51.850 --> 00:09:54.210 deterministic, highly focused. Think of it as 00:09:54.210 --> 00:09:56.710 your super accurate technical writer. This is 00:09:56.710 --> 00:09:59.129 best for tasks needing precision, consistency, 00:09:59.490 --> 00:10:01.850 maybe extracting data or generating code where 00:10:01.850 --> 00:10:03.990 you don't want surprises. Right, factual stuff. 00:10:04.230 --> 00:10:06.129 Exactly. Then you've got a medium temperature, 00:10:06.210 --> 00:10:09.700 maybe 0 .4 to 0 .7. a nice balance. It's great 00:10:09.700 --> 00:10:11.799 for general purpose tasks, drafting emails, writing 00:10:11.799 --> 00:10:14.500 reports, a standard chatbot response, but if 00:10:14.500 --> 00:10:16.899 you want something really creative, surprising, 00:10:16.960 --> 00:10:19.299 maybe even a little bit risky, you crank that 00:10:19.299 --> 00:10:22.259 temperature up. High setting, like 0 .8 to 1 00:10:22.259 --> 00:10:25.379 .0. This is your brainstorming artist mode. Fantastic 00:10:25.379 --> 00:10:27.919 for marketing slogans, creative stories, maybe 00:10:27.919 --> 00:10:30.440 even artistic content generation. Simple rule 00:10:30.440 --> 00:10:32.940 of thumb, if your responses are too boring, turn 00:10:32.940 --> 00:10:34.779 the temperature up. If they're too random or 00:10:34.779 --> 00:10:37.320 nonsensical, turn it down. A good starting point 00:10:37.320 --> 00:10:40.100 for general used is often around 0 .7. What's 00:10:40.100 --> 00:10:42.679 so fascinating about how this setting, temperature, 00:10:43.120 --> 00:10:45.559 directly impacts the feel the character of the 00:10:45.559 --> 00:10:49.220 AI's output. It's just captivating because temperature 00:10:49.220 --> 00:10:51.139 doesn't merely change the words on the page. 00:10:51.639 --> 00:10:53.539 It fundamentally changes the perceived character 00:10:53.539 --> 00:10:56.700 of the AI interacting with you. A well -tuned 00:10:56.700 --> 00:10:59.059 temperature setting can genuinely transform an 00:10:59.059 --> 00:11:02.600 AI. It can go from feeling like this rigid factual 00:11:02.600 --> 00:11:05.279 database spitting out information to feeling 00:11:05.279 --> 00:11:08.340 like a dynamic, even insightful creative partner. 00:11:09.059 --> 00:11:11.700 It directly influences that perceived intelligence 00:11:11.700 --> 00:11:14.559 and usefulness, especially for tasks that demand 00:11:14.559 --> 00:11:17.480 originality or a fresh perspective. It dictates 00:11:17.480 --> 00:11:19.720 the overall flavor, the surprise factor of the 00:11:19.720 --> 00:11:22.070 output. which makes it one of the most powerful 00:11:22.070 --> 00:11:24.570 and actually quite intuitive ways to direct the 00:11:24.570 --> 00:11:26.929 AI's creative potential. Character, that's a 00:11:26.929 --> 00:11:29.029 perfect word for it. Okay, now let's switch gears 00:11:29.029 --> 00:11:30.769 a bit and talk about something more operational. 00:11:31.929 --> 00:11:34.409 Timeout. This setting simply determines how long 00:11:34.409 --> 00:11:36.470 your system will wait for the AI to give a response 00:11:36.470 --> 00:11:38.750 before it just gives up and throws an error message. 00:11:38.909 --> 00:11:40.769 It's usually measured in milliseconds. Important 00:11:40.769 --> 00:11:43.700 for user experience. Hugely. The default can 00:11:43.700 --> 00:11:45.639 be quite generous, actually, often something 00:11:45.639 --> 00:11:49.179 like 360 ,000 milliseconds, which is a full six 00:11:49.179 --> 00:11:51.820 minutes. That's designed to handle even really 00:11:51.820 --> 00:11:54.320 complex requests that take a lot of compute time. 00:11:54.700 --> 00:11:57.220 But you can set a custom value, like maybe 30 00:11:57.220 --> 00:11:59.120 ,000 milliseconds for 30 seconds. You definitely 00:11:59.120 --> 00:12:01.480 want a lower timeout, maybe somewhere in the 00:12:01.480 --> 00:12:03.940 20 ,000 to 60 ,000 millisecond range, that's 00:12:03.940 --> 00:12:06.220 20 to 60 seconds, for things like real -time 00:12:06.220 --> 00:12:09.320 chatbots. Or any user -facing app or making someone 00:12:09.320 --> 00:12:11.179 wait ages would just be a terrible experience. 00:12:11.179 --> 00:12:14.600 Absolutely. For background processes, maybe generating 00:12:14.600 --> 00:12:17.279 long reports or doing very complex data analysis, 00:12:17.480 --> 00:12:19.820 you might set a much higher timeout, maybe 600 00:12:19.820 --> 00:12:21.779 ,000 milliseconds, 10 minutes, or even more. 00:12:22.059 --> 00:12:24.899 Give it plenty of time to finish its work. So 00:12:24.899 --> 00:12:27.009 practical guidelines often look like... Live 00:12:27.009 --> 00:12:30.049 chat bots, maybe 15, 30 seconds max. Content 00:12:30.049 --> 00:12:32.450 generation, perhaps three, five minutes. Complex 00:12:32.450 --> 00:12:34.190 data analysis, you might allow 10 minutes or 00:12:34.190 --> 00:12:37.129 more. How do we effectively balance those user 00:12:37.129 --> 00:12:40.169 expectations for speed with the reality of the 00:12:40.169 --> 00:12:42.029 computational work happening behind the scenes? 00:12:42.289 --> 00:12:43.870 Well, that's the core challenge this setting 00:12:43.870 --> 00:12:45.970 helps you manage, isn't it? For an application 00:12:45.970 --> 00:12:48.450 where an end user is waiting, immediate feedback, 00:12:48.629 --> 00:12:51.019 or at least fast feedback. is critical. Users 00:12:51.019 --> 00:12:52.740 generally prefer getting a quick error message 00:12:52.740 --> 00:12:55.360 saying, something went wrong, try again, rather 00:12:55.360 --> 00:12:57.259 than just staring at a spinning yelp for minutes 00:12:57.259 --> 00:13:01.120 on end. So, in that case, you prioritize a lower 00:13:01.120 --> 00:13:03.299 timeout. You protect the user experience above 00:13:03.299 --> 00:13:06.139 all else. But for a backend process, something 00:13:06.139 --> 00:13:08.039 that can run asynchronously, meaning it doesn't 00:13:08.039 --> 00:13:10.779 need to reply instantly, allowing more time ensures 00:13:10.779 --> 00:13:12.820 those complex computations can actually complete 00:13:12.820 --> 00:13:15.879 successfully. There, you're prioritizing reliability 00:13:15.879 --> 00:13:18.700 and getting the full, comprehensive result, even 00:13:18.700 --> 00:13:20.980 if it takes longer. It really is all about the 00:13:20.980 --> 00:13:23.519 context of where and how the AI is being used. 00:13:23.860 --> 00:13:27.299 Context is king, definitely. Okay, but what happens 00:13:27.299 --> 00:13:29.620 when things inevitably do go wrong? you know, 00:13:29.980 --> 00:13:31.899 temporary glitches. That's where max retries 00:13:31.899 --> 00:13:34.659 comes into play. This setting controls how many 00:13:34.659 --> 00:13:36.700 times your workflow will automatically try again 00:13:36.700 --> 00:13:39.740 if a request fails. This is super useful for 00:13:39.740 --> 00:13:42.019 handling transient issues like a brief network 00:13:42.019 --> 00:13:45.159 hiccup or maybe the AI model itself is just momentarily 00:13:45.159 --> 00:13:47.559 overloaded and can't respond right away. Handling 00:13:47.559 --> 00:13:50.340 flakes, basically. Exactly. By default, it might 00:13:50.340 --> 00:13:52.600 be set to something reasonable, like two retries. 00:13:52.700 --> 00:13:54.950 You could set it to zero. if you want your system 00:13:54.950 --> 00:13:57.129 to just give up immediately on the very first 00:13:57.129 --> 00:14:00.250 failure. That can actually be useful during development 00:14:00.250 --> 00:14:02.309 or testing when you want to see errors quickly. 00:14:02.960 --> 00:14:05.399 Or you can crank it up, maybe set it to five, 00:14:05.740 --> 00:14:08.340 to significantly increase its persistence, its 00:14:08.340 --> 00:14:10.980 determination to get through. Generally, you'd 00:14:10.980 --> 00:14:13.100 set it lower when you prefer fast failure for 00:14:13.100 --> 00:14:15.639 debugging. But for critical tasks like, say, 00:14:15.919 --> 00:14:18.100 processing a customer's order or if you know 00:14:18.100 --> 00:14:20.120 you're dealing with an API that's occasionally 00:14:20.120 --> 00:14:22.500 unreliable or flaky, setting it higher, maybe 00:14:22.500 --> 00:14:24.960 three to five retries can be a real lifesaver. 00:14:25.059 --> 00:14:27.139 It ensures that crucial task eventually gets 00:14:27.139 --> 00:14:29.879 completed. But this brings up a really important 00:14:29.879 --> 00:14:32.659 point, a caution perhaps, about balancing that 00:14:32.659 --> 00:14:36.080 reliability against the cost implications. While 00:14:36.080 --> 00:14:39.580 yes, more retries absolutely increase the chances 00:14:39.580 --> 00:14:42.259 of a critical task succeeding eventually. You 00:14:42.259 --> 00:14:45.000 have to be mindful. If the API call you're making 00:14:45.000 --> 00:14:47.820 costs money each time you make it, setting max 00:14:47.820 --> 00:14:50.159 retries to five means you could potentially be 00:14:50.159 --> 00:14:52.460 charged up to six times for a single problematic 00:14:52.460 --> 00:14:55.220 request. That's the initial attempt plus those 00:14:55.220 --> 00:14:57.820 five retries. It can multiply your costs very 00:14:57.820 --> 00:14:59.539 quickly if the underlying issue persists and 00:14:59.539 --> 00:15:02.179 you're not careful. Yeah, that's a huge hidden 00:15:02.179 --> 00:15:04.779 cost multiplier many people probably don't think 00:15:04.779 --> 00:15:07.840 about initially. Good warning. Okay, last one 00:15:07.840 --> 00:15:10.460 on our list, top P. This is basically an alternative 00:15:10.460 --> 00:15:13.259 method to temperature for controlling the randomness, 00:15:13.399 --> 00:15:15.700 the unpredictability of your AI's output. It's 00:15:15.700 --> 00:15:17.399 sometimes called nuclear sampling. Instead of 00:15:17.399 --> 00:15:19.860 broadly adjusting the creativity feel, like temperature 00:15:19.860 --> 00:15:22.600 does, TopP focuses specifically on selecting 00:15:22.600 --> 00:15:24.799 from the most probable set of next possible words. 00:15:24.940 --> 00:15:26.480 Yeah, a different way to slice the probability 00:15:26.480 --> 00:15:29.539 pie. Exactly. It runs on a scale, typically 0 00:15:29.539 --> 00:15:32.769 .1 up to 1 .0. A setting of 1 .0 means the AI 00:15:32.769 --> 00:15:35.250 considers all possible words in its vocabulary 00:15:35.250 --> 00:15:37.570 when deciding what comes next. A setting like 00:15:37.570 --> 00:15:40.070 0 .5, though, means it only considers the smallest 00:15:40.070 --> 00:15:42.330 group of the most likely words whose combined 00:15:42.330 --> 00:15:45.450 probability adds up to 50%. And a really low 00:15:45.450 --> 00:15:48.289 top P, like 0 .1, means it's only looking at 00:15:48.289 --> 00:15:51.769 the top 10 % most probable words. This leads 00:15:51.769 --> 00:15:55.169 to very safe, very predictable, often quite conservative 00:15:55.169 --> 00:15:58.450 text. Now, a key pro tip you hear a lot. Most 00:15:58.450 --> 00:16:00.649 experts strongly recommend using other temperature 00:16:00.649 --> 00:16:03.110 or top P, but generally not both at the same 00:16:03.110 --> 00:16:04.750 time. They can kind of interfere with each other 00:16:04.750 --> 00:16:06.769 and produce weird results if you try to tune 00:16:06.769 --> 00:16:08.809 both simultaneously. And get one lane. Right. 00:16:09.190 --> 00:16:11.490 So when would someone actually choose to use 00:16:11.490 --> 00:16:13.490 top P instead of the more common temperature 00:16:13.490 --> 00:16:15.850 setting for their AI agent? That's a great question 00:16:15.850 --> 00:16:18.169 because they both touch on randomness, but in 00:16:18.169 --> 00:16:20.370 slightly different ways. Generally speaking, 00:16:20.490 --> 00:16:22.269 temperature is more intuitive for most people. 00:16:22.610 --> 00:16:25.419 It controls that overall feel. of creativity, 00:16:25.639 --> 00:16:27.340 the vibe, and it's usually the first thing people 00:16:27.340 --> 00:16:31.059 reach for. However, top P gives you more direct, 00:16:31.320 --> 00:16:34.659 maybe more fine grained control over the uniqueness 00:16:34.659 --> 00:16:37.200 or the range of the vocabulary the AI chooses 00:16:37.200 --> 00:16:39.759 from. So if you're generating text where you 00:16:39.759 --> 00:16:41.799 want a certain degree of creativity, but you 00:16:41.799 --> 00:16:44.120 absolutely need to prevent the AI from suddenly 00:16:44.120 --> 00:16:46.100 throwing in very strange or completely out of 00:16:46.100 --> 00:16:48.820 left field words. Think maybe technical prose 00:16:48.820 --> 00:16:51.159 that needs to be precise, but still varied and 00:16:51.159 --> 00:16:53.539 not robotic. Or maybe certain types of formal 00:16:53.539 --> 00:16:56.259 writing. In those cases, TopP can be incredibly 00:16:56.259 --> 00:16:59.080 useful. It essentially ensures the AI stays within 00:16:59.080 --> 00:17:01.720 a statistically safer, more coherent set of word 00:17:01.720 --> 00:17:03.840 choices, even when you're pushing for a bit more 00:17:03.840 --> 00:17:05.920 variety than zero temperature. OK, fascinating. 00:17:06.319 --> 00:17:08.720 So we've unpacked these eight really powerful 00:17:08.720 --> 00:17:11.470 settings. Now the big question is... How do we 00:17:11.470 --> 00:17:14.089 actually use them effectively in practice? We've 00:17:14.089 --> 00:17:16.430 put together a simple, but we think pretty powerful 00:17:16.430 --> 00:17:19.049 four -step optimization workflow for you to follow. 00:17:19.549 --> 00:17:22.299 Step one, define the problem. You need to be 00:17:22.299 --> 00:17:24.339 super specific about what's actually going wrong 00:17:24.339 --> 00:17:26.099 with your AI's output. Is it too repetitive? 00:17:26.299 --> 00:17:28.579 Okay, you know to like a frequency penalty. Is 00:17:28.579 --> 00:17:30.880 it too random and chaotic or maybe just too boring 00:17:30.880 --> 00:17:32.960 and predictable? That points towards temperature. 00:17:33.259 --> 00:17:35.720 Is it too slow? Check the timeout setting. Are 00:17:35.720 --> 00:17:38.380 the responses too long or maybe too short? That's 00:17:38.380 --> 00:17:40.640 max tokens. Do you need structured data? Use 00:17:40.640 --> 00:17:43.059 response format, probably JSON. Pinpoint the 00:17:43.059 --> 00:17:46.400 issue first. Then step two, change one setting 00:17:46.400 --> 00:17:49.759 at a time. This is absolutely critical and it's 00:17:49.759 --> 00:17:52.170 honestly where a lot of people stumble. Resist 00:17:52.170 --> 00:17:54.069 that temptation to just go in and tweak five 00:17:54.069 --> 00:17:56.210 different knobs at once. You want to approach 00:17:56.210 --> 00:17:58.450 this scientifically. Make a small, deliberate 00:17:58.450 --> 00:18:01.049 adjustment to one single setting, and then test 00:18:01.049 --> 00:18:03.950 the result thoroughly. This is the only reliable 00:18:03.950 --> 00:18:06.230 way to figure out what specific change actually 00:18:06.230 --> 00:18:08.769 worked and why it worked. If you change five 00:18:08.769 --> 00:18:10.930 things, you'll have no clue which one really 00:18:10.930 --> 00:18:13.549 made the difference, positive or negative. Isolate 00:18:13.549 --> 00:18:16.309 the variable. Exactly. Which leads perfectly 00:18:16.309 --> 00:18:18.980 into step three. test with realistic scenarios. 00:18:19.299 --> 00:18:21.500 Don't just ask your AI to, you know, tell me 00:18:21.500 --> 00:18:24.640 a joke or write a poem about cats. Use the actual 00:18:24.640 --> 00:18:27.440 kind of prompts and the real world data you expect 00:18:27.440 --> 00:18:29.920 it to handle in its final application. Feed it 00:18:29.920 --> 00:18:32.079 real customer emails if that's what it's for. 00:18:32.180 --> 00:18:34.740 Give it the specific tasks it needs to do. Use 00:18:34.740 --> 00:18:36.680 the data your business actually works with. This 00:18:36.680 --> 00:18:38.900 makes sure your adjustments are genuinely relevant 00:18:38.900 --> 00:18:41.740 and actually impactful for your specific use 00:18:41.740 --> 00:18:44.880 case, not just some abstract test. And finally, 00:18:45.220 --> 00:18:49.500 step four. Document and create profiles. This 00:18:49.500 --> 00:18:51.539 sounds simple, but it's so important. Keep a 00:18:51.539 --> 00:18:53.460 record, maybe just a simple spreadsheet or notes, 00:18:53.920 --> 00:18:56.240 of what settings work well for different tasks 00:18:56.240 --> 00:18:59.140 and different desired outcomes. Over time, you'll 00:18:59.140 --> 00:19:01.400 find yourself building up these specific profiles. 00:19:01.940 --> 00:19:03.960 Maybe you have a creative content profile with 00:19:03.960 --> 00:19:05.960 high temperature and high presence penalty settings 00:19:05.960 --> 00:19:08.619 saved, or perhaps a professional summary profile 00:19:08.619 --> 00:19:10.559 with low temperature, maybe a slight negative 00:19:10.559 --> 00:19:12.940 frequency penalty, and a specific max tokens 00:19:12.940 --> 00:19:15.680 limit for conciseness. So what does this mean 00:19:15.680 --> 00:19:17.940 for your long -term efficiency? It means you 00:19:17.940 --> 00:19:19.920 build this reusable library of configurations 00:19:19.920 --> 00:19:22.400 that you know work well. It saves immense time, 00:19:22.440 --> 00:19:24.539 effort, and probably even costs on future AI 00:19:24.539 --> 00:19:26.740 projects because you're not starting from scratch 00:19:26.740 --> 00:19:29.259 every single time. And there you have it. We've 00:19:29.259 --> 00:19:31.160 kind of journeyed from that initial frustration 00:19:31.160 --> 00:19:34.339 maybe of unpredictable AI agents all the way 00:19:34.339 --> 00:19:36.640 to understanding how you can wield this precise, 00:19:36.900 --> 00:19:39.640 fine -tuned control over their output. Just remember, 00:19:39.819 --> 00:19:42.319 optimization. It's an iterative process, right? 00:19:42.319 --> 00:19:45.259 It takes a bit of trial and error. Our advice, 00:19:45.619 --> 00:19:47.599 start with the most impactful settings first, 00:19:47.700 --> 00:19:49.880 usually temperature and max tokens. They often 00:19:49.880 --> 00:19:51.519 solve the biggest, most common issues right out 00:19:51.519 --> 00:19:53.859 of the gate. And then from there, you can layer 00:19:53.859 --> 00:19:55.920 in adjustments to the other parameters like frequency 00:19:55.920 --> 00:19:58.980 penalty or response format to really refine that 00:19:58.980 --> 00:20:01.279 output for your specific needs. Yeah. And if 00:20:01.279 --> 00:20:03.059 we connect this back to the bigger picture for 00:20:03.059 --> 00:20:06.980 a second, the future of working with AI, it really 00:20:06.980 --> 00:20:09.579 isn't just about having access to ever more powerful 00:20:09.579 --> 00:20:12.359 models. It's increasingly about skillfully directing 00:20:12.359 --> 00:20:15.740 them. With these settings now hopefully clearer 00:20:15.740 --> 00:20:18.319 and more accessible in your toolkit, you're really 00:20:18.319 --> 00:20:20.940 equipped to build that next generation of intelligent, 00:20:21.259 --> 00:20:23.579 effective AI agents, whether you're building, 00:20:23.579 --> 00:20:25.940 you know, a sophisticated business automation 00:20:25.940 --> 00:20:29.140 or maybe a groundbreaking content engine or even 00:20:29.140 --> 00:20:31.440 just a more responsive personal AI assistant, 00:20:32.079 --> 00:20:33.619 which kind of raises an important question for 00:20:33.619 --> 00:20:36.660 you, the listener. How will you apply these insights 00:20:36.660 --> 00:20:38.599 to your own projects? What are you going to build 00:20:38.599 --> 00:20:41.369 or improve first? Yeah, and maybe think about 00:20:41.369 --> 00:20:44.049 this as we wrap up. Mastering these seemingly 00:20:44.049 --> 00:20:47.410 small knobs and dials, it doesn't just improve 00:20:47.410 --> 00:20:50.190 the individual outputs of your AI. It can fundamentally 00:20:50.190 --> 00:20:52.430 change how you approach problem solving and innovation 00:20:52.430 --> 00:20:54.890 using intelligent systems really in any field. 00:20:55.450 --> 00:20:57.490 What new possibilities really open up for you 00:20:57.490 --> 00:20:59.630 when you realize you have this level of precise 00:20:59.630 --> 00:21:01.029 control over your AI?