1 00:00:00,000 --> 00:00:03,440 Okay, so get this, today we're diving into AI, right? 2 00:00:03,440 --> 00:00:07,280 But there's a twist, something you might not see coming. 3 00:00:07,280 --> 00:00:08,560 Are we gonna be talking about politics? 4 00:00:08,560 --> 00:00:10,280 Politics and AI. 5 00:00:10,280 --> 00:00:11,240 I know, right? 6 00:00:11,240 --> 00:00:12,920 Sounds a little crazy. 7 00:00:12,920 --> 00:00:14,720 But stay with me, okay? 8 00:00:14,720 --> 00:00:16,640 We're going deep on this research paper, 9 00:00:16,640 --> 00:00:19,040 straight out of MIT. 10 00:00:19,040 --> 00:00:21,840 It's called On the Relationship Between Truth 11 00:00:21,840 --> 00:00:24,200 and Political Bias in Language Models. 12 00:00:24,200 --> 00:00:26,920 Yeah, it's a fascinating paper, really makes you think. 13 00:00:26,920 --> 00:00:27,960 It does, it really does. 14 00:00:27,960 --> 00:00:31,120 So the researchers were trying to make AI more truthful, 15 00:00:31,120 --> 00:00:34,240 which on the surface sounds like a good thing, right? 16 00:00:34,240 --> 00:00:35,080 Absolutely, yeah. 17 00:00:35,080 --> 00:00:36,760 But what they found was that by doing that, 18 00:00:36,760 --> 00:00:38,920 they might actually be making these AI systems 19 00:00:38,920 --> 00:00:40,200 more politically biased. 20 00:00:40,200 --> 00:00:44,720 Wait, hold on, so making AI better at spotting facts 21 00:00:44,720 --> 00:00:46,640 and stuff could actually make it more biased. 22 00:00:46,640 --> 00:00:47,680 That's kind of a big deal. 23 00:00:47,680 --> 00:00:48,960 It is a big deal, yeah. 24 00:00:48,960 --> 00:00:50,840 But before we go any further, right, 25 00:00:50,840 --> 00:00:52,560 let's make sure we're all on the same page here. 26 00:00:52,560 --> 00:00:54,960 For anyone listening who's maybe not super familiar 27 00:00:54,960 --> 00:00:57,840 with AI, what exactly is a language model? 28 00:00:57,840 --> 00:00:59,640 Hmm, okay, good question. 29 00:00:59,640 --> 00:01:00,840 So think of it this way. 30 00:01:00,840 --> 00:01:03,240 A language model is kind of like that auto-complete 31 00:01:03,240 --> 00:01:05,440 on your phone, but supercharged. 32 00:01:06,440 --> 00:01:09,920 It's an AI system that can predict what word 33 00:01:09,920 --> 00:01:11,400 comes next in a sentence, 34 00:01:11,400 --> 00:01:13,560 which is pretty amazing when you think about it. 35 00:01:13,560 --> 00:01:15,280 This lets it do some cool stuff, 36 00:01:15,280 --> 00:01:19,520 like write different kinds of texts, translate languages, 37 00:01:19,520 --> 00:01:20,960 even answer your questions. 38 00:01:20,960 --> 00:01:24,000 It can even sound pretty human sometimes. 39 00:01:24,000 --> 00:01:27,400 And it does all this by learning from massive amounts 40 00:01:27,400 --> 00:01:29,760 of text data, like a ton of information. 41 00:01:29,760 --> 00:01:32,040 So it's basically like gobbling up all this info 42 00:01:32,040 --> 00:01:34,240 and then figuring out how to use language like we do. 43 00:01:34,240 --> 00:01:35,880 That's pretty wild. 44 00:01:35,880 --> 00:01:36,720 It is pretty wild. 45 00:01:36,720 --> 00:01:38,640 Okay, but back to the research, right. 46 00:01:38,640 --> 00:01:43,320 Why are these scientists so focused on making AI truthful? 47 00:01:43,320 --> 00:01:44,680 Like why is that so important? 48 00:01:44,680 --> 00:01:47,480 Well, imagine like getting medical advice from AI 49 00:01:47,480 --> 00:01:49,320 or reading news articles written by AI. 50 00:01:49,320 --> 00:01:50,160 Oh yeah. 51 00:01:50,160 --> 00:01:51,680 You'd wanna be sure it was accurate, right? 52 00:01:51,680 --> 00:01:52,520 Absolutely. 53 00:01:52,520 --> 00:01:55,120 That's why truthfulness is a big focus in AI right now. 54 00:01:55,120 --> 00:01:57,120 We need to know we can trust these systems 55 00:01:57,120 --> 00:01:59,160 to give us reliable information. 56 00:01:59,160 --> 00:02:00,600 Yeah, no, that makes total sense. 57 00:02:00,600 --> 00:02:03,760 So how did the researchers actually teach these AI models 58 00:02:03,760 --> 00:02:06,240 to know what's true and what's not? 59 00:02:06,240 --> 00:02:09,280 Okay, so they use something called reward models. 60 00:02:09,280 --> 00:02:11,520 Imagine you're like training a dog 61 00:02:11,520 --> 00:02:14,840 and you give it a treat when it does something good. 62 00:02:14,840 --> 00:02:16,200 Reward models kind of like that. 63 00:02:16,200 --> 00:02:19,560 It gives higher scores to statements that seem true 64 00:02:19,560 --> 00:02:22,320 and lower scores to ones that seem false. 65 00:02:22,320 --> 00:02:24,760 So it's basically like encouraging the AI 66 00:02:24,760 --> 00:02:25,960 to figure out what's true, right? 67 00:02:25,960 --> 00:02:27,120 Exactly, exactly. 68 00:02:27,120 --> 00:02:27,960 Pretty clever. 69 00:02:27,960 --> 00:02:30,080 So I'm guessing they didn't just use any information 70 00:02:30,080 --> 00:02:31,880 to train these models though, right? 71 00:02:31,880 --> 00:02:34,160 Right, they were very specific about it. 72 00:02:34,160 --> 00:02:37,600 They used all these data sets full of truthful statements, 73 00:02:37,600 --> 00:02:40,280 scientific facts, stuff from Wikipedia. 74 00:02:40,280 --> 00:02:42,240 They even used tricky questions like, 75 00:02:42,240 --> 00:02:44,960 designed to see if the AI could spot a lie. 76 00:02:44,960 --> 00:02:46,360 Wow, they really covered all the bases. 77 00:02:46,360 --> 00:02:48,640 Sounds like they really do their homework. 78 00:02:48,640 --> 00:02:52,040 So drumroll please, did it work? 79 00:02:52,040 --> 00:02:54,440 Were the AI models able to tell the truth 80 00:02:54,440 --> 00:02:56,520 from what wasn't true? 81 00:02:56,520 --> 00:02:57,520 Well, they definitely got better 82 00:02:57,520 --> 00:03:00,120 at identifying true statements, that's for sure. 83 00:03:00,120 --> 00:03:02,200 But here's where things get interesting. 84 00:03:02,200 --> 00:03:04,400 They found that a lot of these AIs, 85 00:03:04,400 --> 00:03:07,160 the ones that were supposedly good at spotting truth, 86 00:03:07,160 --> 00:03:09,640 started showing a pretty clear political bias 87 00:03:09,640 --> 00:03:13,240 and it was mostly leaning to the left. 88 00:03:13,240 --> 00:03:14,840 Whoa, hold up, are you telling me 89 00:03:14,840 --> 00:03:19,080 that by trying to make AI more truthful, 90 00:03:19,080 --> 00:03:21,560 it somehow became politically biased? 91 00:03:21,560 --> 00:03:22,800 How is that even possible? 92 00:03:22,800 --> 00:03:24,760 Yeah, that's the million dollar question. 93 00:03:24,760 --> 00:03:25,720 To measure this bias, 94 00:03:25,720 --> 00:03:28,640 they used this clever data set they made called TwinViews. 95 00:03:28,640 --> 00:03:29,480 The weight views. 96 00:03:29,480 --> 00:03:31,720 Yeah, it's got thousands of these pairs of statements 97 00:03:31,720 --> 00:03:33,200 on all sorts of hot button issues, 98 00:03:33,200 --> 00:03:37,000 you know like climate change, LGBTQ plus rights. 99 00:03:37,000 --> 00:03:37,840 But here's the thing, 100 00:03:37,840 --> 00:03:39,800 each pair presents opposite views, 101 00:03:39,800 --> 00:03:41,600 one leaning left and one leaning right. 102 00:03:41,600 --> 00:03:42,440 Oh, okay, I see. 103 00:03:42,440 --> 00:03:44,360 So they're like putting these politically charged 104 00:03:44,360 --> 00:03:46,800 statements head to head and then seeing which side 105 00:03:46,800 --> 00:03:49,960 the quote unquote truthful AI picks. 106 00:03:49,960 --> 00:03:50,920 That's pretty smart. 107 00:03:50,920 --> 00:03:52,360 But how they make sure those statements 108 00:03:52,360 --> 00:03:55,600 actually matched up with real world political views. 109 00:03:55,600 --> 00:03:59,600 Okay, so they used another AI for this GPT 3.5. 110 00:03:59,600 --> 00:04:01,920 They basically told it to create statements 111 00:04:01,920 --> 00:04:05,480 that aligned with either left or right leaning views. 112 00:04:05,480 --> 00:04:10,480 So they were grounded in like real world political ideology. 113 00:04:10,800 --> 00:04:13,400 So they used AI to build the tool 114 00:04:13,400 --> 00:04:16,160 to test for bias in other AI. 115 00:04:16,160 --> 00:04:17,000 Exactly. 116 00:04:17,000 --> 00:04:18,360 That's wild. 117 00:04:18,360 --> 00:04:19,200 And what happened? 118 00:04:19,200 --> 00:04:21,800 Did the truthful AI show a preference? 119 00:04:21,800 --> 00:04:23,640 Like consistently for one side or the other? 120 00:04:23,640 --> 00:04:24,600 Yeah, it did. 121 00:04:24,600 --> 00:04:27,480 In most cases, the AI rated the left leaning statements 122 00:04:27,480 --> 00:04:28,480 as more truthful. 123 00:04:28,480 --> 00:04:30,360 And the bigger the AI model, 124 00:04:30,360 --> 00:04:32,240 the stronger this bias seemed to be. 125 00:04:32,240 --> 00:04:34,480 Okay, so now we've got a real mystery on our hands. 126 00:04:34,480 --> 00:04:36,480 If these AI's are learning from like, you know, 127 00:04:36,480 --> 00:04:37,960 factual information, 128 00:04:37,960 --> 00:04:39,600 where's this political bias coming from? 129 00:04:39,600 --> 00:04:41,000 Right, it's a head scratcher. 130 00:04:41,000 --> 00:04:42,360 The researchers were stumped too. 131 00:04:42,360 --> 00:04:44,920 First they thought maybe it's in the data they used 132 00:04:44,920 --> 00:04:46,120 to train the AI. 133 00:04:46,120 --> 00:04:46,960 Yeah, that makes sense. 134 00:04:46,960 --> 00:04:48,440 But they looked really closely 135 00:04:48,440 --> 00:04:51,000 and there wasn't much political content there. 136 00:04:51,000 --> 00:04:53,000 So it's not like they were feeding the AI 137 00:04:53,000 --> 00:04:55,880 a bunch of political stuff, you know, to warp its view. 138 00:04:55,880 --> 00:04:57,280 Nope, not at all. 139 00:04:57,280 --> 00:04:58,120 Then they thought, okay, 140 00:04:58,120 --> 00:05:00,240 maybe there are these hidden clues 141 00:05:00,240 --> 00:05:01,800 in how the statements are written. 142 00:05:01,800 --> 00:05:02,680 Hidden clues. 143 00:05:02,680 --> 00:05:04,440 Yeah, like maybe truthful statements 144 00:05:04,440 --> 00:05:07,360 tend to use certain words or phrases 145 00:05:07,360 --> 00:05:09,320 that are also more common in, you know, 146 00:05:09,320 --> 00:05:10,720 left leaning language. 147 00:05:10,720 --> 00:05:13,680 Oh, so the AI might be picking up on those patterns 148 00:05:13,680 --> 00:05:15,240 even if we don't notice them. 149 00:05:15,240 --> 00:05:16,160 Exactly. 150 00:05:16,160 --> 00:05:17,840 But to test that, 151 00:05:17,840 --> 00:05:19,160 they used a simpler model 152 00:05:19,160 --> 00:05:21,000 that just focused on word patterns 153 00:05:21,000 --> 00:05:23,360 and they didn't find anything conclusive. 154 00:05:23,360 --> 00:05:25,080 So no sneaky clues there. 155 00:05:25,080 --> 00:05:26,600 So we've ruled out the training data 156 00:05:26,600 --> 00:05:29,880 and we've ruled out these sneaky clues in the language. 157 00:05:29,880 --> 00:05:32,200 That leaves us with a pretty big question mark. 158 00:05:32,200 --> 00:05:33,040 It does. 159 00:05:33,040 --> 00:05:34,760 The researchers basically said 160 00:05:34,760 --> 00:05:37,200 this bias might be coming from somewhere else entirely, 161 00:05:37,200 --> 00:05:38,840 something we haven't even thought of yet. 162 00:05:38,840 --> 00:05:40,640 They said they need to do a lot more research 163 00:05:40,640 --> 00:05:41,480 to figure it out. 164 00:05:41,480 --> 00:05:42,680 Yeah, sounds like it. 165 00:05:42,680 --> 00:05:45,240 This is turning into a real AI detective story. 166 00:05:45,240 --> 00:05:46,600 It is, it really is. 167 00:05:46,600 --> 00:05:49,240 So it's like we've got this truthful AI 168 00:05:49,240 --> 00:05:51,760 but it's showing this political slant 169 00:05:53,080 --> 00:05:54,640 and we don't really know why. 170 00:05:54,640 --> 00:05:55,840 It's kind of freaky, you know? 171 00:05:55,840 --> 00:05:57,520 Yeah, it definitely raises some questions. 172 00:05:57,520 --> 00:06:00,280 It just shows how complex these AI systems really are. 173 00:06:00,280 --> 00:06:02,720 Like we're only beginning to understand 174 00:06:02,720 --> 00:06:05,320 how they learn and process information 175 00:06:05,320 --> 00:06:06,360 and this research, it's like, 176 00:06:06,360 --> 00:06:08,920 whoa, there's a lot more going on than we thought. 177 00:06:08,920 --> 00:06:10,200 Yeah, it makes me wonder if there are other 178 00:06:10,200 --> 00:06:12,200 like hidden biases in these systems. 179 00:06:12,200 --> 00:06:13,760 Things we haven't even discovered yet. 180 00:06:13,760 --> 00:06:14,840 Kind of scary actually. 181 00:06:14,840 --> 00:06:15,680 It is a little bit, yeah. 182 00:06:15,680 --> 00:06:17,640 But going back to this political bias thing, 183 00:06:17,640 --> 00:06:22,120 did the researchers find it was like across all topics 184 00:06:22,120 --> 00:06:24,520 or did it show up more in some areas than others? 185 00:06:24,520 --> 00:06:25,680 That's a good question. 186 00:06:25,680 --> 00:06:29,240 They found it was strongest with topics like climate change, 187 00:06:29,240 --> 00:06:33,280 renewable energy and labor unions, issues 188 00:06:33,280 --> 00:06:34,600 where there's already a lot of debate 189 00:06:34,600 --> 00:06:36,080 and division in the real world. 190 00:06:36,080 --> 00:06:36,920 Right, right. 191 00:06:36,920 --> 00:06:38,880 So it's like the AI is picking up 192 00:06:38,880 --> 00:06:40,560 on those existing tensions 193 00:06:40,560 --> 00:06:42,040 and then somehow reflecting them 194 00:06:42,040 --> 00:06:43,600 in how it judges truth. 195 00:06:43,600 --> 00:06:44,680 That's pretty interesting. 196 00:06:44,680 --> 00:06:49,040 Did they find any topics where the bias was like less noticeable 197 00:06:49,040 --> 00:06:50,880 or maybe even flip the other way? 198 00:06:50,880 --> 00:06:51,720 They did actually. 199 00:06:51,720 --> 00:06:54,040 They found it was weaker or even reversed 200 00:06:54,040 --> 00:06:56,960 on topics like taxes and the death penalty. 201 00:06:56,960 --> 00:06:59,800 In those cases, the AI actually seemed to lean more 202 00:06:59,800 --> 00:07:01,520 toward conservative views. 203 00:07:01,520 --> 00:07:05,200 Huh, so it's not just like always leaning left. 204 00:07:05,200 --> 00:07:06,160 There's more to it than that. 205 00:07:06,160 --> 00:07:07,600 This is getting more and more interesting. 206 00:07:07,600 --> 00:07:08,520 It is, yeah. 207 00:07:08,520 --> 00:07:10,640 But you know, regardless of where it's coming from 208 00:07:10,640 --> 00:07:12,640 or which way it leans, this bias thing 209 00:07:12,640 --> 00:07:14,480 has some pretty big implications for AI. 210 00:07:14,480 --> 00:07:15,320 Don't you think? 211 00:07:15,320 --> 00:07:16,140 Oh, absolutely. 212 00:07:16,140 --> 00:07:19,480 If we want AI systems we can really rely on 213 00:07:19,480 --> 00:07:23,400 for accurate information, information that's not biased. 214 00:07:23,400 --> 00:07:25,480 We need to understand how this bias works 215 00:07:25,480 --> 00:07:27,840 and how to stop it from getting into the system. 216 00:07:27,840 --> 00:07:31,400 We might even have to rethink how we train AI completely. 217 00:07:31,400 --> 00:07:34,360 So it's not just about feeding AI more data. 218 00:07:34,360 --> 00:07:38,880 It's about being aware of these subtle ways bias can creep in 219 00:07:38,880 --> 00:07:41,960 even when we're trying to be objective about it. 220 00:07:41,960 --> 00:07:43,960 It really makes you wonder if we can ever create 221 00:07:43,960 --> 00:07:45,920 a perfectly objective AI. 222 00:07:45,920 --> 00:07:48,360 Yeah, I mean these systems are learning from data 223 00:07:48,360 --> 00:07:51,720 made by humans and we're all biased in some way, right? 224 00:07:51,720 --> 00:07:52,560 True, true. 225 00:07:52,560 --> 00:07:56,000 It's like, are we accidentally putting our own biases 226 00:07:56,000 --> 00:07:57,800 into these AI systems? 227 00:07:57,800 --> 00:08:00,460 And if so, what does that mean for, you know, 228 00:08:00,460 --> 00:08:02,760 how AI will be used in the future? 229 00:08:02,760 --> 00:08:04,520 Big questions, for sure. 230 00:08:04,520 --> 00:08:06,680 We can't ignore them, especially now that AI 231 00:08:06,680 --> 00:08:09,640 is becoming so, so integrated into our lives. 232 00:08:09,640 --> 00:08:12,280 Think about it, we're already using AI for things like 233 00:08:12,280 --> 00:08:16,040 hiring, loan approvals, even the criminal justice system. 234 00:08:16,040 --> 00:08:18,440 If those systems have hidden biases, 235 00:08:18,440 --> 00:08:20,440 that could have a huge impact on people's lives. 236 00:08:20,440 --> 00:08:22,680 Exactly, and it might not always be fair. 237 00:08:22,680 --> 00:08:23,520 Right. 238 00:08:23,520 --> 00:08:26,720 And these biases, they might not always be as clear as, 239 00:08:26,720 --> 00:08:28,440 you know, a political leaning. 240 00:08:28,440 --> 00:08:31,840 They could be about gender, race, religion, 241 00:08:31,840 --> 00:08:32,840 all sorts of things. 242 00:08:32,840 --> 00:08:33,760 It's a lot to think about. 243 00:08:33,760 --> 00:08:34,600 It is. 244 00:08:34,600 --> 00:08:36,480 So what does this all mean for, you know, 245 00:08:36,480 --> 00:08:39,360 everyone listening, the people using AI every day? 246 00:08:39,360 --> 00:08:40,520 What's the takeaway here? 247 00:08:40,520 --> 00:08:42,800 I think the biggest thing is awareness. 248 00:08:42,800 --> 00:08:45,640 We have to remember that even the smartest AI systems, 249 00:08:45,640 --> 00:08:46,680 they're not perfect. 250 00:08:46,680 --> 00:08:49,320 They can be influenced by bias, just like us. 251 00:08:49,320 --> 00:08:51,640 Even when we're trying our best to make them objective. 252 00:08:51,640 --> 00:08:53,480 So you can't just, like, blindly trust 253 00:08:53,480 --> 00:08:54,960 everything AI tells us. 254 00:08:54,960 --> 00:08:58,520 Exactly, don't be afraid to question the information, 255 00:08:58,520 --> 00:08:59,840 you know, look into it a bit more. 256 00:08:59,840 --> 00:09:01,160 Think about other perspectives, 257 00:09:01,160 --> 00:09:03,560 just like you would with any other information. 258 00:09:03,560 --> 00:09:05,040 That's good advice. 259 00:09:05,040 --> 00:09:08,120 Yeah, it's definitely a lot to consider. 260 00:09:08,120 --> 00:09:11,440 I mean, we're trying to use AI to find truth, 261 00:09:11,440 --> 00:09:15,160 but it turns out even that can be kind of messy. 262 00:09:15,160 --> 00:09:17,920 I guess, like you said, it just shows how complex 263 00:09:17,920 --> 00:09:19,120 all this stuff really is. 264 00:09:19,120 --> 00:09:20,440 It really does. 265 00:09:20,440 --> 00:09:22,800 So as we're wrapping up this deep dive, 266 00:09:22,800 --> 00:09:25,000 what's the one thing you really hope our listeners 267 00:09:25,000 --> 00:09:25,840 take away from this? 268 00:09:25,840 --> 00:09:28,560 What's that final thought they can keep in mind 269 00:09:28,560 --> 00:09:31,200 as they, you know, navigate this whole AI world? 270 00:09:31,200 --> 00:09:34,720 Okay, so next time you're interacting with an AI system, 271 00:09:34,720 --> 00:09:36,800 a chatbot, a search engine, whatever, 272 00:09:36,800 --> 00:09:39,880 just take a second to think about the data it was trained on. 273 00:09:39,880 --> 00:09:40,720 Okay, yeah. 274 00:09:40,720 --> 00:09:43,080 Like who put that data together, what their goals were, 275 00:09:43,080 --> 00:09:45,240 and think about whether there might be some, 276 00:09:45,240 --> 00:09:46,760 you know, unconscious biases 277 00:09:46,760 --> 00:09:48,440 shaping the information you're getting. 278 00:09:48,440 --> 00:09:49,720 That's a really good point. 279 00:09:49,720 --> 00:09:51,200 We should always be thinking critically 280 00:09:51,200 --> 00:09:53,400 about where information comes from, 281 00:09:53,400 --> 00:09:55,320 whether it's from a person or an AI. 282 00:09:55,320 --> 00:09:57,760 Exactly, it's a fascinating topic, 283 00:09:57,760 --> 00:10:01,360 and this research, it's just scratching the surface, 284 00:10:01,360 --> 00:10:04,360 but by being aware of these potential biases, 285 00:10:04,360 --> 00:10:07,000 we can start using AI in a more responsible way. 286 00:10:07,000 --> 00:10:08,680 So it's not about being afraid of AI, right? 287 00:10:08,680 --> 00:10:10,160 It's about understanding it, 288 00:10:10,160 --> 00:10:12,520 questioning it, and using it to make things better. 289 00:10:12,520 --> 00:10:15,680 Exactly, AI has so much potential to do good, 290 00:10:15,680 --> 00:10:17,320 but it's up to us to make sure it's developed 291 00:10:17,320 --> 00:10:19,080 in a way that benefits everyone. 292 00:10:19,080 --> 00:10:21,040 Couldn't have said it better myself. 293 00:10:21,040 --> 00:10:23,040 That's a perfect note to end on. 294 00:10:23,040 --> 00:10:26,760 We need to approach AI with like a healthy mix 295 00:10:26,760 --> 00:10:30,080 of curiosity and caution, recognizing its power, 296 00:10:30,080 --> 00:10:32,880 but also being aware of the potential downsides. 297 00:10:32,880 --> 00:10:35,520 Thanks for joining us for this deep dive into AI and truth. 298 00:10:35,520 --> 00:10:37,760 Until next time, keep exploring, keep questioning, 299 00:10:37,760 --> 00:11:03,760 and stay curious out there.