1 00:00:00,000 --> 00:00:04,840 All right, get ready, because today we're diving into a paper about AI intelligence, 2 00:00:05,040 --> 00:00:09,480 but it's not just about like getting a computer pass a math test or something. 3 00:00:09,680 --> 00:00:11,640 It's way more interesting than that. 4 00:00:11,840 --> 00:00:12,120 Yeah. 5 00:00:12,320 --> 00:00:15,520 This paper says we need to go deeper. 6 00:00:15,720 --> 00:00:16,000 Yeah. 7 00:00:16,200 --> 00:00:19,440 Like understand how AI solves problems, 8 00:00:19,640 --> 00:00:23,120 especially those that require like abstract thinking, you know. 9 00:00:23,320 --> 00:00:24,440 It is fascinating. 10 00:00:24,640 --> 00:00:29,040 They use this framework called Algebraic Circuit Complexity 11 00:00:29,040 --> 00:00:31,800 to analyze how AI thinks. 12 00:00:31,800 --> 00:00:37,000 OK, so you're saying it's like we're trying to make a blueprint of AI's thought process. 13 00:00:37,000 --> 00:00:37,560 Exactly. 14 00:00:37,560 --> 00:00:38,040 Break it down. 15 00:00:38,040 --> 00:00:38,560 Break it down. 16 00:00:38,760 --> 00:00:41,080 Into like individual steps and connections. 17 00:00:41,280 --> 00:00:43,920 So imagine AI's thinking process. 18 00:00:44,120 --> 00:00:47,840 Like a circuit board with all these different paths and connections. 19 00:00:48,040 --> 00:00:53,800 Exactly. Those circuits represent how AI tackles these problems, especially ones 20 00:00:54,000 --> 00:00:57,080 that involve abstract concepts like we see in algebra. 21 00:00:57,280 --> 00:00:58,080 So like algebra. 22 00:00:58,080 --> 00:01:00,240 Yeah. And the complexity of those circuits. 23 00:01:00,440 --> 00:01:05,400 Things like how many connections they have are like the longest path from input to output. 24 00:01:05,600 --> 00:01:08,440 Those tell us how hard a problem is. 25 00:01:08,640 --> 00:01:09,760 For the AI to solve. 26 00:01:09,960 --> 00:01:11,000 Yeah, for the AI to solve. 27 00:01:11,200 --> 00:01:14,960 So we're not just seeing if the AI gets the right answer. 28 00:01:14,960 --> 00:01:15,240 Yeah. 29 00:01:15,440 --> 00:01:16,240 We're like, wait a minute. 30 00:01:16,440 --> 00:01:16,720 Exactly. 31 00:01:16,920 --> 00:01:17,880 How did you get there? 32 00:01:18,080 --> 00:01:18,960 It's the how. 33 00:01:18,960 --> 00:01:19,680 Here were the steps. 34 00:01:19,880 --> 00:01:21,280 Not just what, but how. 35 00:01:21,480 --> 00:01:21,680 Yeah. 36 00:01:21,880 --> 00:01:27,600 And it's about going beyond simply testing if AI can handle bigger numbers or longer 37 00:01:27,600 --> 00:01:28,240 equations. 38 00:01:28,440 --> 00:01:29,360 Right. Like you were saying. 39 00:01:29,560 --> 00:01:29,880 Yeah. 40 00:01:30,080 --> 00:01:31,760 Just because it can do a simple thing. 41 00:01:31,960 --> 00:01:32,440 Exactly. 42 00:01:32,640 --> 00:01:33,920 Doesn't mean it's actually thinking. 43 00:01:34,120 --> 00:01:36,720 It doesn't mean it understands the concepts behind it. 44 00:01:36,920 --> 00:01:37,040 Right. 45 00:01:37,240 --> 00:01:41,400 This paper argues that true symbolic reasoning is about understanding 46 00:01:41,600 --> 00:01:41,960 OK. 47 00:01:42,160 --> 00:01:44,560 The relationships between different pieces of information. 48 00:01:44,760 --> 00:01:47,080 So like we do in algebra when we solve for X. 49 00:01:47,280 --> 00:01:49,680 Exactly like when we solve for X in algebra. 50 00:01:49,880 --> 00:01:50,080 Got it. 51 00:01:50,280 --> 00:01:52,520 It's about understanding those relationships. 52 00:01:52,720 --> 00:01:53,640 Not just crunching numbers. 53 00:01:53,840 --> 00:01:54,000 Right. 54 00:01:54,200 --> 00:01:55,480 It's got to understand the concept. 55 00:01:55,480 --> 00:01:57,760 It's about understanding the concepts behind them. 56 00:01:57,960 --> 00:01:58,760 Yeah. Behind them. 57 00:01:58,960 --> 00:01:59,560 Yeah. OK. 58 00:01:59,760 --> 00:02:03,000 And the paper highlights this idea of composable functions. 59 00:02:03,200 --> 00:02:03,520 OK. 60 00:02:03,720 --> 00:02:09,400 Which is AI's ability to break down a complex problem into smaller parts. 61 00:02:09,600 --> 00:02:09,880 OK. 62 00:02:10,080 --> 00:02:12,240 Figure out the rules for each part. 63 00:02:12,440 --> 00:02:12,560 Right. 64 00:02:12,760 --> 00:02:13,840 And then put it all back together. 65 00:02:14,040 --> 00:02:15,040 To solve something new. 66 00:02:15,240 --> 00:02:16,640 To solve something new and more challenging. 67 00:02:16,840 --> 00:02:18,240 I'm thinking like learning to bake. 68 00:02:18,440 --> 00:02:18,640 Yes. 69 00:02:18,840 --> 00:02:21,120 You need to like master every little step. 70 00:02:21,320 --> 00:02:22,080 Perfect analogy. 71 00:02:22,280 --> 00:02:24,640 Before you can make like a crazy cake. 72 00:02:24,840 --> 00:02:25,200 Exactly. 73 00:02:25,200 --> 00:02:25,640 Here's something. 74 00:02:25,840 --> 00:02:28,440 One way researchers test this ability in AI. 75 00:02:28,640 --> 00:02:28,880 OK. 76 00:02:29,080 --> 00:02:32,600 Is through something called polynomial identity testing. 77 00:02:32,800 --> 00:02:34,360 Polynomial identity testing. 78 00:02:34,560 --> 00:02:37,800 Imagine showing AI two recipes that look totally different. 79 00:02:38,000 --> 00:02:38,200 OK. 80 00:02:38,400 --> 00:02:42,480 And asking do these actually make the same cake. 81 00:02:42,680 --> 00:02:43,280 Interesting. 82 00:02:43,480 --> 00:02:44,480 That's essentially what they're doing. 83 00:02:44,680 --> 00:02:45,200 They're seeing. 84 00:02:45,400 --> 00:02:49,840 Challenging AI to see through different forms of an equation. 85 00:02:50,040 --> 00:02:50,520 If it can. 86 00:02:50,720 --> 00:02:51,800 Like cut through the noise. 87 00:02:52,000 --> 00:02:53,960 That they are actually equivalent. 88 00:02:54,160 --> 00:02:54,240 OK. 89 00:02:54,240 --> 00:02:55,640 They might look different. 90 00:02:55,840 --> 00:02:56,120 Got it. 91 00:02:56,320 --> 00:02:57,880 But they're actually the same. 92 00:02:58,080 --> 00:02:59,960 You're like are they actually getting it. 93 00:03:00,160 --> 00:03:00,360 It is. 94 00:03:00,560 --> 00:03:01,040 They just like. 95 00:03:01,240 --> 00:03:01,560 You know. 96 00:03:01,760 --> 00:03:02,280 Parrot. 97 00:03:02,480 --> 00:03:03,320 Regurgitating. 98 00:03:03,520 --> 00:03:05,040 Just repeating what they've seen before. 99 00:03:05,240 --> 00:03:05,880 Yeah. 100 00:03:06,080 --> 00:03:06,320 OK. 101 00:03:06,520 --> 00:03:09,880 So it's like we're testing if it can see. 102 00:03:10,080 --> 00:03:10,560 Right. 103 00:03:10,760 --> 00:03:12,040 The underlying structure. 104 00:03:12,240 --> 00:03:13,600 The essence of the problem. 105 00:03:14,000 --> 00:03:16,040 Beyond just the superficial differences. 106 00:03:16,240 --> 00:03:16,520 Got it. 107 00:03:16,720 --> 00:03:19,240 And to really push AI's limits. 108 00:03:19,440 --> 00:03:22,640 The paper proposes a few specific benchmark. 109 00:03:22,840 --> 00:03:23,720 Exit like tests. 110 00:03:23,720 --> 00:03:23,920 Yeah. 111 00:03:24,120 --> 00:03:24,520 Tests. 112 00:03:24,720 --> 00:03:24,840 Right. 113 00:03:25,040 --> 00:03:28,520 One they call systematic compositional generalization. 114 00:03:28,720 --> 00:03:31,200 Systematic compositional generalization. 115 00:03:31,400 --> 00:03:38,120 It's like testing if AI after learning to combine basic ingredients can now create 116 00:03:38,320 --> 00:03:41,960 variations of a recipe at the same complexity level. 117 00:03:42,160 --> 00:03:44,040 So like once you've learned how to make cookies. 118 00:03:44,240 --> 00:03:44,520 Yes. 119 00:03:44,720 --> 00:03:47,120 Can you make different flavors of cookies. 120 00:03:47,320 --> 00:03:47,720 You got it. 121 00:03:47,920 --> 00:03:48,920 But it's still cookies. 122 00:03:49,120 --> 00:03:49,880 It's still cookies. 123 00:03:50,080 --> 00:03:50,360 OK. 124 00:03:50,360 --> 00:03:53,560 Then there's productive compositional generalization. 125 00:03:53,760 --> 00:03:55,360 Productive compositional generalization. 126 00:03:55,560 --> 00:03:57,440 Which is about taking things up a notch. 127 00:03:57,640 --> 00:04:05,280 This benchmark tests if AI can generalize to entirely new levels of complexity. 128 00:04:05,480 --> 00:04:06,400 So now we're talking like. 129 00:04:06,600 --> 00:04:08,240 Now we're talking instead of cookies. 130 00:04:08,440 --> 00:04:08,680 Yeah. 131 00:04:08,880 --> 00:04:09,720 It's a wedding cake. 132 00:04:09,920 --> 00:04:10,800 It's a wedding cake. 133 00:04:11,000 --> 00:04:12,320 A multi tiered wedding cake. 134 00:04:12,520 --> 00:04:13,200 Oh OK. 135 00:04:13,400 --> 00:04:17,520 We're talking about AI tackling problems that are significantly more complex. 136 00:04:17,720 --> 00:04:18,000 OK. 137 00:04:18,000 --> 00:04:21,040 And these benchmarks involve changing the structure. 138 00:04:21,240 --> 00:04:21,440 OK. 139 00:04:21,640 --> 00:04:23,280 Those algebraic circuits we talked about. 140 00:04:23,480 --> 00:04:23,560 OK. 141 00:04:23,760 --> 00:04:27,600 Making them larger and deeper to represent more complex calculations. 142 00:04:27,800 --> 00:04:29,480 So we're really trying to push AI. 143 00:04:29,680 --> 00:04:31,160 We're really pushing those limits. 144 00:04:31,360 --> 00:04:31,960 So what's limits here. 145 00:04:32,160 --> 00:04:33,480 To see how far it can go. 146 00:04:33,680 --> 00:04:34,240 See what it can do. 147 00:04:34,440 --> 00:04:36,560 With its symbolic reasoning muscles. 148 00:04:36,760 --> 00:04:36,880 Yeah. 149 00:04:37,080 --> 00:04:39,240 And it's not just about like I said before. 150 00:04:39,440 --> 00:04:41,280 It's not just about can it get the right answer. 151 00:04:41,480 --> 00:04:41,640 Yeah. 152 00:04:41,840 --> 00:04:43,200 But like how is it. 153 00:04:43,400 --> 00:04:45,760 It's about understanding the process. 154 00:04:45,960 --> 00:04:46,360 Getting there. 155 00:04:46,560 --> 00:04:47,600 Yeah. 156 00:04:47,600 --> 00:04:48,640 You're absolutely right. 157 00:04:48,840 --> 00:04:50,160 It's one of the biggest challenges in AI. 158 00:04:50,360 --> 00:04:50,600 Right. 159 00:04:50,800 --> 00:04:51,040 It is. 160 00:04:51,240 --> 00:04:52,520 Figuring out. 161 00:04:52,720 --> 00:04:52,920 What's. 162 00:04:53,120 --> 00:04:54,960 Keeking inside the black box. 163 00:04:55,160 --> 00:04:58,080 Trying to understand why AI makes the decisions it does. 164 00:04:58,280 --> 00:04:59,400 Like trying to see the gears turning. 165 00:04:59,600 --> 00:05:00,040 Exactly. 166 00:05:00,240 --> 00:05:01,560 Trying to see those gears turning. 167 00:05:01,760 --> 00:05:02,720 In the AI's brain. 168 00:05:02,920 --> 00:05:06,400 And the paper suggests that this framework. 169 00:05:06,600 --> 00:05:06,880 Uh huh. 170 00:05:07,080 --> 00:05:11,160 Of algebraic circuits could be a powerful tool for doing just that. 171 00:05:11,360 --> 00:05:11,600 OK. 172 00:05:11,800 --> 00:05:15,080 So researchers can compare what's happening inside AI. 173 00:05:15,280 --> 00:05:15,400 Yeah. 174 00:05:15,400 --> 00:05:19,720 For example where a language model focuses its attention. 175 00:05:19,920 --> 00:05:20,200 OK. 176 00:05:20,400 --> 00:05:23,960 To the actual circuit diagram that represents the problem. 177 00:05:24,160 --> 00:05:25,680 So we can see if it's thinking. 178 00:05:25,880 --> 00:05:26,120 Yeah. 179 00:05:26,320 --> 00:05:26,720 Like we would. 180 00:05:26,920 --> 00:05:28,120 Is it thinking like we do. 181 00:05:28,320 --> 00:05:28,640 Yeah. 182 00:05:28,840 --> 00:05:30,240 If it's solving the problem. 183 00:05:30,440 --> 00:05:32,920 Is it aligning with our understanding of math. 184 00:05:33,120 --> 00:05:34,440 In a way that makes sense to us. 185 00:05:34,640 --> 00:05:35,080 Exactly. 186 00:05:35,280 --> 00:05:35,520 OK. 187 00:05:35,720 --> 00:05:37,720 And this brings us to a really exciting point. 188 00:05:37,920 --> 00:05:38,320 OK. 189 00:05:38,520 --> 00:05:42,200 You know those impressive large language models like chat GPT. 190 00:05:42,400 --> 00:05:42,520 Yeah. 191 00:05:42,720 --> 00:05:43,840 Everybody's talking about those. 192 00:05:43,840 --> 00:05:45,360 Everyone's talking about them. 193 00:05:45,560 --> 00:05:45,640 Yeah. 194 00:05:45,840 --> 00:05:47,440 Well even these language models. 195 00:05:47,640 --> 00:05:47,880 OK. 196 00:05:48,080 --> 00:05:50,080 Can be evaluated using this framework. 197 00:05:50,280 --> 00:05:52,160 Wait so even though they're about language. 198 00:05:52,360 --> 00:05:52,560 Yeah. 199 00:05:52,760 --> 00:05:54,360 They could be thinking mathematically. 200 00:05:54,560 --> 00:05:55,080 Potentially. 201 00:05:55,280 --> 00:05:59,480 By designing problems that require abstract reasoning on variables. 202 00:05:59,680 --> 00:05:59,840 Uh huh. 203 00:06:00,040 --> 00:06:03,440 We can challenge them to go beyond just spitting out text. 204 00:06:03,640 --> 00:06:03,880 OK. 205 00:06:04,080 --> 00:06:06,240 And see if they can truly grasp. 206 00:06:06,440 --> 00:06:06,880 Wow. 207 00:06:07,080 --> 00:06:08,320 These complex concepts. 208 00:06:08,520 --> 00:06:09,280 This is blowing my mind. 209 00:06:09,480 --> 00:06:10,240 It's exciting stuff. 210 00:06:10,440 --> 00:06:11,080 So we're really. 211 00:06:11,280 --> 00:06:12,200 We're pushing those boundaries. 212 00:06:12,400 --> 00:06:13,320 Pushing the boundaries here. 213 00:06:13,320 --> 00:06:15,280 Of what we understand about AI. 214 00:06:15,480 --> 00:06:15,680 Yeah. 215 00:06:15,880 --> 00:06:19,840 And by understanding how AI tackles symbolic reasoning. 216 00:06:20,040 --> 00:06:20,240 OK. 217 00:06:20,440 --> 00:06:23,240 Through this lens of algebraic circuits. 218 00:06:23,440 --> 00:06:25,680 We can get a much clearer picture. 219 00:06:25,880 --> 00:06:26,680 Of its capability. 220 00:06:26,880 --> 00:06:29,880 Of what it's capable of and develop more reliable. 221 00:06:30,080 --> 00:06:30,400 OK. 222 00:06:30,600 --> 00:06:31,760 Interpretable. 223 00:06:31,960 --> 00:06:32,040 Yeah. 224 00:06:32,240 --> 00:06:35,000 And ultimately more trustworthy AI systems. 225 00:06:35,200 --> 00:06:36,320 This is fascinating. 226 00:06:36,520 --> 00:06:37,080 It is. 227 00:06:37,280 --> 00:06:38,680 So we've covered a lot of ground here. 228 00:06:38,880 --> 00:06:39,240 We have. 229 00:06:39,440 --> 00:06:41,760 I want to hear more about this whole circuit divergence idea. 230 00:06:41,960 --> 00:06:42,320 Absolutely. 231 00:06:42,320 --> 00:06:43,360 Because it sounds like that's. 232 00:06:43,560 --> 00:06:44,760 It's a key part of this research. 233 00:06:44,960 --> 00:06:45,680 Really important. 234 00:06:45,880 --> 00:06:49,640 It's all about how researchers are tweaking these circuits. 235 00:06:49,840 --> 00:06:50,080 OK. 236 00:06:50,280 --> 00:06:52,840 To create these different levels of challenge for AI. 237 00:06:53,040 --> 00:06:53,080 All right. 238 00:06:53,280 --> 00:06:54,280 So like. 239 00:06:54,480 --> 00:06:56,320 Like setting up an obstacle course. 240 00:06:56,520 --> 00:06:56,600 Yeah. 241 00:06:56,800 --> 00:06:57,720 An obstacle course. 242 00:06:57,920 --> 00:06:59,440 For its reasoning abilities. 243 00:06:59,640 --> 00:07:00,120 I love that. 244 00:07:00,320 --> 00:07:00,520 Yeah. 245 00:07:00,720 --> 00:07:00,840 OK. 246 00:07:01,040 --> 00:07:01,240 I'm ready. 247 00:07:01,440 --> 00:07:02,160 Let's unpack it. 248 00:07:02,360 --> 00:07:02,880 To unpack it. 249 00:07:03,080 --> 00:07:05,040 Let's dive deeper into the specifics. 250 00:07:05,240 --> 00:07:05,560 Right. 251 00:07:05,760 --> 00:07:07,200 Of how these circuits are designed. 252 00:07:07,400 --> 00:07:07,520 All right. 253 00:07:07,720 --> 00:07:10,520 And how they're being used to test AI's limits. 254 00:07:10,720 --> 00:07:11,480 Do it. 255 00:07:11,480 --> 00:07:12,960 Let's go. 256 00:07:13,160 --> 00:07:16,040 So this whole idea of circuit divergence. 257 00:07:16,240 --> 00:07:19,400 It's about creating these different versions. 258 00:07:19,600 --> 00:07:20,760 Of algebraic circuits. 259 00:07:20,960 --> 00:07:22,800 Each with its own level of complexity. 260 00:07:23,000 --> 00:07:24,560 Like different levels of difficulty. 261 00:07:24,760 --> 00:07:25,120 Yeah. 262 00:07:25,320 --> 00:07:25,600 Yeah. 263 00:07:25,800 --> 00:07:28,560 It's like setting up an obstacle course with different challenges. 264 00:07:28,760 --> 00:07:30,320 To test how far. 265 00:07:30,520 --> 00:07:30,920 I like that. 266 00:07:31,120 --> 00:07:31,840 AI can jump. 267 00:07:32,040 --> 00:07:32,520 So to speak. 268 00:07:32,720 --> 00:07:33,840 Yeah. 269 00:07:34,040 --> 00:07:36,160 But before we get into like the obstacles themselves. 270 00:07:36,360 --> 00:07:36,720 Yeah. 271 00:07:36,920 --> 00:07:38,080 Can we just like back up for a sec. 272 00:07:38,280 --> 00:07:38,480 Sure. 273 00:07:38,680 --> 00:07:40,520 And remind me how these circuits actually work. 274 00:07:40,720 --> 00:07:41,200 Of course. 275 00:07:41,200 --> 00:07:42,720 I think I still wrap in my head around it. 276 00:07:42,920 --> 00:07:44,200 So think of it like this. 277 00:07:44,400 --> 00:07:47,120 An algebraic circuit is like a visual representation. 278 00:07:47,320 --> 00:07:47,360 Oh yeah. 279 00:07:47,560 --> 00:07:48,760 Of a mathematical formula. 280 00:07:48,960 --> 00:07:50,560 Each part of the circuit. 281 00:07:50,760 --> 00:07:50,840 Yeah. 282 00:07:51,040 --> 00:07:52,160 It's called a gate. 283 00:07:52,360 --> 00:07:54,760 And it performs a specific operation. 284 00:07:54,960 --> 00:07:56,360 So it's kind of like those old calculators. 285 00:07:56,560 --> 00:07:56,920 Yeah. 286 00:07:57,120 --> 00:08:00,120 Where you'd input the numbers and then you'd press the plus sign or the minus sign. 287 00:08:00,440 --> 00:08:02,440 Like a plus or minus or multiply. 288 00:08:02,640 --> 00:08:02,840 Yeah. 289 00:08:03,040 --> 00:08:03,840 Anything like that. 290 00:08:04,040 --> 00:08:04,120 Yeah. 291 00:08:04,320 --> 00:08:05,320 Dealing with variables. 292 00:08:05,520 --> 00:08:05,680 OK. 293 00:08:05,880 --> 00:08:07,320 But this is all happening visually. 294 00:08:07,520 --> 00:08:07,680 OK. 295 00:08:07,880 --> 00:08:09,240 Like a flow chart for math. 296 00:08:09,440 --> 00:08:10,280 I like that flow chart. 297 00:08:10,680 --> 00:08:10,920 Yeah. 298 00:08:10,920 --> 00:08:13,080 So for a simple equation like I don't know. 299 00:08:13,280 --> 00:08:13,400 Yeah. 300 00:08:13,600 --> 00:08:14,600 Five plus seven. 301 00:08:14,800 --> 00:08:15,000 OK. 302 00:08:15,200 --> 00:08:18,480 So in a circuit you'd have two input gates. 303 00:08:18,680 --> 00:08:21,960 One for the number five one for the number seven. 304 00:08:22,160 --> 00:08:25,440 Those would feed into a sum gate which does the addition. 305 00:08:25,640 --> 00:08:25,880 OK. 306 00:08:26,080 --> 00:08:29,040 And the output of that sum gate would be 12. 307 00:08:29,240 --> 00:08:29,440 OK. 308 00:08:29,640 --> 00:08:31,880 So it's like five goes into one gate. 309 00:08:32,080 --> 00:08:32,400 One gate. 310 00:08:32,600 --> 00:08:33,840 Seven goes into the other. 311 00:08:34,040 --> 00:08:34,600 The other gate. 312 00:08:34,800 --> 00:08:36,440 And then they come together in the sum gate. 313 00:08:36,640 --> 00:08:37,240 In the sum gate. 314 00:08:37,440 --> 00:08:38,240 And it spits out 12. 315 00:08:38,440 --> 00:08:39,520 It spits out 12 exactly. 316 00:08:39,720 --> 00:08:39,800 OK. 317 00:08:40,000 --> 00:08:40,120 Got it. 318 00:08:40,120 --> 00:08:45,760 So for more complex equations you'd just have more gates and different types of gates. 319 00:08:45,960 --> 00:08:46,120 OK. 320 00:08:46,320 --> 00:08:49,840 Representing different operations all connected in a specific way. 321 00:08:50,040 --> 00:08:51,280 Like a more complex flow chart. 322 00:08:51,480 --> 00:08:53,600 Exactly a more complex flow chart. 323 00:08:53,800 --> 00:08:53,960 Right. 324 00:08:54,160 --> 00:08:58,520 And the beauty of this is we can visualize how AI tackles a problem. 325 00:08:58,720 --> 00:09:01,800 Step by step by following that flow through the circuit. 326 00:09:02,000 --> 00:09:02,320 OK. 327 00:09:02,520 --> 00:09:05,480 Now when researchers talk about circuit divergence. 328 00:09:05,680 --> 00:09:05,920 OK. 329 00:09:06,120 --> 00:09:08,400 They're essentially changing different aspects. 330 00:09:08,600 --> 00:09:08,840 OK. 331 00:09:08,840 --> 00:09:09,920 Of these circuits. 332 00:09:10,120 --> 00:09:10,200 Yeah. 333 00:09:10,400 --> 00:09:13,640 To create more challenging problems for the AI. 334 00:09:13,840 --> 00:09:13,880 OK. 335 00:09:14,080 --> 00:09:15,720 So like they're changing the obstacle course. 336 00:09:16,120 --> 00:09:17,320 They're like changing the hurdles. 337 00:09:17,520 --> 00:09:17,960 Exactly. 338 00:09:18,160 --> 00:09:19,160 Making them higher or lower. 339 00:09:19,360 --> 00:09:21,160 Making them higher or lower wider. 340 00:09:21,360 --> 00:09:21,560 You know. 341 00:09:21,760 --> 00:09:22,560 Different things like that. 342 00:09:22,920 --> 00:09:24,840 So what kind of changes are they making. 343 00:09:25,040 --> 00:09:28,000 So one way is by changing the depth of the circuit. 344 00:09:28,200 --> 00:09:29,040 The depth. 345 00:09:29,240 --> 00:09:31,120 Think of depth as the number of steps. 346 00:09:31,320 --> 00:09:31,480 OK. 347 00:09:31,680 --> 00:09:33,240 Involved in solving the problem. 348 00:09:33,440 --> 00:09:33,680 OK. 349 00:09:33,880 --> 00:09:37,080 A deeper circuit means a more complex calculation. 350 00:09:37,280 --> 00:09:38,120 So more steps. 351 00:09:38,120 --> 00:09:41,040 It requires more steps to reach that final answer. 352 00:09:41,240 --> 00:09:43,200 So it's like adding more layers to the cake. 353 00:09:43,400 --> 00:09:44,240 Perfect analogy. 354 00:09:44,440 --> 00:09:45,880 Adding more layers to that cake. 355 00:09:46,080 --> 00:09:47,400 Making it harder to bake. 356 00:09:47,600 --> 00:09:48,600 Making it more intricate. 357 00:09:48,800 --> 00:09:50,240 More time consuming to bake. 358 00:09:50,440 --> 00:09:50,800 Got it. 359 00:09:51,000 --> 00:09:52,760 And then there's the size of the circuit. 360 00:09:52,960 --> 00:09:53,240 OK. 361 00:09:53,440 --> 00:09:56,560 Which refers to the number of gates and connections involved. 362 00:09:56,760 --> 00:09:57,000 OK. 363 00:09:57,200 --> 00:10:01,440 A larger circuit generally means more complex calculations. 364 00:10:01,640 --> 00:10:01,880 OK. 365 00:10:02,080 --> 00:10:04,520 And more information for the AI to process. 366 00:10:04,720 --> 00:10:05,560 It's a bigger circuit. 367 00:10:05,760 --> 00:10:06,440 Bigger circuit. 368 00:10:06,640 --> 00:10:07,320 More ingredients. 369 00:10:07,320 --> 00:10:08,240 More ingredients. 370 00:10:08,440 --> 00:10:08,920 More steps. 371 00:10:09,120 --> 00:10:10,440 Harder to keep track of. 372 00:10:10,640 --> 00:10:13,440 More challenging for AI to keep track of everything. 373 00:10:13,640 --> 00:10:13,760 Yeah. 374 00:10:13,960 --> 00:10:15,240 And get that right outcome. 375 00:10:15,440 --> 00:10:15,720 Yeah. 376 00:10:15,920 --> 00:10:19,720 And by systematically varying these properties. 377 00:10:19,920 --> 00:10:20,120 OK. 378 00:10:20,320 --> 00:10:23,640 The depth, the size and even the types of gates used. 379 00:10:23,840 --> 00:10:24,120 OK. 380 00:10:24,320 --> 00:10:26,360 Researchers create this whole range of problems. 381 00:10:26,560 --> 00:10:27,520 To really test it. 382 00:10:27,720 --> 00:10:30,520 To test AI's symbolic reasoning abilities. 383 00:10:30,720 --> 00:10:31,520 To see what it can do. 384 00:10:31,720 --> 00:10:31,920 Yeah. 385 00:10:32,120 --> 00:10:36,840 Throwing everything they can at it to see how well it can adapt and generalize its knowledge. 386 00:10:36,840 --> 00:10:37,240 OK. 387 00:10:37,440 --> 00:10:38,160 So just to be clear. 388 00:10:38,360 --> 00:10:38,440 Yeah. 389 00:10:38,640 --> 00:10:39,960 We're not just talking about numbers here. 390 00:10:40,160 --> 00:10:41,440 We're not just talking about the numbers themselves. 391 00:10:41,840 --> 00:10:42,680 It's about concepts. 392 00:10:42,880 --> 00:10:46,520 It's about abstract concepts represented by variables. 393 00:10:46,720 --> 00:10:47,400 Like the X. 394 00:10:47,600 --> 00:10:48,200 Like X. 395 00:10:48,400 --> 00:10:48,760 Exactly. 396 00:10:48,960 --> 00:10:49,760 In algebra. 397 00:10:49,960 --> 00:10:55,480 The real test is whether AI can understand the relationships between those variables. 398 00:10:55,680 --> 00:10:57,680 Manipulate them according to the rules. 399 00:10:57,880 --> 00:10:58,040 OK. 400 00:10:58,240 --> 00:11:01,800 And come up with a solution that applies in different contexts. 401 00:11:02,000 --> 00:11:04,200 So it's not just memorizing. 402 00:11:04,400 --> 00:11:05,600 It's not about memorization. 403 00:11:05,800 --> 00:11:06,440 An equation. 404 00:11:06,440 --> 00:11:09,040 It's about understanding the underlying principles. 405 00:11:09,240 --> 00:11:09,520 OK. 406 00:11:09,720 --> 00:11:13,320 Those abstract ideas that govern these mathematical relationships. 407 00:11:13,520 --> 00:11:13,880 Got it. 408 00:11:14,080 --> 00:11:16,640 And that's where those benchmarks we talked about earlier come in. 409 00:11:16,840 --> 00:11:17,000 Right. 410 00:11:17,200 --> 00:11:19,440 Remember polynomial identity testing. 411 00:11:19,640 --> 00:11:19,840 Yeah. 412 00:11:20,040 --> 00:11:20,480 With the cakes. 413 00:11:20,680 --> 00:11:21,680 Remember the cake analogy. 414 00:11:21,880 --> 00:11:21,960 Yeah. 415 00:11:22,160 --> 00:11:26,200 Polynomial identity testing is about presenting AI with two different looking 416 00:11:26,400 --> 00:11:31,440 equations and asking are these actually the same just written in different ways. 417 00:11:31,640 --> 00:11:31,840 Got it. 418 00:11:32,040 --> 00:11:35,920 It's challenging AI to see through the superficial differences. 419 00:11:35,920 --> 00:11:36,320 Uh huh. 420 00:11:36,520 --> 00:11:39,920 And recognize that underlying mathematical equivalence. 421 00:11:40,120 --> 00:11:42,040 So it's like can you simplify this fraction. 422 00:11:42,240 --> 00:11:42,640 Exactly. 423 00:11:42,840 --> 00:11:43,760 And get to the same answer. 424 00:11:43,960 --> 00:11:46,880 Like recognizing that two fourths is the same as one half. 425 00:11:47,080 --> 00:11:47,200 Yeah. 426 00:11:47,400 --> 00:11:48,280 Even though it looks different. 427 00:11:48,480 --> 00:11:50,320 Even though they look different on the surface. 428 00:11:50,520 --> 00:11:50,720 Yeah. 429 00:11:50,920 --> 00:11:54,400 It's about understanding that core mathematical concept. 430 00:11:54,600 --> 00:11:55,600 It's about understanding it. 431 00:11:55,800 --> 00:11:58,400 Not just memorizing specific forms of an equation. 432 00:11:58,600 --> 00:11:58,880 OK. 433 00:11:59,080 --> 00:12:01,200 I've started to see how these tests can reveal. 434 00:12:01,400 --> 00:12:01,720 Yeah. 435 00:12:01,920 --> 00:12:04,400 A lot about how AI is actually thinking. 436 00:12:04,600 --> 00:12:04,960 They do. 437 00:12:04,960 --> 00:12:05,920 They really do. 438 00:12:06,120 --> 00:12:06,400 Yeah. 439 00:12:06,600 --> 00:12:10,960 And then there's the polynomial expansion and factorization benchmark. 440 00:12:11,160 --> 00:12:11,280 OK. 441 00:12:11,480 --> 00:12:12,520 So more polynomial. 442 00:12:12,720 --> 00:12:13,520 More polynomials. 443 00:12:13,720 --> 00:12:15,520 There's got to be a reason why we keep talking about this. 444 00:12:15,720 --> 00:12:18,880 There is polynomials are just a powerful way to represent. 445 00:12:19,080 --> 00:12:21,600 Oh, I complex mathematical relationships. 446 00:12:21,800 --> 00:12:22,000 OK. 447 00:12:22,200 --> 00:12:27,640 Think of polynomial expansion like translating a concise set of instructions. 448 00:12:27,840 --> 00:12:28,000 OK. 449 00:12:28,200 --> 00:12:31,520 Into a more elaborate program with all the details spelled out. 450 00:12:31,720 --> 00:12:31,920 OK. 451 00:12:32,120 --> 00:12:34,040 And then factorization is the reverse. 452 00:12:34,240 --> 00:12:34,400 OK. 453 00:12:34,400 --> 00:12:38,440 Taking that complex program and simplifying it back to those core instructions. 454 00:12:38,640 --> 00:12:39,560 So it can go back and forth. 455 00:12:39,760 --> 00:12:42,240 It's like going back and forth between different levels of complexity. 456 00:12:42,440 --> 00:12:42,560 Got it. 457 00:12:42,760 --> 00:12:49,320 And this ability is crucial for all sorts of tasks like understanding how AI might translate. 458 00:12:49,520 --> 00:12:49,840 Yeah. 459 00:12:50,040 --> 00:12:55,840 Between different programming languages or decipher complex scientific formulas. 460 00:12:56,040 --> 00:12:57,840 So it's not just about the math. 461 00:12:58,040 --> 00:12:58,960 It's not just about the math. 462 00:12:59,160 --> 00:13:00,000 It's about the thinking. 463 00:13:00,200 --> 00:13:02,760 It's about the underlying thinking and reasoning. 464 00:13:02,960 --> 00:13:03,120 OK. 465 00:13:03,120 --> 00:13:06,240 It's about those fundamental skills that are necessary. 466 00:13:06,440 --> 00:13:06,680 Yeah. 467 00:13:06,880 --> 00:13:09,080 For really any kind of complex problem solving. 468 00:13:09,280 --> 00:13:10,320 So these benchmarks. 469 00:13:10,520 --> 00:13:10,680 Yeah. 470 00:13:10,880 --> 00:13:13,160 They're really putting AI through its paces. 471 00:13:13,360 --> 00:13:13,680 They are. 472 00:13:13,880 --> 00:13:16,320 They're really challenging it to see what it can do. 473 00:13:16,520 --> 00:13:19,240 And one of the things that you said that they're looking for. 474 00:13:19,440 --> 00:13:19,760 Yes. 475 00:13:19,960 --> 00:13:21,520 Is this circuit divergence. 476 00:13:21,720 --> 00:13:22,920 It could diverge. 477 00:13:23,120 --> 00:13:24,880 I feel like we keep dancing around this idea. 478 00:13:25,080 --> 00:13:25,360 We do. 479 00:13:25,560 --> 00:13:25,920 We do. 480 00:13:26,120 --> 00:13:27,360 So can you break it down for me? 481 00:13:27,560 --> 00:13:27,840 Absolutely. 482 00:13:28,040 --> 00:13:28,480 What is it? 483 00:13:28,680 --> 00:13:32,520 So remember how we talked about researchers creating these different 484 00:13:32,520 --> 00:13:33,080 versions. 485 00:13:33,280 --> 00:13:33,360 Yeah. 486 00:13:33,560 --> 00:13:34,560 Different difficulty levels. 487 00:13:34,760 --> 00:13:37,680 Of circuits with varying levels of complexity. 488 00:13:37,880 --> 00:13:38,040 Uh-huh. 489 00:13:38,240 --> 00:13:40,000 Circuit divergence is a way to measure. 490 00:13:40,200 --> 00:13:40,480 OK. 491 00:13:40,680 --> 00:13:44,600 How much AI is problem solving strategy changes. 492 00:13:44,800 --> 00:13:45,000 OK. 493 00:13:45,200 --> 00:13:47,240 When it's faced with these different challenges. 494 00:13:47,440 --> 00:13:48,440 But they give it an easy one. 495 00:13:48,640 --> 00:13:48,840 Yeah. 496 00:13:49,040 --> 00:13:49,920 And then a hard one. 497 00:13:50,120 --> 00:13:50,480 Yeah. 498 00:13:50,680 --> 00:13:51,960 Does circuit divergence tell us. 499 00:13:52,160 --> 00:13:54,840 It tells us how much its approach has to adapt. 500 00:13:55,040 --> 00:13:56,000 To solve the harder one. 501 00:13:56,200 --> 00:13:57,640 To solve that harder problem. 502 00:13:57,840 --> 00:13:58,080 Got it. 503 00:13:58,280 --> 00:14:02,000 It's like observing how a chef's techniques might change. 504 00:14:02,000 --> 00:14:04,680 When they go from baking a simple cookie. 505 00:14:04,880 --> 00:14:05,160 OK. 506 00:14:05,360 --> 00:14:08,200 To creating a complex multi-layered cake. 507 00:14:08,400 --> 00:14:09,560 I like where this is going. 508 00:14:09,760 --> 00:14:12,280 Do they use completely different tools and methods. 509 00:14:12,480 --> 00:14:12,680 Yeah. 510 00:14:12,880 --> 00:14:15,720 Or are there some fundamental skills that carry over. 511 00:14:15,920 --> 00:14:16,200 OK. 512 00:14:16,400 --> 00:14:19,720 You know it's about understanding that adaptation process. 513 00:14:19,920 --> 00:14:21,320 So there's high circuit divergence. 514 00:14:21,520 --> 00:14:21,960 Yeah. 515 00:14:22,160 --> 00:14:24,920 It means that the AI had to really change things up. 516 00:14:25,120 --> 00:14:25,280 Yeah. 517 00:14:25,480 --> 00:14:29,160 It suggests that AI is maybe struggling to generalize its knowledge. 518 00:14:29,360 --> 00:14:34,000 That it's relying on very specific strategies that don't transfer well to more 519 00:14:34,200 --> 00:14:35,040 complex problems. 520 00:14:35,240 --> 00:14:35,440 OK. 521 00:14:35,640 --> 00:14:37,640 And a low degree of circuit divergence. 522 00:14:37,840 --> 00:14:38,640 So not a lot of change. 523 00:14:38,840 --> 00:14:43,040 Not a lot of change might indicate that AI has learned more fundamental principles. 524 00:14:43,240 --> 00:14:43,560 OK. 525 00:14:43,760 --> 00:14:46,880 That can be applied across different levels of complexity. 526 00:14:47,080 --> 00:14:51,480 So it's about understanding how well those core skills transfer. 527 00:14:51,680 --> 00:14:51,760 Yeah. 528 00:14:51,960 --> 00:14:53,320 To those more challenging tasks. 529 00:14:53,520 --> 00:14:54,920 So we can actually use these circuits. 530 00:14:55,120 --> 00:14:56,000 You can use these circuits. 531 00:14:56,200 --> 00:14:58,240 Not just to test if the AI can do it. 532 00:14:58,240 --> 00:15:00,200 Not just to test the abilities. 533 00:15:00,400 --> 00:15:01,640 But like see how it's learning. 534 00:15:01,840 --> 00:15:06,480 But to gain insights into its learning process and its problem solving strategies. 535 00:15:06,680 --> 00:15:07,400 That's really cool. 536 00:15:07,600 --> 00:15:08,000 It is. 537 00:15:08,200 --> 00:15:14,080 And this whole framework of algebraic circuit complexity is still relatively new. 538 00:15:14,280 --> 00:15:14,480 OK. 539 00:15:14,680 --> 00:15:17,720 So there's a lot more research to be done to be done in this area. 540 00:15:17,920 --> 00:15:18,760 But it's giving us new ways. 541 00:15:18,960 --> 00:15:24,400 It's opening up exciting new avenues of looking at AI for understanding how AI 542 00:15:24,600 --> 00:15:25,400 thinks and learns. 543 00:15:25,800 --> 00:15:26,640 So I'm following all this. 544 00:15:26,840 --> 00:15:26,960 Yeah. 545 00:15:26,960 --> 00:15:28,120 But I got to ask. 546 00:15:28,320 --> 00:15:28,360 Yeah. 547 00:15:28,560 --> 00:15:30,520 Like how much of this is actually applicable. 548 00:15:30,720 --> 00:15:31,640 That's a great question. 549 00:15:31,840 --> 00:15:32,480 To you know. 550 00:15:32,680 --> 00:15:32,840 Yeah. 551 00:15:33,040 --> 00:15:35,000 The AI we see in the real world. 552 00:15:35,200 --> 00:15:36,360 It's a very valid question. 553 00:15:36,560 --> 00:15:39,520 Because it seems very focused on these abstract math problems. 554 00:15:39,720 --> 00:15:40,120 And you're right. 555 00:15:40,320 --> 00:15:43,480 This research is very much focused on the foundations of symbolic reasoning. 556 00:15:43,680 --> 00:15:49,200 But the insights we gain from studying AI's ability to handle these abstract 557 00:15:49,400 --> 00:15:51,840 problems can have far reaching implications. 558 00:15:52,040 --> 00:15:52,280 OK. 559 00:15:52,480 --> 00:15:52,920 Tell me more. 560 00:15:53,120 --> 00:15:53,840 I'd be happy to. 561 00:15:54,040 --> 00:15:54,080 OK. 562 00:15:54,280 --> 00:15:56,400 So how does all this abstract math stuff. 563 00:15:56,400 --> 00:15:56,840 Yeah. 564 00:15:57,040 --> 00:15:58,480 Help us build better AI. 565 00:15:58,680 --> 00:16:03,520 So think of it like this symbolic reasoning the kind we're exploring with these circuits. 566 00:16:03,720 --> 00:16:03,840 OK. 567 00:16:04,040 --> 00:16:06,080 It's about understanding relationships. 568 00:16:06,280 --> 00:16:06,560 OK. 569 00:16:06,760 --> 00:16:08,320 Rules and logic. 570 00:16:08,520 --> 00:16:13,600 And those are fundamental skills for any kind of intelligent system. 571 00:16:13,800 --> 00:16:14,000 OK. 572 00:16:14,200 --> 00:16:18,480 Whether it's solving equations or navigating a busy city street. 573 00:16:18,680 --> 00:16:20,200 So even though we're testing it with math. 574 00:16:20,400 --> 00:16:20,680 Yeah. 575 00:16:20,880 --> 00:16:22,080 The skills are more general. 576 00:16:22,280 --> 00:16:22,880 Exactly. 577 00:16:22,880 --> 00:16:27,480 If we can understand how AI learns to manipulate these abstract concepts. 578 00:16:27,680 --> 00:16:27,920 OK. 579 00:16:28,120 --> 00:16:32,680 And solve complex problems in a structured domain like math. 580 00:16:32,880 --> 00:16:35,640 We can apply those insights to other areas. 581 00:16:35,840 --> 00:16:36,560 Like what kind of areas. 582 00:16:36,760 --> 00:16:38,160 Like natural language processing. 583 00:16:38,360 --> 00:16:38,560 OK. 584 00:16:38,760 --> 00:16:39,800 Image recognition. 585 00:16:40,000 --> 00:16:41,160 Even robotics. 586 00:16:41,360 --> 00:16:43,160 So it's like teaching the basics. 587 00:16:43,360 --> 00:16:47,520 It's like teaching AI the fundamental grammar of intelligence. 588 00:16:47,720 --> 00:16:48,080 I like that. 589 00:16:48,280 --> 00:16:48,880 So to speak. 590 00:16:49,080 --> 00:16:50,320 The grammar of intelligence. 591 00:16:50,320 --> 00:16:53,000 And this research also raises some fascinating questions. 592 00:16:53,200 --> 00:16:53,360 OK. 593 00:16:53,560 --> 00:16:56,480 About the nature of AI's learning process. 594 00:16:56,680 --> 00:16:57,120 Like what. 595 00:16:57,320 --> 00:17:01,320 For instance one of the things researchers are exploring is something called algebraic 596 00:17:01,520 --> 00:17:03,080 representation learning. 597 00:17:03,280 --> 00:17:05,040 Algebraic representation learning. 598 00:17:05,240 --> 00:17:05,560 Yeah. 599 00:17:05,760 --> 00:17:06,320 What is that. 600 00:17:06,520 --> 00:17:11,960 So traditionally we train AI by feeding it tons of data. 601 00:17:12,160 --> 00:17:13,840 Like showing it equations and answers. 602 00:17:14,040 --> 00:17:17,240 Showing it examples of equations and their solutions. 603 00:17:17,440 --> 00:17:17,680 OK. 604 00:17:17,680 --> 00:17:21,880 But algebraic representation learning is about going a step further. 605 00:17:22,080 --> 00:17:22,240 OK. 606 00:17:22,440 --> 00:17:26,440 And investigating how AI actually represents and manipulates. 607 00:17:26,640 --> 00:17:27,880 But we want to know how it's thinking. 608 00:17:28,080 --> 00:17:30,240 These algebraic concepts internally. 609 00:17:30,440 --> 00:17:32,080 Not just that it gets the right answer. 610 00:17:32,280 --> 00:17:34,560 It's not enough to know if AI gets the right answer. 611 00:17:34,760 --> 00:17:35,600 But how it gets there. 612 00:17:35,800 --> 00:17:37,320 We want to understand the thought process. 613 00:17:37,520 --> 00:17:37,680 Yeah. 614 00:17:37,880 --> 00:17:39,520 The internal language it's using. 615 00:17:39,720 --> 00:17:39,840 Yeah. 616 00:17:40,040 --> 00:17:40,720 To solve the problem. 617 00:17:40,920 --> 00:17:41,040 OK. 618 00:17:41,240 --> 00:17:42,200 And how do we do that. 619 00:17:42,400 --> 00:17:46,200 And this is where those algebraic circuits become really valuable tools. 620 00:17:46,400 --> 00:17:46,480 Right. 621 00:17:46,480 --> 00:17:48,440 Because they're like a visual representation. 622 00:17:48,640 --> 00:17:49,080 Exactly. 623 00:17:49,280 --> 00:17:49,840 The steps. 624 00:17:50,040 --> 00:17:53,840 They provide that visual representation of the problem solving steps. 625 00:17:54,040 --> 00:17:57,880 So researchers can compare AI's internal representation. 626 00:17:58,080 --> 00:17:58,720 To the circuits. 627 00:17:58,920 --> 00:18:01,280 To these circuits and see if they align. 628 00:18:01,480 --> 00:18:02,240 See if it's thinking. 629 00:18:02,440 --> 00:18:04,560 See if it's thinking the way we expect it to. 630 00:18:04,760 --> 00:18:05,400 The way that makes sense. 631 00:18:05,600 --> 00:18:05,800 Yeah. 632 00:18:06,000 --> 00:18:07,400 And one of the fascinating things. 633 00:18:07,600 --> 00:18:07,920 Yeah. 634 00:18:08,120 --> 00:18:12,240 Is that AI might develop its own internal algorithms. 635 00:18:12,440 --> 00:18:12,960 Whoa. 636 00:18:13,160 --> 00:18:16,160 Different from the ones that we explicitly program into it. 637 00:18:16,160 --> 00:18:18,920 So it's like it's like coming up with its own way. 638 00:18:19,120 --> 00:18:22,640 Figuring out their own unique way to solve a math problem. 639 00:18:22,840 --> 00:18:24,000 Of solving the problem. 640 00:18:24,200 --> 00:18:26,480 Even though they were taught a specific method. 641 00:18:26,680 --> 00:18:28,160 That's kind of scary. 642 00:18:28,360 --> 00:18:29,000 It's exciting. 643 00:18:29,200 --> 00:18:30,560 It's exciting but also like. 644 00:18:30,760 --> 00:18:32,480 But it also raises some questions. 645 00:18:32,680 --> 00:18:32,760 Yeah. 646 00:18:32,960 --> 00:18:34,080 Like what if we don't understand. 647 00:18:34,280 --> 00:18:34,680 Exactly. 648 00:18:34,880 --> 00:18:35,560 How do we ensure. 649 00:18:35,760 --> 00:18:36,480 Oh it's getting there. 650 00:18:36,680 --> 00:18:39,240 That these internal algorithms are reliable. 651 00:18:39,440 --> 00:18:39,680 Right. 652 00:18:39,880 --> 00:18:41,080 How do we debug them. 653 00:18:41,280 --> 00:18:42,400 If they lead to errors. 654 00:18:42,600 --> 00:18:44,200 Because it's not just a black box anymore. 655 00:18:44,400 --> 00:18:45,480 It's not just a black box. 656 00:18:45,480 --> 00:18:45,960 Right. 657 00:18:46,160 --> 00:18:46,920 We want to understand. 658 00:18:47,120 --> 00:18:50,440 We want to understand the reasoning so we can trust the decisions. 659 00:18:50,640 --> 00:18:50,800 OK. 660 00:18:51,000 --> 00:18:51,800 So what do we do. 661 00:18:52,000 --> 00:18:54,120 Well the paper highlights a few approaches. 662 00:18:54,320 --> 00:18:54,400 OK. 663 00:18:54,600 --> 00:18:55,560 For tackling this challenge. 664 00:18:55,760 --> 00:18:56,160 Right. 665 00:18:56,360 --> 00:19:00,160 One is using techniques called mechanistic interpretability. 666 00:19:00,360 --> 00:19:01,680 Mechanistic interpretability. 667 00:19:01,880 --> 00:19:06,520 It's all about analyzing AI's internal workings. 668 00:19:06,720 --> 00:19:10,720 Kind of like taking it apart and figuring out how each component functions. 669 00:19:10,920 --> 00:19:12,320 So we're like reverse engineering it. 670 00:19:12,520 --> 00:19:12,800 In a way. 671 00:19:13,000 --> 00:19:13,160 Yes. 672 00:19:13,360 --> 00:19:17,280 Remember how we talked about comparing the attention weights in a language model 673 00:19:17,480 --> 00:19:19,480 to the actual circuit diagram. 674 00:19:19,680 --> 00:19:19,840 Uh-huh. 675 00:19:20,040 --> 00:19:22,800 That's one example of mechanistic interpretability. 676 00:19:23,000 --> 00:19:23,080 OK. 677 00:19:23,280 --> 00:19:26,040 By looking at where AI focuses its attention. 678 00:19:26,240 --> 00:19:26,440 OK. 679 00:19:26,640 --> 00:19:29,240 We can get clues about how it's breaking down the problem. 680 00:19:29,640 --> 00:19:31,600 What aspects it considers most important. 681 00:19:31,800 --> 00:19:32,800 We'll kind of see how it's thinking. 682 00:19:33,000 --> 00:19:33,320 Exactly. 683 00:19:33,520 --> 00:19:34,680 We can see those gears turning. 684 00:19:34,880 --> 00:19:35,040 Yeah. 685 00:19:35,240 --> 00:19:39,080 Another promising approach is the development of neurosymbolic AI system. 686 00:19:39,280 --> 00:19:39,960 Norsymbolic. 687 00:19:39,960 --> 00:19:43,840 These systems combine elements of both symbolic AI, 688 00:19:44,040 --> 00:19:46,480 which is all about rules and logic. 689 00:19:46,680 --> 00:19:47,520 Right. 690 00:19:47,720 --> 00:19:48,840 And neural networks. 691 00:19:49,040 --> 00:19:49,240 OK. 692 00:19:49,440 --> 00:19:51,400 Which are inspired by the human brain. 693 00:19:51,600 --> 00:19:53,280 So it's like the best of both worlds. 694 00:19:53,480 --> 00:19:54,120 Exactly. 695 00:19:54,320 --> 00:19:55,800 Blending the best of both worlds. 696 00:19:56,000 --> 00:19:56,640 The logic. 697 00:19:56,840 --> 00:19:58,920 The precision of symbolic reasoning. 698 00:19:59,120 --> 00:19:59,960 And the flexibility. 699 00:20:00,160 --> 00:20:02,200 And the adaptability of neural networks. 700 00:20:02,400 --> 00:20:02,880 Club of brain. 701 00:20:03,080 --> 00:20:07,560 And the idea is that these hybrid systems might be more transparent. 702 00:20:07,760 --> 00:20:08,000 OK. 703 00:20:08,200 --> 00:20:09,200 And interpretable. 704 00:20:09,200 --> 00:20:10,280 So we can understand them. 705 00:20:10,480 --> 00:20:12,960 Then purely neural network based approaches. 706 00:20:13,160 --> 00:20:13,280 OK. 707 00:20:13,480 --> 00:20:14,640 So it's not just about. 708 00:20:14,840 --> 00:20:16,160 That's just an academic exercise. 709 00:20:16,360 --> 00:20:18,040 Making AI that can do stuff. 710 00:20:18,240 --> 00:20:24,000 It's about building AI that we can understand, trust and collaborate with. 711 00:20:24,200 --> 00:20:24,440 Wow. 712 00:20:24,640 --> 00:20:25,720 This has been really interesting. 713 00:20:25,920 --> 00:20:26,240 It has. 714 00:20:26,440 --> 00:20:27,280 We've learned a lot. 715 00:20:27,480 --> 00:20:30,840 We've gone from the nuts and bolts of algebraic circuits. 716 00:20:31,040 --> 00:20:31,240 Yeah. 717 00:20:31,440 --> 00:20:32,320 All the way to like. 718 00:20:32,520 --> 00:20:36,160 To the grand vision of trustworthy and collaborative AI. 719 00:20:36,360 --> 00:20:36,400 Yeah. 720 00:20:36,600 --> 00:20:37,640 The future. 721 00:20:37,640 --> 00:20:39,920 It's clear that there's still so much to discover. 722 00:20:40,120 --> 00:20:40,320 Yeah. 723 00:20:40,520 --> 00:20:42,280 But this research is pushing the boundaries. 724 00:20:42,480 --> 00:20:43,080 It really is. 725 00:20:43,280 --> 00:20:45,000 Of what we know about AI's capability. 726 00:20:45,200 --> 00:20:45,600 Wow. 727 00:20:45,800 --> 00:20:49,840 And paving the way for a future where humans and machines. 728 00:20:50,040 --> 00:20:50,360 Yeah. 729 00:20:50,560 --> 00:20:52,960 Can work together in truly meaningful ways. 730 00:20:53,160 --> 00:20:54,040 I completely agree. 731 00:20:54,240 --> 00:20:54,920 I couldn't agree more. 732 00:20:55,120 --> 00:20:56,280 It's a really exciting time. 733 00:20:56,480 --> 00:20:59,720 It is an exciting time to be at the forefront of AI research. 734 00:20:59,920 --> 00:21:00,960 You'd be working in AI. 735 00:21:01,160 --> 00:21:02,480 Unraveling the complexity. 736 00:21:02,680 --> 00:21:03,680 Of how these things think. 737 00:21:03,880 --> 00:21:06,000 Of how these systems think and learn. 738 00:21:06,000 --> 00:21:07,840 Thank you so much for. 739 00:21:08,040 --> 00:21:09,080 It was my pleasure. 740 00:21:09,280 --> 00:21:10,160 Talking to us about this. 741 00:21:10,360 --> 00:21:12,680 I'm always thrilled to share my passion for AI. 742 00:21:12,880 --> 00:21:13,000 Yeah. 743 00:21:13,200 --> 00:21:15,280 And explore these thought provoking ideas. 744 00:21:15,480 --> 00:21:17,200 And to all of our listeners out there. 745 00:21:17,400 --> 00:21:20,360 Thank you so much for joining us on this deep dive. 746 00:21:20,560 --> 00:21:21,920 And we hope this is. 747 00:21:22,120 --> 00:21:22,320 You know. 748 00:21:22,520 --> 00:21:23,600 Sparked your curiosity. 749 00:21:23,800 --> 00:21:24,560 Sparked your curiosity. 750 00:21:24,760 --> 00:21:26,200 About the future of AI. 751 00:21:26,400 --> 00:21:26,920 About AI. 752 00:21:27,120 --> 00:21:30,120 And the quest to understand its remarkable potential. 753 00:21:30,320 --> 00:21:30,640 Yeah. 754 00:21:30,840 --> 00:21:32,440 And keep exploring. 755 00:21:32,640 --> 00:21:34,360 Keep asking questions. 756 00:21:34,360 --> 00:21:36,560 Keep pushing the boundaries of what's possible. 757 00:21:36,760 --> 00:21:37,240 Exactly. 758 00:21:37,240 --> 00:22:05,320 Until next time.