WEBVTT 00:00:00.000 --> 00:00:02.240 I want you to picture something for me. Think 00:00:02.240 --> 00:00:06.559 of a NASA spacecraft. Specifically, the 2006 00:00:06.559 --> 00:00:11.449 NASA ST -5. Oh yeah! The SD5. Right. So picture 00:00:11.449 --> 00:00:13.230 its communication antenna. I mean, if you're 00:00:13.230 --> 00:00:16.730 imagining a sleek, symmetrical satellite dish 00:00:16.730 --> 00:00:19.230 or maybe a perfectly straight metal rod. Like 00:00:19.230 --> 00:00:21.469 what a normal human engineer would build. Exactly. 00:00:21.550 --> 00:00:24.109 You're visualizing traditional design. But the 00:00:24.109 --> 00:00:26.850 SD5 antenna looks bizarre. It looks like a twisted, 00:00:27.370 --> 00:00:29.870 mangled paper clip. It really does. It's jagged. 00:00:29.989 --> 00:00:32.750 It's asymmetrical. To the untrained eye, it honestly 00:00:32.750 --> 00:00:36.030 looks like a manufacturing mistake. Human engineers 00:00:36.030 --> 00:00:39.469 did not design that shape. They didn't. And the 00:00:39.469 --> 00:00:41.429 computer that generated that shape didn't strictly 00:00:41.429 --> 00:00:43.729 design it either, at least not in the traditional 00:00:43.729 --> 00:00:46.049 top -down sense we usually associate with engineering. 00:00:46.189 --> 00:00:49.009 Right. It actually evolved it. It ran a program 00:00:49.009 --> 00:00:51.810 that simulated millions of generations of tiny 00:00:51.810 --> 00:00:54.609 random changes until the absolute optimal shape 00:00:54.609 --> 00:00:56.770 for broadcasting a signal just naturally emerged. 00:00:56.890 --> 00:00:58.869 It's just wild to think about. It completely 00:00:58.869 --> 00:01:01.229 challenges our basic assumptions about how technology 00:01:01.229 --> 00:01:03.929 is created. I mean, we are talking about biological 00:01:03.929 --> 00:01:06.750 code here. Yeah, it's a fascinating concept. 00:01:06.909 --> 00:01:09.790 So today, our mission on this deep dive is to 00:01:09.790 --> 00:01:12.689 unpack the Wikipedia article on genetic algorithms. 00:01:13.069 --> 00:01:16.090 We're going to explore how computer science borrows 00:01:16.090 --> 00:01:18.569 the rules of natural selection, like Charles 00:01:18.569 --> 00:01:21.409 Darwin's playbook, to solve these staggeringly 00:01:21.409 --> 00:01:23.609 complex problems. Right, because you see these 00:01:23.609 --> 00:01:26.909 algorithms applied all over the place, like hyperparameter 00:01:26.909 --> 00:01:29.310 optimization. Which is essentially tuning the 00:01:29.310 --> 00:01:32.390 complex dials of an AI model so it learns efficiently, 00:01:32.450 --> 00:01:34.969 right? Exactly. Or they're used in causal inference, 00:01:35.049 --> 00:01:37.480 which is the process of of untangling actual 00:01:37.480 --> 00:01:41.099 cause and effect relationships from massive messy 00:01:41.099 --> 00:01:43.400 piles of data. So for you listening, whether 00:01:43.400 --> 00:01:45.340 you're a coder or just someone fascinated by 00:01:45.340 --> 00:01:48.159 how nature inspires tech, this deep dive is going 00:01:48.159 --> 00:01:50.159 to give you a pretty powerful new mental model 00:01:50.159 --> 00:01:53.079 for problem solving. But before we get too far 00:01:53.079 --> 00:01:55.719 into the weeds, what exactly is a genetic algorithm? 00:01:55.920 --> 00:01:59.090 Well. A genetic algorithm, or a GA, is a specific 00:01:59.090 --> 00:02:02.450 type of metaheuristic. Metaheuristic. OK. Right. 00:02:02.890 --> 00:02:05.469 In computer science, a metaheuristic is basically 00:02:05.469 --> 00:02:08.110 a higher level procedure designed to find a good 00:02:08.110 --> 00:02:11.050 enough solution to an optimization problem, especially 00:02:11.050 --> 00:02:14.870 when you have incomplete information or limited 00:02:14.870 --> 00:02:17.050 computing capacity. Got it. So when the problem 00:02:17.050 --> 00:02:19.810 is just too big. Exactly. When you have a massive 00:02:19.810 --> 00:02:22.789 problem space, meaning there are so many possible 00:02:22.789 --> 00:02:26.069 solutions that a computer couldn't possibly check 00:02:26.069 --> 00:02:28.870 them all one by one. A GA searches that space 00:02:28.870 --> 00:02:31.930 using biologically inspired operators. Biologically 00:02:31.930 --> 00:02:35.449 inspired operators. Things like selection, crossover, 00:02:35.810 --> 00:02:38.449 and mutation. OK, but here's the fascinating 00:02:38.449 --> 00:02:41.060 tension we need to explore today. While NASA 00:02:41.060 --> 00:02:43.439 is flying these evolved designs in outer space, 00:02:43.819 --> 00:02:45.900 and logistics companies use them to route trucks, 00:02:46.439 --> 00:02:49.000 some highly respected computer scientists absolutely 00:02:49.000 --> 00:02:52.240 despise them. Oh, yeah. The haters are very vocal. 00:02:52.439 --> 00:02:55.120 Right. The source material features this notoriously 00:02:55.120 --> 00:02:57.500 harsh critique from Steven Skianna. He's the 00:02:57.500 --> 00:02:59.740 author of the widely read algorithm design manual, 00:02:59.840 --> 00:03:03.039 and he famously calls this entire approach pseudobiology 00:03:03.039 --> 00:03:06.780 voodoo. Voodoo. That's a strong word. He actively 00:03:06.780 --> 00:03:10.270 warns programmers to stay away from it. So as 00:03:10.270 --> 00:03:12.849 we walk through how this works, we need to figure 00:03:12.849 --> 00:03:17.009 out who is right. Is this a profound tool or 00:03:17.009 --> 00:03:19.919 just pseudoscientific hype? That's a great lens 00:03:19.919 --> 00:03:22.280 for this discussion. To evaluate Skeena's claim 00:03:22.280 --> 00:03:24.639 that it's just voodoo, we really have to look 00:03:24.639 --> 00:03:26.919 at the actual mechanics of the algorithm. We 00:03:26.919 --> 00:03:29.520 have to see how you actually encode evolution. 00:03:29.860 --> 00:03:31.699 Right. How do you code Charles Darwin? Well, 00:03:31.759 --> 00:03:34.740 it begins with creating a sort of digital glopagos. 00:03:34.919 --> 00:03:37.199 We start with a phase called initialization. 00:03:37.699 --> 00:03:40.099 The algorithm generates a massive population 00:03:40.099 --> 00:03:43.340 of completely random candidate solutions. So 00:03:43.340 --> 00:03:46.120 like the first generation of digital organisms. 00:03:46.460 --> 00:03:49.639 Exactly. And traditionally, The properties of 00:03:49.639 --> 00:03:52.240 these candidates, their chromosomes or genotypes, 00:03:52.379 --> 00:03:54.539 if we're keeping the biology metaphor, are represented 00:03:54.539 --> 00:03:57.379 simply as strings of binary code. Just strings 00:03:57.379 --> 00:03:59.900 of ones and zeros. Just ones and zeros. I mentioned 00:03:59.900 --> 00:04:01.819 most of those random first generation organisms 00:04:01.819 --> 00:04:03.460 are going to be absolutely terrible at whatever 00:04:03.460 --> 00:04:05.780 problem we're trying to solve. I mean, if you 00:04:05.780 --> 00:04:07.860 randomly mash together a shape for an antenna, 00:04:08.120 --> 00:04:09.759 it probably won't broadcast a signal at all. 00:04:10.099 --> 00:04:13.580 Oh, they're completely useless. That is the expected 00:04:13.580 --> 00:04:16.310 outcome for generation one. But that brings us 00:04:16.310 --> 00:04:18.750 to the next critical step, which is the fitness 00:04:18.750 --> 00:04:21.529 function. The fitness function. We have to evaluate 00:04:21.529 --> 00:04:24.470 every single individual in that generation to 00:04:24.470 --> 00:04:27.430 see how fit they are. The fitness function acts 00:04:27.430 --> 00:04:30.250 as the harsh environment that decides who lives 00:04:30.250 --> 00:04:32.829 and who dies. Let's ground this in a concrete 00:04:32.829 --> 00:04:34.790 example from the source material so you can really 00:04:34.790 --> 00:04:37.750 visualize the math here. There's a classic computer 00:04:37.750 --> 00:04:40.110 science puzzle called the knapsack problem. Oh, 00:04:40.110 --> 00:04:42.589 that's a perfect example. Right. So imagine you 00:04:42.589 --> 00:04:45.160 have a backpack. and it has a strict weight limit, 00:04:45.779 --> 00:04:49.560 let's say 15 pounds. And you have a collection 00:04:49.560 --> 00:04:51.939 of items you could put inside, each with a different 00:04:51.939 --> 00:04:54.139 weight and a different monetary value. So you 00:04:54.139 --> 00:04:55.899 have a lightweight first aid kit that's highly 00:04:55.899 --> 00:04:58.279 valuable, a heavy cast iron pan that's worth 00:04:58.279 --> 00:05:00.420 very little, maybe a moderately heavy tent that's 00:05:00.420 --> 00:05:04.040 very valuable, and so on. Your goal is to maximize 00:05:04.040 --> 00:05:07.680 the monetary value inside the bag without exceeding 00:05:07.680 --> 00:05:09.879 that 15 pound limit and ripping the bag apart. 00:05:10.079 --> 00:05:12.939 Right, so using a genetic algorithm to solve 00:05:12.939 --> 00:05:16.300 this means every candidate solution is represented 00:05:16.300 --> 00:05:19.439 by one of those binary strings. Each bit, each 00:05:19.439 --> 00:05:22.079 slot in the string represents a specific item 00:05:22.079 --> 00:05:25.500 from your collection. So a one in the first slot 00:05:25.500 --> 00:05:27.699 means the first aid kit goes into the backpack. 00:05:28.399 --> 00:05:30.839 A zero in the second slot means the cast iron 00:05:30.839 --> 00:05:34.500 pan stays out. Ah, okay, so... The fitness function 00:05:34.500 --> 00:05:36.899 comes along and looks at a specific binary string, 00:05:37.300 --> 00:05:40.759 say 101. It adds up the weight of the items represented 00:05:40.759 --> 00:05:42.800 by the ones. Exactly. And if the total weight 00:05:42.800 --> 00:05:46.100 is like 20 pounds, which exceeds your 15 pound 00:05:46.100 --> 00:05:48.620 capacity, what happens? Your fitness score drops 00:05:48.620 --> 00:05:51.360 to zero. That digital organism is dead. It failed 00:05:51.360 --> 00:05:54.329 the environment test. Wow. Brutal. Very. But 00:05:54.329 --> 00:05:56.110 if the weight is under the limit, your fitness 00:05:56.110 --> 00:05:58.649 score becomes the combined monetary value of 00:05:58.649 --> 00:06:00.790 those specific items. OK, so we've got our scores. 00:06:00.889 --> 00:06:03.069 What happens to the survivors? Once every candidate 00:06:03.069 --> 00:06:05.430 in that first generation is scored, the algorithm 00:06:05.430 --> 00:06:07.970 moves to the selection phase. It stochastically 00:06:07.970 --> 00:06:10.069 selects the individuals with the highest fitness 00:06:10.069 --> 00:06:14.170 scores. Stochastically, meaning randomly. Yes, 00:06:14.449 --> 00:06:18.089 but a weighted random process. Picture a loaded 00:06:18.089 --> 00:06:20.540 roulette wheel. OK. The candidates with the highest 00:06:20.540 --> 00:06:23.139 fitness scores get the widest slots on the wheel. 00:06:23.540 --> 00:06:25.480 So they aren't guaranteed to be selected, but 00:06:25.480 --> 00:06:27.879 the odds are heavily in their favor. The fittest 00:06:27.879 --> 00:06:31.199 survive. Got it. And then we apply genetic operators 00:06:31.199 --> 00:06:33.980 to create the second generation. The primary 00:06:33.980 --> 00:06:38.259 operator is crossover or recombination. This 00:06:38.259 --> 00:06:40.139 is where it gets really cool. The crossover phase 00:06:40.139 --> 00:06:42.300 is where the algorithm actually builds something 00:06:42.300 --> 00:06:44.910 new. Instead of just picking a winner, it combines 00:06:44.910 --> 00:06:47.269 them. I always think of it like breeding a champion 00:06:47.269 --> 00:06:49.350 racehorse, right? You take the endurance of one 00:06:49.350 --> 00:06:53.389 parent and the speed of another. Or imagine taking 00:06:53.389 --> 00:06:55.850 two different Lego vehicles. Okay, Legos. Yeah. 00:06:56.209 --> 00:06:58.410 You have a race car that scored high for speed 00:06:58.410 --> 00:07:01.170 and a monster truck that scored high for durability. 00:07:01.829 --> 00:07:04.129 You snap both of them in half down the middle 00:07:04.129 --> 00:07:06.889 and you push the front half of the race car onto 00:07:06.889 --> 00:07:09.420 the back wheels of the monster truck. That LEGO 00:07:09.420 --> 00:07:11.959 analogy perfectly illustrates the mechanical 00:07:11.959 --> 00:07:15.600 splicing. With binary strings, the computer literally 00:07:15.600 --> 00:07:18.699 slices the one and zero sequences of two successful 00:07:18.699 --> 00:07:22.079 parents at a random midpoint and splices them 00:07:22.079 --> 00:07:25.759 together to make a child chromosome. Wow. Literally 00:07:25.759 --> 00:07:27.939 cutting the code in half. Right. The assumption 00:07:27.939 --> 00:07:30.079 is that by combining the genetic material of 00:07:30.079 --> 00:07:32.860 two strong performers, the resulting child might 00:07:32.860 --> 00:07:35.620 inherit the best traits of both and achieve an 00:07:35.620 --> 00:07:38.959 even higher fitness score. But wait. If we just 00:07:38.959 --> 00:07:40.860 take the front half of the race car and the back 00:07:40.860 --> 00:07:42.579 half of the truck and we just keep recombining 00:07:42.579 --> 00:07:45.040 those same successful parts over and over across 00:07:45.040 --> 00:07:47.490 hundreds of generations. Wouldn't we eventually 00:07:47.490 --> 00:07:49.589 hit a wall? Yes. Like the population would just 00:07:49.589 --> 00:07:52.069 become a monoculture of identical clones? Isn't 00:07:52.069 --> 00:07:54.029 that just digital inbreeding the algorithm would 00:07:54.029 --> 00:07:56.709 run out of new ideas? That is a fundamental vulnerability. 00:07:56.790 --> 00:07:59.930 It's known as premature convergence. The algorithm 00:07:59.930 --> 00:08:02.709 gets stuck because it exhausted its genetic diversity 00:08:02.709 --> 00:08:05.029 too early. So how do you fix that? To prevent 00:08:05.029 --> 00:08:07.589 this, developers rely on the second genetic operator, 00:08:07.810 --> 00:08:12.670 which is mutation. Mutation. Right. Just as ultraviolet 00:08:12.670 --> 00:08:15.209 light might cause a mutation in biological DNA, 00:08:15.569 --> 00:08:17.129 the computer The computer occasionally flips 00:08:17.129 --> 00:08:19.829 a bit at random while copying the genetic code 00:08:19.829 --> 00:08:23.129 to the child. So a 1 becomes a 0. Or a 0 becomes 00:08:23.129 --> 00:08:25.949 a 1. Exactly. So going back to the analogies, 00:08:26.430 --> 00:08:28.569 a random item gets thrown into the backpack. 00:08:28.810 --> 00:08:32.129 or a random piece of Lego gets swept out, introducing 00:08:32.129 --> 00:08:34.809 a completely new trait that wasn't present in 00:08:34.809 --> 00:08:37.629 either parent. Yes. And developers also implement 00:08:37.629 --> 00:08:40.590 rules like the speciation heuristic to actively 00:08:40.590 --> 00:08:43.169 fight that monoculture. Speciation heuristic. 00:08:43.169 --> 00:08:45.889 What does that do? This rule mathematically analyzes 00:08:45.889 --> 00:08:48.990 the population and penalizes crossover between 00:08:48.990 --> 00:08:51.009 candidate solutions that are too genetically 00:08:51.009 --> 00:08:53.470 similar to each other. Oh, no. Yeah. It effectively 00:08:53.470 --> 00:08:55.950 forces the population to remain diverse by making 00:08:55.950 --> 00:08:58.870 sure distant cousins breed rather than That's 00:08:58.870 --> 00:09:00.850 incredibly clever. It is. And interestingly, 00:09:01.129 --> 00:09:03.210 while using two parents is the most intuitive 00:09:03.210 --> 00:09:05.730 biologically inspired method, the research notes 00:09:05.730 --> 00:09:08.710 something wild. Some algorithms generate much 00:09:08.710 --> 00:09:11.049 higher quality chromosomes by pulling genetic 00:09:11.049 --> 00:09:13.350 material from three or four parents to create 00:09:13.350 --> 00:09:16.309 a single child. Okay, now that is a place where 00:09:16.309 --> 00:09:19.190 computer science definitely leaves biology behind. 00:09:20.210 --> 00:09:22.710 We don't see four -parent organisms in nature. 00:09:22.769 --> 00:09:25.490 Very true. So we know the mechanics now. Selection, 00:09:25.610 --> 00:09:27.950 crossover, mutation. But we still need to address 00:09:27.950 --> 00:09:30.669 the underlying logic. Why does a random mashup 00:09:30.669 --> 00:09:33.789 of ones and zeros actually result in a brilliantly 00:09:33.789 --> 00:09:37.149 engineered spacecraft antenna instead of a pile 00:09:37.149 --> 00:09:39.529 of useless digital junk? Right. It seems like 00:09:39.529 --> 00:09:42.419 magic. But to understand the logic, we look to 00:09:42.419 --> 00:09:44.720 the building block hypothesis. This was proposed 00:09:44.720 --> 00:09:47.960 by John Holland in the 1970s. Holland suggested 00:09:47.960 --> 00:09:50.759 that genetic algorithms don't just blindly stumble 00:09:50.759 --> 00:09:53.220 into a massive perfectly formed answer all at 00:09:53.220 --> 00:09:56.059 once. Instead, they operate by identifying and 00:09:56.059 --> 00:09:58.720 recombining what he called schemata. Schemata. 00:09:58.860 --> 00:10:01.600 Like, pieces of a schema. Exactly. These are 00:10:01.600 --> 00:10:04.240 short, low order partial solutions. Just tiny 00:10:04.240 --> 00:10:06.179 fragments of the binary string that happen to 00:10:06.179 --> 00:10:08.779 offer a slight fitness advantage. The text actually 00:10:08.779 --> 00:10:10.840 includes a great quote from David E. Goldberg 00:10:10.840 --> 00:10:14.600 that visualizes this perfectly. He wrote, uh, 00:10:14.600 --> 00:10:17.200 just as a child creates magnificent fortresses 00:10:17.200 --> 00:10:19.399 through the arrangement of simple blocks of wood, 00:10:19.960 --> 00:10:22.659 so does a genetic algorithm seek near optimal 00:10:22.659 --> 00:10:25.500 performance through the juxtaposition of short, 00:10:25.659 --> 00:10:28.799 low order, high performance schemata. That's 00:10:28.799 --> 00:10:31.000 the core of it. The algorithm might discover 00:10:31.000 --> 00:10:33.240 through random mutation that a specific sequence 00:10:33.240 --> 00:10:36.200 of just three bits works remarkably well for 00:10:36.200 --> 00:10:38.799 the base of our antenna. Right. Simultaneously 00:10:38.799 --> 00:10:40.519 in a totally different part of the population, 00:10:40.720 --> 00:10:43.159 it discovers a sequence of four bits that works 00:10:43.159 --> 00:10:45.700 well for the tip of the antenna. Over successive 00:10:45.700 --> 00:10:48.500 generations through crossover, those two high 00:10:48.500 --> 00:10:50.639 -performing building blocks naturally find each 00:10:50.639 --> 00:10:53.519 other and snap together. Wow. We're talking about 00:10:53.519 --> 00:10:55.620 combining hundreds or thousands of these tiny 00:10:55.620 --> 00:10:57.960 building block schemata over millions of generations 00:10:57.960 --> 00:11:01.419 to form something as complex as an antenna. Yes. 00:11:01.620 --> 00:11:04.080 That sounds computationally exhausting. At what 00:11:04.080 --> 00:11:06.980 point does the sheer math of running this simulation 00:11:06.980 --> 00:11:09.659 weigh the benefits? And that brings us to the 00:11:09.659 --> 00:11:12.399 severe limitations of this approach. It really 00:11:12.399 --> 00:11:15.080 begins to explain why critics like Steven Skeena 00:11:15.080 --> 00:11:18.039 are so wary of it. Right, the voodoo guy. Exactly, 00:11:18.340 --> 00:11:21.759 because evaluating a digital backpack is instantaneous. 00:11:22.440 --> 00:11:24.960 But imagine a structural optimization problem, 00:11:25.399 --> 00:11:27.960 like evolving the aerodynamic shape of a vehicle. 00:11:28.200 --> 00:11:31.600 Oh, man. Yeah. A single fitness evaluation there 00:11:31.600 --> 00:11:34.019 requires running a computational fluid dynamic 00:11:34.019 --> 00:11:37.279 simulation, which means simulating how air flows 00:11:37.279 --> 00:11:40.679 over every square millimeter of a 3D shape in 00:11:40.679 --> 00:11:43.659 real time. So testing just one candidate solution 00:11:43.659 --> 00:11:46.639 might take, what, hours? Hours or even days on 00:11:46.639 --> 00:11:49.440 a supercomputer. Just for one. If a single generation 00:11:49.440 --> 00:11:51.840 has a thousand candidates and you need to run 00:11:51.840 --> 00:11:54.519 thousands of generations to evolve a good design, 00:11:54.830 --> 00:11:57.789 That process would literally take years. Exactly. 00:11:58.169 --> 00:12:00.990 Genetic algorithms simply do not scale well with 00:12:00.990 --> 00:12:03.590 massive complexity because the search space explodes 00:12:03.590 --> 00:12:06.629 exponentially. This is why you might see an evolutionary 00:12:06.629 --> 00:12:09.090 algorithm used to encode the design for a single 00:12:09.090 --> 00:12:11.470 fan blade, but you would never use it to evolve 00:12:11.470 --> 00:12:14.230 the design for an entire jet engine. The computational 00:12:14.230 --> 00:12:17.100 cost is just astronomical. Way too high. The 00:12:17.100 --> 00:12:20.440 sources also point out another massive flaw in 00:12:20.440 --> 00:12:22.860 how these algorithms navigate their problem space. 00:12:23.480 --> 00:12:26.200 They're incredibly vulnerable to getting trapped 00:12:26.200 --> 00:12:29.080 on what is called a local optima. Right, the 00:12:29.080 --> 00:12:32.159 local optima trap. Picture the fitness landscape 00:12:32.159 --> 00:12:35.080 as a vast mountain range. Your goal is to reach 00:12:35.080 --> 00:12:37.980 the absolute highest peak, the global optimum. 00:12:38.220 --> 00:12:41.220 The Mount Everest of solutions. Yes. But a genetic 00:12:41.220 --> 00:12:44.200 algorithm operates somewhat like a blindfolded 00:12:44.200 --> 00:12:46.919 climber. It's very good at feeling the slope 00:12:46.919 --> 00:12:49.220 of the ground directly under its feet and walking 00:12:49.220 --> 00:12:52.820 uphill. But it has no vision of the broader landscape. 00:12:52.899 --> 00:12:54.759 OK, I see where this is going. It might climb 00:12:54.759 --> 00:12:57.200 up the nearest smallest hill. Once it reaches 00:12:57.200 --> 00:12:59.279 the top of that small hill, any step it takes 00:12:59.279 --> 00:13:01.559 in any direction goes downward. So the algorithm 00:13:01.559 --> 00:13:04.019 assumes it's finished the job. found the peak. 00:13:04.340 --> 00:13:06.080 Right. It doesn't realize there is a massive 00:13:06.080 --> 00:13:08.919 Mount Everest sitting just one mile away, because 00:13:08.919 --> 00:13:10.799 getting to Mount Everest would require walking 00:13:10.799 --> 00:13:13.220 down into the valley first. It doesn't know how 00:13:13.220 --> 00:13:15.980 to sacrifice short -term fitness for long -term 00:13:15.980 --> 00:13:18.279 gains. So it's stuck on a molehill thinking it's 00:13:18.279 --> 00:13:21.340 on Everest. Which brings us fully back to Stevens 00:13:21.340 --> 00:13:23.759 -Guinness critique in the algorithm design manual. 00:13:24.320 --> 00:13:26.580 He argues that modeling applications with genetic 00:13:26.580 --> 00:13:30.100 operators on bit strings is, in his words, quite 00:13:30.100 --> 00:13:32.769 unnatural. He really doesn't pull his punches. 00:13:33.009 --> 00:13:35.509 He believes the pseudobiology adds an unnecessary 00:13:35.509 --> 00:13:38.149 layer of complexity between the programmer and 00:13:38.149 --> 00:13:41.029 the actual problem. He recommends abandoning 00:13:41.029 --> 00:13:43.990 Darwin entirely and sticking to what he calls 00:13:43.990 --> 00:13:46.570 simulated annealing for heuristic search needs. 00:13:47.110 --> 00:13:49.529 Right. And simulated annealing borrows its logic 00:13:49.529 --> 00:13:52.570 from metallurgy rather than biology, right? It 00:13:52.570 --> 00:13:54.690 does. It's inspired by the controlled cooling 00:13:54.690 --> 00:13:58.049 of materials to reduce structural defects. Simulated 00:13:58.049 --> 00:14:00.289 annealing introduces a temperature parameter 00:14:00.289 --> 00:14:02.490 to the search process. A temperature parameter? 00:14:02.590 --> 00:14:05.289 How does that work? When the temperature is artificially 00:14:05.289 --> 00:14:08.129 high at the start of the algorithm, it's programmed 00:14:08.129 --> 00:14:11.309 to frequently accept worse solutions. Going back 00:14:11.309 --> 00:14:13.830 to our blindfolded climber, a high temperature 00:14:13.830 --> 00:14:16.129 means the climber is willing to wander downhill 00:14:16.129 --> 00:14:19.090 into the valleys. Oh. which allows them to escape 00:14:19.090 --> 00:14:22.309 those small local hills. Exactly. And as the 00:14:22.309 --> 00:14:24.429 algorithm runs, the temperature slowly cools 00:14:24.429 --> 00:14:27.009 down and the climber becomes more strictly focused 00:14:27.009 --> 00:14:31.090 on climbing purely upward. Skiana vastly prefers 00:14:31.090 --> 00:14:34.190 this mathematical thermodynamic approach over 00:14:34.190 --> 00:14:36.769 tracking digital chromosomes. Okay, he calls 00:14:36.769 --> 00:14:39.909 genetic algorithms voodoo, but we know for a 00:14:39.909 --> 00:14:42.070 fact they're successfully used out in the wild. 00:14:42.440 --> 00:14:45.419 I mean the source lists them being used to schedule 00:14:45.419 --> 00:14:48.860 incredibly complex timetables, design aerodynamic 00:14:48.860 --> 00:14:51.879 bodies, and generate walking methods for computer 00:14:51.879 --> 00:14:54.629 animated figures. They absolutely are. So, if 00:14:54.629 --> 00:14:56.850 Skeena's criticisms about premature convergence 00:14:56.850 --> 00:14:59.429 and getting stuck on local hills are valid, how 00:14:59.429 --> 00:15:02.070 do modern developers actually fix those flaws 00:15:02.070 --> 00:15:04.350 to make these algorithms work? Well, they fix 00:15:04.350 --> 00:15:07.450 them by evolving the algorithm itself. Developers 00:15:07.450 --> 00:15:09.649 have created powerful variants of the standard 00:15:09.649 --> 00:15:12.330 genetic algorithm to address every one of Skeena's 00:15:12.330 --> 00:15:14.870 points. Oh, like a meta -evolution. Pretty much. 00:15:14.950 --> 00:15:17.629 Let's look at elitism, for example. In a standard 00:15:17.629 --> 00:15:19.750 crossover and mutation process, there's always 00:15:19.750 --> 00:15:21.990 a mathematical risk that you accidentally destroy 00:15:21.990 --> 00:15:23.950 your absolute best solution while trying to breed 00:15:23.950 --> 00:15:26.110 it. Like you mutate the one perfect gene it had? 00:15:26.570 --> 00:15:30.250 Exactly. Elitism is a variant rule that mandates 00:15:30.250 --> 00:15:33.090 the absolute fittest organism, or maybe a small 00:15:33.090 --> 00:15:35.190 group of the best, is carried over to the next 00:15:35.190 --> 00:15:38.789 generation completely unaltered. It guarantees 00:15:38.789 --> 00:15:42.149 your solution quality will never drop from one 00:15:42.149 --> 00:15:44.720 generation to the next. You essentially keep 00:15:44.720 --> 00:15:47.340 your champion safe on the sidelines, preserving 00:15:47.340 --> 00:15:49.580 that perfect genetic sequence while the rest 00:15:49.580 --> 00:15:52.059 of the population continues to breed and mutate. 00:15:52.519 --> 00:15:54.240 Right. What about the problem of the algorithm 00:15:54.240 --> 00:15:56.419 losing its diversity and getting stuck on a small 00:15:56.419 --> 00:15:59.279 hill? Developers solve that with adaptive genetic 00:15:59.279 --> 00:16:03.360 algorithms, or AGAs. In an early basic GA, a 00:16:03.360 --> 00:16:05.500 programmer might hard code the mutation rate 00:16:05.500 --> 00:16:09.720 at a fixed 1%, but an adaptive GA monitors its 00:16:09.720 --> 00:16:12.960 own progress. It watches itself. Yeah. It adaptively 00:16:12.960 --> 00:16:15.559 adjusts the probabilities of crossover and mutation 00:16:15.559 --> 00:16:18.220 on the fly based on the population's current 00:16:18.220 --> 00:16:20.580 fitness landscape. That is brilliant. It's like 00:16:20.580 --> 00:16:22.639 a video game adjusting its own difficulty behind 00:16:22.639 --> 00:16:24.919 the scenes. Exactly like that. The algorithm 00:16:24.919 --> 00:16:27.200 senses that every organism in the population 00:16:27.200 --> 00:16:30.039 is becoming too genetically similar, which means 00:16:30.039 --> 00:16:33.220 they're likely stuck on a local hill. So it automatically 00:16:33.220 --> 00:16:35.940 cranks up the mutation dial to 10 or 20 percent. 00:16:36.159 --> 00:16:39.299 Right. It forces a wave of weird, highly diverse 00:16:39.299 --> 00:16:41.620 children into the environment to see if one of 00:16:41.620 --> 00:16:45.120 those radical mutations happens to land on the 00:16:45.120 --> 00:16:47.340 slope of a bigger mountain. It's just so elegant. 00:16:47.610 --> 00:16:50.769 They also implement structural fixes for how 00:16:50.769 --> 00:16:53.490 the chromosomes themselves are written. The text 00:16:53.490 --> 00:16:56.269 highlights a technique called gray coding, which 00:16:56.269 --> 00:16:58.629 solves a critical mathematical trap known as 00:16:58.629 --> 00:17:00.889 a Hamming wall. Walk us through the mechanics 00:17:00.889 --> 00:17:02.970 of a Hamming wall, because this gets to the core 00:17:02.970 --> 00:17:05.569 of why binary code can be so tricky for evolution. 00:17:05.849 --> 00:17:08.289 Okay, so in standard binary code, numbers are 00:17:08.289 --> 00:17:10.829 represented by patterns of ones and zeros. The 00:17:10.829 --> 00:17:14.589 number seven is written as zero, one, one one 00:17:14.589 --> 00:17:16.809 zero one one one and the number eight is written 00:17:16.809 --> 00:17:20.049 as one zero zero zero wait so every single digit 00:17:20.049 --> 00:17:23.710 flips exactly if your digital organism is currently 00:17:23.710 --> 00:17:25.970 at a fitness level of seven and the environment 00:17:25.970 --> 00:17:28.450 demands it evolved to an eight the algorithm 00:17:28.450 --> 00:17:31.009 has to flip four separate bits perfectly and 00:17:31.009 --> 00:17:33.470 simultaneously it has to change that zero to 00:17:33.470 --> 00:17:37.029 a one and all three ones to zeros if it relies 00:17:37.029 --> 00:17:39.789 on random mutation The odds of flipping four 00:17:39.789 --> 00:17:42.569 specific bits at the exact same time are incredibly 00:17:42.569 --> 00:17:45.230 low. Extremely low. If it only slips three of 00:17:45.230 --> 00:17:48.589 them, it might land on like the number 15 or 00:17:48.589 --> 00:17:51.329 the number three. It's a sheer wall in the fitness 00:17:51.329 --> 00:17:53.549 landscape. The organism can't make the leap. 00:17:53.670 --> 00:17:56.430 That's the hamming wall. So gray coding is an 00:17:56.430 --> 00:17:58.670 alternative binary numbering system designed 00:17:58.670 --> 00:18:02.019 specifically to dismantle those walls. In gray 00:18:02.019 --> 00:18:04.279 code, the numerical alphabet is rewritten so 00:18:04.279 --> 00:18:06.819 that any two successive numbers differ by only 00:18:06.819 --> 00:18:09.539 a single bit. Oh, wow. So moving from a 7 to 00:18:09.539 --> 00:18:12.519 an 8 only requires one tiny mutation. Right. 00:18:12.640 --> 00:18:15.059 It smooths out the massive leaps into a gentle 00:18:15.059 --> 00:18:17.839 staircase, making it much easier for the algorithm 00:18:17.839 --> 00:18:19.940 to inch its way toward a better solution without 00:18:19.940 --> 00:18:22.180 getting stuck. Seeing all these clever variants, 00:18:22.559 --> 00:18:25.680 adaptive GAs, elitism, gray coding, it becomes 00:18:25.680 --> 00:18:28.640 really clear that this isn't just pseudobiology 00:18:28.640 --> 00:18:32.019 voodoo. It is a highly refined mathematical tool. 00:18:32.160 --> 00:18:34.599 And it's worth noting that the timeline of this 00:18:34.599 --> 00:18:37.059 tool's development is astonishing. I mean, this 00:18:37.059 --> 00:18:40.299 idea of computational evolution is deeply foundational 00:18:40.299 --> 00:18:43.160 to computer science. It really is. Alan Turing, 00:18:43.420 --> 00:18:45.539 who is widely considered the father of modern 00:18:45.539 --> 00:18:48.019 computing, proposed the concept of an evolutionary 00:18:48.019 --> 00:18:50.759 learning machine in his papers back in 1950. 00:18:51.019 --> 00:18:54.640 1950. Yeah. And by 1954, a scientist named Nils 00:18:54.640 --> 00:18:57.460 Al -Bercelli was actually running computer simulations 00:18:57.460 --> 00:19:00.160 of evolution at the Institute for Advanced Study 00:19:00.160 --> 00:19:03.140 in Princeton. Consider the context of 1954 for 00:19:03.140 --> 00:19:05.180 you listening. They were working with massive 00:19:05.180 --> 00:19:07.460 room -sized machines that relied on punch cards 00:19:07.460 --> 00:19:09.920 and vacuum tubes. They barely had functional 00:19:09.920 --> 00:19:12.559 screens or memory storage, yet they were already 00:19:12.559 --> 00:19:14.619 attempting to encode Darwinian evolution into 00:19:14.619 --> 00:19:17.460 silicon. The theory was sound 70 years ago. It 00:19:17.460 --> 00:19:19.720 just took decades for hardware processing power 00:19:19.720 --> 00:19:22.099 to finally catch up to the vision. The persistence 00:19:22.099 --> 00:19:24.859 of the idea speaks to its underlying power. It 00:19:24.859 --> 00:19:27.200 offers a paradigm shift in how we approach problem 00:19:27.200 --> 00:19:29.789 -solving. It absolutely does. If you take away 00:19:29.789 --> 00:19:32.750 one mental model from this deep dive, let it 00:19:32.750 --> 00:19:36.730 be this. Genetic algorithms remind us that sometimes 00:19:36.730 --> 00:19:39.509 the best way to solve a staggeringly complex 00:19:39.509 --> 00:19:43.269 challenge is not to meticulously design the perfect 00:19:43.269 --> 00:19:46.369 solution from the top down. It is to design the 00:19:46.369 --> 00:19:48.589 conditions for the perfect solution to emerge 00:19:48.589 --> 00:19:51.430 on its own. It's about setting the rules of survival, 00:19:51.789 --> 00:19:54.250 letting go of absolute control, and allowing 00:19:54.250 --> 00:19:57.420 iteration and time to do the heavy lifting. That 00:19:57.420 --> 00:20:00.039 is the ultimate value of the genetic algorithm. 00:20:00.539 --> 00:20:02.599 But before we finish, the source material notes 00:20:02.599 --> 00:20:05.119 one final critical limitation regarding dynamic 00:20:05.119 --> 00:20:07.500 datasets that I think leaves us with a pretty 00:20:07.500 --> 00:20:09.859 profound question about the systems we rely on 00:20:09.859 --> 00:20:12.960 every day. Right. How do genetic algorithms handle 00:20:12.960 --> 00:20:15.230 an environment that is constantly changing? They 00:20:15.230 --> 00:20:17.950 struggle immensely. Genetic algorithms are incredibly 00:20:17.950 --> 00:20:19.990 effective at finding a permanent solution for 00:20:19.990 --> 00:20:22.430 a static problem. The genomes converge early 00:20:22.430 --> 00:20:24.970 on toward a highly optimized shape or route for 00:20:24.970 --> 00:20:26.970 the exact data they have right now. OK, but what 00:20:26.970 --> 00:20:29.230 if that environment suddenly shifts? If the rules 00:20:29.230 --> 00:20:31.470 of the game change after the population has already 00:20:31.470 --> 00:20:34.349 converged, that highly evolved solution becomes 00:20:34.349 --> 00:20:36.890 entirely invalid. So all that work is just wiped 00:20:36.890 --> 00:20:40.039 out. Basically, developers try to patch this 00:20:40.039 --> 00:20:42.380 with techniques like triggered hypermutation, 00:20:42.700 --> 00:20:44.859 which is basically injecting chaos back into 00:20:44.859 --> 00:20:47.619 the system, or by dropping random immigrants 00:20:47.619 --> 00:20:50.500 into the gene pool to force new traits. But the 00:20:50.500 --> 00:20:53.960 core vulnerability remains. The organism is perfected 00:20:53.960 --> 00:20:56.339 for yesterday's environment. Think about how 00:20:56.339 --> 00:20:59.339 heavily our modern world relies on dynamic data. 00:20:59.640 --> 00:21:03.019 Supply chains, traffic routing software, financial 00:21:03.019 --> 00:21:05.500 markets, these are environments that change by 00:21:05.500 --> 00:21:08.220 the second. If we're relying on highly optimized, 00:21:08.539 --> 00:21:11.220 evolved algorithms to run the logistics of, say, 00:21:11.299 --> 00:21:13.720 a grocery store chain and a sudden weather event 00:21:13.720 --> 00:21:16.019 completely changes the shipping routes, the system 00:21:16.019 --> 00:21:18.220 might completely break down. It could. It makes 00:21:18.220 --> 00:21:20.759 you wonder about the fragility of perfect optimization. 00:21:21.359 --> 00:21:23.940 If the environment suddenly changes, will these 00:21:23.940 --> 00:21:25.940 digital organisms running our infrastructure 00:21:25.940 --> 00:21:29.779 be able to adapt in time? Or, because they've 00:21:29.779 --> 00:21:31.940 already locked in their genetic traits to be 00:21:31.940 --> 00:21:33.799 absolutely perfect for a world that no longer 00:21:33.799 --> 00:21:36.880 exists, are they doomed to immediate extinction? 00:21:37.200 --> 00:21:40.519 It's a scary thought. Picture that twisted, mangled 00:21:40.519 --> 00:21:42.980 NASA antenna again. It's the absolute perfect, 00:21:43.460 --> 00:21:45.980 flawless shape to transmit data in the exact 00:21:45.980 --> 00:21:48.839 atmospheric conditions it was evolved for. But 00:21:48.839 --> 00:21:51.339 if the atmosphere changes, it's just a bent piece 00:21:51.339 --> 00:21:53.400 of metal. Something for you to consider as you 00:21:53.400 --> 00:21:55.990 navigate. the highly optimized, ever -changing 00:21:55.990 --> 00:21:57.710 systems in your own life. Thanks for joining 00:21:57.710 --> 00:21:58.609 us on this deep dive.