WEBVTT 00:00:00.000 --> 00:00:03.520 It is really funny how quickly we just get used 00:00:03.520 --> 00:00:05.580 to magic. Oh, completely. We take it entirely 00:00:05.580 --> 00:00:09.019 for granted. Right. Like every single day, you 00:00:09.019 --> 00:00:11.320 probably dictate a text message to your phone 00:00:11.320 --> 00:00:13.820 while you're just walking down the street. Or 00:00:13.820 --> 00:00:17.179 you type half a question into a search engine, 00:00:17.660 --> 00:00:19.579 and it just instantly finishes your sentence 00:00:19.579 --> 00:00:21.699 for you. Exactly. It knows what you want before 00:00:21.699 --> 00:00:23.879 you do. Yeah. You interact with it constantly. 00:00:24.429 --> 00:00:27.030 But you probably rarely stop to think about the 00:00:27.030 --> 00:00:29.769 massive invisible machinery making all of that 00:00:29.769 --> 00:00:33.990 happen. So welcome to the deep dive. Today, our 00:00:33.990 --> 00:00:37.109 mission is to decode that exact machinery. It 00:00:37.109 --> 00:00:39.750 is quite the rabbit hole. It really is. We are 00:00:39.750 --> 00:00:41.909 looking at a comprehensive overview of natural 00:00:41.909 --> 00:00:44.850 language processing, or NLP, to figure out exactly 00:00:44.850 --> 00:00:47.670 how computers learn to understand, process, and 00:00:47.670 --> 00:00:49.670 even generate human language. And, you know, 00:00:49.789 --> 00:00:51.590 the journey to get to this point is just full 00:00:51.590 --> 00:00:54.549 of surprises, dead ends, and some truly profound 00:00:54.549 --> 00:00:57.210 questions about what it actually means to communicate. 00:00:57.329 --> 00:00:59.789 Oh, absolutely. We're going to see how the scientific 00:00:59.789 --> 00:01:02.009 community had to, well, completely change its 00:01:02.009 --> 00:01:04.250 mind several times over the decades to finally 00:01:04.250 --> 00:01:07.730 crack this code. It is a wild ride. And to get 00:01:07.730 --> 00:01:10.129 our heads around the core challenge here, I want 00:01:10.129 --> 00:01:13.870 you to imagine trying to teach a highly logical, 00:01:14.349 --> 00:01:17.450 incredibly literal minded alien how to appreciate 00:01:17.450 --> 00:01:20.400 a poem. That is a great analogy. Because for 00:01:20.400 --> 00:01:22.819 a computer, the math part is easy, right? Crunching 00:01:22.819 --> 00:01:25.459 numbers, storing massive spreadsheets, no problem 00:01:25.459 --> 00:01:28.840 at all. But human language is a nightmare. It 00:01:28.840 --> 00:01:31.480 really is. It's messy, it's full of sarcasm and 00:01:31.480 --> 00:01:34.659 double meanings, and the rules are broken as 00:01:34.659 --> 00:01:36.590 often as they are followed. If we connect this 00:01:36.590 --> 00:01:38.849 to the bigger picture, it is crucial to understand 00:01:38.849 --> 00:01:41.629 that NLP isn't just about building a better spell 00:01:41.629 --> 00:01:45.170 check. In the field of computer science, mastering 00:01:45.170 --> 00:01:48.049 natural language is actually considered an AI 00:01:48.049 --> 00:01:50.769 complete problem. Wait, AI complete? That sounds 00:01:50.769 --> 00:01:53.450 intense. Yeah, it does. What it means is that 00:01:53.450 --> 00:01:56.230 to fully solve language, you know, to get a machine 00:01:56.230 --> 00:01:58.829 to truly comprehend it the way you or I do, you 00:01:58.829 --> 00:02:00.849 essentially have to solve artificial intelligence 00:02:00.849 --> 00:02:02.930 itself. Because language doesn't exist in a vacuum. 00:02:03.390 --> 00:02:06.069 Exactly. To understand a sentence? You need real 00:02:06.069 --> 00:02:08.310 -world knowledge. You need an understanding of 00:02:08.310 --> 00:02:11.090 context, and you need logical reasoning. You 00:02:11.090 --> 00:02:13.270 can't just program a dictionary into a machine. 00:02:13.349 --> 00:02:15.349 You literally have to program an understanding 00:02:15.349 --> 00:02:17.789 of the entire human experience. Which is exactly 00:02:17.789 --> 00:02:19.710 why the early attempts to solve this problem 00:02:19.710 --> 00:02:22.770 were just, well, so overly optimistic. Oh, they 00:02:22.770 --> 00:02:25.710 were incredibly naive. Right. To understand how 00:02:25.710 --> 00:02:28.490 we got to the seemingly magical AI you have in 00:02:28.490 --> 00:02:30.729 your pocket today, we really have to start with 00:02:30.729 --> 00:02:34.069 how early computer scientists in the 1950s thought 00:02:34.069 --> 00:02:36.689 language worked. Basically, they treated it like 00:02:36.689 --> 00:02:39.909 rigid algebra. Yeah, the roots of NLP go all 00:02:39.909 --> 00:02:42.009 the way back to Alan Turing in the 1950s. I mean, 00:02:42.009 --> 00:02:44.930 he proposed the Turing test, which inherently 00:02:44.930 --> 00:02:48.370 relied on a machine's ability to interpret and 00:02:48.370 --> 00:02:52.150 generate natural language to prove its intelligence. 00:02:52.389 --> 00:02:54.550 Right. The ultimate benchmark. But the real burst 00:02:54.550 --> 00:02:56.849 of early confidence, though, that came in 1954 00:02:56.849 --> 00:02:58.849 was something called the Georgetown experiment. 00:02:58.969 --> 00:03:01.210 Oh, the Georgetown experiment, where the researchers 00:03:01.210 --> 00:03:03.949 managed to get a computer to automatically translate 00:03:03.949 --> 00:03:07.750 over 60 Russian sentences into English. Exactly. 00:03:08.909 --> 00:03:11.569 And based on that tiny, highly controlled success, 00:03:12.210 --> 00:03:15.169 the authors boldly claimed that machine translation 00:03:15.169 --> 00:03:18.370 would be a completely solved problem within like 00:03:18.370 --> 00:03:21.210 three to five years. Wow. Three to five years 00:03:21.210 --> 00:03:24.409 to solve language. It's a classic case of underestimating 00:03:24.409 --> 00:03:26.870 the complexity of a problem. They thought language 00:03:26.870 --> 00:03:29.680 was just a matter of swapping words and applying 00:03:29.680 --> 00:03:31.860 a few structural rules. Like, you look up the 00:03:31.860 --> 00:03:33.580 Russian word, you find the English equivalent, 00:03:33.780 --> 00:03:35.580 you swap them, and you apply a grammar rule. 00:03:35.740 --> 00:03:39.409 Boom. Done. Right. But real progress was agonizingly 00:03:39.409 --> 00:03:41.530 slow because language isn't just a simple cipher. 00:03:42.069 --> 00:03:45.449 And by 1966, a famous evaluation called the ALPAC 00:03:45.449 --> 00:03:47.669 report came out. Oh yeah, the ALPAC report. That 00:03:47.669 --> 00:03:50.409 was devastating. It brutally concluded that 10 00:03:50.409 --> 00:03:53.229 years of really expensive research had basically 00:03:53.229 --> 00:03:56.189 failed to fulfill any of those grand expectations. 00:03:56.330 --> 00:03:58.669 And that report didn't just hurt feelings. It 00:03:58.669 --> 00:04:01.469 crushed the entire industry. Funding for machine 00:04:01.469 --> 00:04:03.830 translation in the US was dramatically slashed, 00:04:04.310 --> 00:04:06.530 and serious research was largely paused until 00:04:06.530 --> 00:04:09.409 the late 1980s. Yeah, that whole era is often 00:04:09.409 --> 00:04:12.990 referred to as symbolic NLP. The philosophy was 00:04:12.990 --> 00:04:16.610 basically to hand code complex sets of rules 00:04:16.610 --> 00:04:19.470 to manipulate symbols. Which sounds good on paper. 00:04:19.709 --> 00:04:22.870 Sure, but to understand the fundamental flaw 00:04:22.870 --> 00:04:25.230 in this approach, you just have to look at John 00:04:25.230 --> 00:04:28.329 Searle's famous Chinese room thought experiment 00:04:28.329 --> 00:04:30.649 from the source text. Okay, lay that out for 00:04:30.649 --> 00:04:33.410 us. So imagine a person locked in a room who 00:04:33.410 --> 00:04:35.870 doesn't speak a single word of Chinese. They 00:04:35.870 --> 00:04:38.430 are given a batch of Chinese writing and a giant 00:04:38.430 --> 00:04:41.089 rule book in English. And that rule book tells 00:04:41.089 --> 00:04:43.649 them exactly how to match certain Chinese symbols 00:04:43.649 --> 00:04:46.529 with other Chinese symbols to form answers. So 00:04:46.529 --> 00:04:48.569 do the people outside the room slipping questions 00:04:48.569 --> 00:04:50.910 under the door? or it looks like the person inside 00:04:50.910 --> 00:04:54.009 is perfectly fluent in Chinese. Exactly. They 00:04:54.009 --> 00:04:56.589 are getting totally coherent answers back. But 00:04:56.589 --> 00:04:58.689 the computer or the person in the room is just 00:04:58.689 --> 00:05:01.569 emulating understanding by blindly applying rules 00:05:01.569 --> 00:05:04.149 to the data it confronts. There's no actual comprehension 00:05:04.149 --> 00:05:06.290 happening. Not at all. The machine is just moving 00:05:06.290 --> 00:05:08.370 shapes around a board. But they did have some 00:05:08.370 --> 00:05:11.389 wild successes in the 1960s, even with severely 00:05:11.389 --> 00:05:13.879 limited computer memory. I mean, there's a program 00:05:13.879 --> 00:05:17.160 called SHRDLU, which could understand commands 00:05:17.160 --> 00:05:19.379 about manipulating toy blocks in a restricted 00:05:19.379 --> 00:05:22.240 vocabulary. Yeah, SHRDLU is a big deal. And then 00:05:22.240 --> 00:05:24.879 there was ELIZA, created by Joseph Weissenbaum 00:05:24.879 --> 00:05:29.759 between 1964 and 1966. And that simulated a psychotherapist. 00:05:30.100 --> 00:05:32.579 ELIZA is such a fascinating milestone because 00:05:32.579 --> 00:05:34.480 it highlights the human side of the equation. 00:05:34.660 --> 00:05:37.639 OK, let's unpack this. If ELIZA was just following 00:05:37.639 --> 00:05:40.120 these rigid mathematical rules without any real 00:05:40.120 --> 00:05:42.949 comprehension, Why did people feel so deeply 00:05:42.949 --> 00:05:45.470 connected to it? Like, users reportedly shared 00:05:45.470 --> 00:05:47.610 incredibly intimate personal details with this 00:05:47.610 --> 00:05:50.269 program, treating it like a real living therapist. 00:05:50.370 --> 00:05:52.370 Well, because Eliza was a brilliantly designed 00:05:52.370 --> 00:05:54.990 illusion. It used almost no information about 00:05:54.990 --> 00:05:58.149 actual human thought or emotion. Weizenbaum designed 00:05:58.149 --> 00:06:00.389 it so that when the patient, you know, the user, 00:06:00.850 --> 00:06:02.970 exceeded the program's very small knowledge base, 00:06:03.310 --> 00:06:05.329 the system would simply fall back on generic 00:06:05.329 --> 00:06:07.810 deflections. Oh, I see. Yeah. If you typed, my 00:06:07.810 --> 00:06:09.910 head hurts, Eliza would just turn it around and 00:06:09.910 --> 00:06:12.040 say, why do you say your head hurts? Oh, wow. 00:06:12.459 --> 00:06:14.879 It mimicked the active listening of a therapist. 00:06:15.360 --> 00:06:17.759 Precisely. It didn't need to understand pain. 00:06:17.920 --> 00:06:19.879 It just needed to reflect the statement back 00:06:19.879 --> 00:06:23.560 as a question. This made humans project empathy 00:06:23.560 --> 00:06:25.939 onto the machine. That's incredible. But underneath 00:06:25.939 --> 00:06:29.060 it all, rule -based systems hit a massive wall. 00:06:29.480 --> 00:06:31.970 Because language is just too weird. You can hand 00:06:31.970 --> 00:06:34.009 code rules for the most common sentence structures, 00:06:34.230 --> 00:06:36.910 sure, but human language has an infinite number 00:06:36.910 --> 00:06:39.870 of rare, weird exceptions. Absolutely. You simply 00:06:39.870 --> 00:06:43.290 cannot manually write an algebraic rule for every 00:06:43.290 --> 00:06:46.189 single edge case. The complexity becomes totally 00:06:46.189 --> 00:06:48.509 intractable. Which is why the scientific community 00:06:48.509 --> 00:06:51.089 had to completely pivot. Because human language 00:06:51.089 --> 00:06:53.110 had too many exceptions for handwritten rules, 00:06:53.730 --> 00:06:56.689 scientists in the late 1980s and 1990s realized 00:06:56.689 --> 00:06:58.589 they needed a completely different approach. 00:06:59.089 --> 00:07:01.069 They stopped trying to teach computers grammar, 00:07:01.189 --> 00:07:03.850 and they started teaching them probability. Here's 00:07:03.850 --> 00:07:06.269 where it gets really interesting. This is the 00:07:06.269 --> 00:07:08.870 statistical revolution, and it officially ended 00:07:08.870 --> 00:07:11.670 that long AI winter where funding had totally 00:07:11.670 --> 00:07:14.420 dried up. Yeah, finally. Instead of linguists 00:07:14.420 --> 00:07:17.879 writing out complex syntax trees by hand, computer 00:07:17.879 --> 00:07:20.120 scientists started using machine learning algorithms. 00:07:20.800 --> 00:07:25.220 And two crucial things fueled this shift. First, 00:07:25.819 --> 00:07:28.139 Moore's Law meant computers were finally getting 00:07:28.139 --> 00:07:30.579 the processing power and memory they needed to 00:07:30.579 --> 00:07:33.759 handle massive amounts of text. Right. But second, 00:07:33.980 --> 00:07:36.899 there is a huge philosophical shift away from 00:07:36.899 --> 00:07:39.079 the dominant linguistic theories of the time, 00:07:39.660 --> 00:07:42.180 particularly those championed by Noam Chomsky. 00:07:42.379 --> 00:07:45.500 Ah, Chomsky. He argued for something called the 00:07:45.500 --> 00:07:48.779 poverty of the stimulus. Exactly. The basic idea 00:07:48.779 --> 00:07:50.879 was that human children don't get enough exposure 00:07:50.879 --> 00:07:53.600 to language data to learn all its complex rules 00:07:53.600 --> 00:07:56.519 from scratch, so the foundation of language must 00:07:56.519 --> 00:07:59.300 be hardwired into our brains. Yeah, that was 00:07:59.300 --> 00:08:01.339 the prevailing thought. And because of that theoretical 00:08:01.339 --> 00:08:04.079 underpinning, scientists were actually discouraged 00:08:04.079 --> 00:08:06.680 from using real -world corpus data, which is 00:08:06.680 --> 00:08:09.139 just massive collections of text, because the 00:08:09.139 --> 00:08:11.100 theory essentially said general learning algorithms 00:08:11.100 --> 00:08:13.199 wouldn't work on language. But the computer scientists 00:08:13.160 --> 00:08:15.300 decided to try the data anyway. And it worked 00:08:15.300 --> 00:08:18.560 beautifully. It really did. IBM research had 00:08:18.560 --> 00:08:20.819 a massive breakthrough in machine translation 00:08:20.819 --> 00:08:23.759 by feeding their algorithms existing multilingual 00:08:23.759 --> 00:08:26.300 texts. And the texts they used are so perfectly 00:08:26.300 --> 00:08:28.720 mundane. It was the translated proceedings of 00:08:28.720 --> 00:08:31.000 the Parliament of Canada and the European Union. 00:08:31.319 --> 00:08:33.820 Which is so clever when you think about it. Because 00:08:33.820 --> 00:08:36.019 the law required those governmental proceedings 00:08:36.019 --> 00:08:38.279 to be translated into all official languages, 00:08:38.659 --> 00:08:41.879 it created a perfect, naturally occurring data 00:08:41.879 --> 00:08:44.750 set. Yes. The algorithm could look at a sentence 00:08:44.750 --> 00:08:47.309 in English, look at the exact same sentence translated 00:08:47.309 --> 00:08:50.009 into French, and statistically calculate the 00:08:50.009 --> 00:08:52.490 probability that a specific word translates to 00:08:52.490 --> 00:08:55.529 another specific word. Then the 2000s hit, the 00:08:55.529 --> 00:08:57.850 World Wide Web exploded, and suddenly we had 00:08:57.850 --> 00:09:01.220 endless raw unannotated data. Oh, it was a gold 00:09:01.220 --> 00:09:03.779 mine. We moved right into unsupervised and semi 00:09:03.779 --> 00:09:05.759 -supervised learning. The machine could just 00:09:05.759 --> 00:09:08.320 absorb millions of web pages and learn the statistical 00:09:08.320 --> 00:09:10.500 likelihood of which words follow other words. 00:09:10.700 --> 00:09:13.179 To put this in perspective for you, symbolic 00:09:13.179 --> 00:09:16.480 NLP, that 1950s approach we talked about, is 00:09:16.480 --> 00:09:19.659 like trying to navigate a new city using only 00:09:19.659 --> 00:09:23.120 a rigid paper map and a dictionary. You are constantly 00:09:23.120 --> 00:09:25.419 stopping to look up rules, and if a street is 00:09:25.419 --> 00:09:28.000 closed, You were completely lost. You're stuck. 00:09:28.220 --> 00:09:31.440 Right. But statistical NLP is like learning a 00:09:31.440 --> 00:09:34.080 language by moving to the country, binge -watching 00:09:34.080 --> 00:09:36.679 foreign TV shows, and absorbing the patterns 00:09:36.679 --> 00:09:39.179 until it just sounds right. What's fascinating 00:09:39.179 --> 00:09:41.340 here is that these statistical language models 00:09:41.340 --> 00:09:43.879 turned out to be far more robust than the old 00:09:43.879 --> 00:09:46.440 rule -based systems. I mean, a rule -based system 00:09:46.440 --> 00:09:48.419 breaks if you misspell a single word because 00:09:48.419 --> 00:09:50.700 the rigid rule just no longer applies. It crashes. 00:09:50.899 --> 00:09:54.580 Exactly. But a probabilistic model, it has seen 00:09:54.580 --> 00:09:57.230 that exact spelling a thousand times in its web 00:09:57.230 --> 00:10:00.289 data. It calculates that there is a 99 % chance 00:10:00.289 --> 00:10:02.289 you meant to type hello instead of hello. It 00:10:02.289 --> 00:10:04.210 knows what you probably meant. Yeah. With the 00:10:04.210 --> 00:10:06.529 statistical approach, the larger your probabilistic 00:10:06.529 --> 00:10:09.230 model becomes, the more accurate it gets. Counting 00:10:09.230 --> 00:10:11.610 words and calculating probabilities was a huge 00:10:11.610 --> 00:10:14.289 leap, but those statistical methods still required 00:10:14.289 --> 00:10:17.210 a lot of human handholding. engineers had to 00:10:17.210 --> 00:10:19.190 do elaborate feature engineering which basically 00:10:19.190 --> 00:10:21.909 means they had to manually tell the system what 00:10:21.909 --> 00:10:24.070 linguistic features to look for right like pay 00:10:24.070 --> 00:10:26.350 attention to verbs here pay attention to adjectives 00:10:26.350 --> 00:10:30.230 there exactly but the next evolution starting 00:10:30.230 --> 00:10:34.529 in the 2010s removed that human handholding entirely. 00:10:34.789 --> 00:10:37.070 Yeah, the transition really started gaining ground 00:10:37.070 --> 00:10:39.350 when neural networks began outperforming the 00:10:39.350 --> 00:10:43.429 best statistical algorithms. In 2003, a multilayer 00:10:43.429 --> 00:10:46.570 perceptron neural network beat out older models. 00:10:47.049 --> 00:10:50.230 Then in 2010, a researcher named Tomas Mikolov 00:10:50.230 --> 00:10:52.769 applied a recurrent neural network to language 00:10:52.769 --> 00:10:54.809 modeling. And that eventually led to a breakthrough 00:10:54.809 --> 00:10:58.370 tool called Word2Vec. Yes, Word2Vec. We hear 00:10:58.370 --> 00:11:00.610 the phrase neural networks all the time, but 00:11:00.610 --> 00:11:03.700 the way they a language is just wild. We entered 00:11:03.700 --> 00:11:06.360 the era of deep learning and representation learning. 00:11:06.879 --> 00:11:08.879 The machine was no longer just counting how often 00:11:08.879 --> 00:11:11.259 two words appear next to each other, it was figuring 00:11:11.259 --> 00:11:13.379 out the hidden representations and relationships 00:11:13.379 --> 00:11:16.059 between them. Word2vec is a perfect example of 00:11:16.059 --> 00:11:18.620 this. Instead of just seeing words as isolated 00:11:18.620 --> 00:11:21.500 text, it turns words into multi -dimensional 00:11:21.500 --> 00:11:24.740 mathematical vectors. It maps them in a vast 00:11:24.740 --> 00:11:27.059 mathematical space. Okay, how does that work? 00:11:27.220 --> 00:11:29.919 So the distance between the vector for king and 00:11:29.919 --> 00:11:32.899 queen becomes mathematically similar to the distance 00:11:32.899 --> 00:11:35.720 between man and woman. The network discovers 00:11:35.720 --> 00:11:38.200 these underlying semantic relationships purely 00:11:38.200 --> 00:11:41.159 on its own without a human ever defining what 00:11:41.159 --> 00:11:44.879 gender or royalty even is. Wow. It made all those 00:11:44.879 --> 00:11:47.500 intermediate manual steps, like manually aligning 00:11:47.500 --> 00:11:50.539 words, totally obsolete. Completely. But wait, 00:11:50.679 --> 00:11:52.759 before the neural network can map all these concepts, 00:11:53.120 --> 00:11:55.059 it still has to break the raw text down into 00:11:55.059 --> 00:11:57.240 pieces it can actually ingest. You can't just 00:11:57.240 --> 00:12:00.039 swallow a whole book at once. No, it can't. That 00:12:00.039 --> 00:12:02.769 process is called tokenization. It means dividing 00:12:02.769 --> 00:12:05.690 text into fragments tokens and giving them numerical 00:12:05.690 --> 00:12:08.909 identifiers. For a machine to process a sentence, 00:12:09.049 --> 00:12:11.330 it has to turn the words into numbers. Wait, 00:12:11.370 --> 00:12:13.649 for English, isn't tokenization ridiculously 00:12:13.649 --> 00:12:15.470 easy? You just look for the blank spaces between 00:12:15.470 --> 00:12:17.250 the words? Well, yeah, it is easy for English. 00:12:17.350 --> 00:12:19.750 But think about languages like Chinese, Japanese, 00:12:20.149 --> 00:12:23.129 or Thai. Oh, right. They do not mark word boundaries 00:12:23.129 --> 00:12:25.870 with spaces. A sentence is just a continuous 00:12:25.870 --> 00:12:28.610 string of characters. To segment text in those 00:12:28.610 --> 00:12:31.230 languages, the system needs a deep, complex knowledge 00:12:31.080 --> 00:12:34.019 of the vocabulary and the morphology of language. 00:12:34.659 --> 00:12:37.340 You can't just look for a blank space. The algorithm 00:12:37.340 --> 00:12:41.100 has to mathematically deduce where one word concept 00:12:41.100 --> 00:12:43.539 ends and another begins based on surrounding 00:12:43.539 --> 00:12:46.480 context. And it has to be even harder when we 00:12:46.480 --> 00:12:49.000 move from perfectly typed text to just talking 00:12:49.000 --> 00:12:51.740 to our phones. When I speak to my voice assistant, 00:12:51.779 --> 00:12:53.960 I am definitely not leaving neat little spaces 00:12:53.960 --> 00:12:57.120 between my words. Speech recognition is incredibly 00:12:57.120 --> 00:13:00.360 difficult because of a phenomenon called co -articulation. 00:13:01.200 --> 00:13:03.360 Coarticulation. Yeah. When humies speak naturally, 00:13:03.539 --> 00:13:05.600 the sounds representing successive letters actually 00:13:05.600 --> 00:13:08.100 blend together. The end of one word slurs right 00:13:08.100 --> 00:13:10.100 into the beginning of the next. Your human brain 00:13:10.100 --> 00:13:12.320 separates them effortlessly based on context. 00:13:12.860 --> 00:13:15.279 But converting that continuous unbroken analog 00:13:15.279 --> 00:13:18.360 sound wave into discrete textual characters is 00:13:18.360 --> 00:13:20.460 a massive hurdle for a machine. That makes total 00:13:20.460 --> 00:13:23.429 sense. Add in the wide variety of human accents, 00:13:23.789 --> 00:13:25.649 regional dialects, and just random background 00:13:25.649 --> 00:13:28.850 noise. And it's a monumental computational task. 00:13:29.169 --> 00:13:31.470 But the payoff for cracking this is huge. You 00:13:31.470 --> 00:13:33.549 pointed out in our notes how these neural approaches 00:13:33.549 --> 00:13:36.289 are revolutionizing health care right now. Oh, 00:13:36.330 --> 00:13:40.070 absolutely. NLP is unlocking unstructured text 00:13:40.070 --> 00:13:43.090 in electronic health records. Doctors write notes 00:13:43.090 --> 00:13:46.549 in shorthand full of obscure jargon, messy syntax 00:13:46.549 --> 00:13:48.730 and acronyms. Right. Chicken scratch, basically. 00:13:48.789 --> 00:13:51.179 Yeah. And traditional database. can't read that. 00:13:51.659 --> 00:13:54.000 But neural NLP can read those unstructured notes, 00:13:54.360 --> 00:13:56.259 analyze them to spot early warning signs for 00:13:56.259 --> 00:13:59.200 patient care, and simultaneously strip out sensitive 00:13:59.200 --> 00:14:02.039 terms to protect patient privacy. It's turning 00:14:02.039 --> 00:14:05.120 locked away messy text into actionable medical 00:14:05.120 --> 00:14:07.759 insights. So we've got the computer perfectly 00:14:07.759 --> 00:14:10.720 transcribing our blended speech, and it has successfully 00:14:10.720 --> 00:14:13.779 tokenized the words into numbers. But identifying 00:14:13.779 --> 00:14:16.019 a word is very different from actually understanding 00:14:16.019 --> 00:14:18.580 what it means in the real world. Extremely different. 00:14:18.740 --> 00:14:21.200 This is where we hit the absolute wildest quirks 00:14:21.200 --> 00:14:24.100 of human communication. Meaning is incredibly 00:14:24.100 --> 00:14:26.559 messy. Let's start with just figuring out the 00:14:26.559 --> 00:14:29.000 base form of a word. When a machine sees words 00:14:29.000 --> 00:14:32.340 like running, ran, and runs, it needs to know 00:14:32.340 --> 00:14:34.960 they all stem from the same concept. Right. You 00:14:34.960 --> 00:14:38.659 have two approaches for this. Stemming. and lemmatization. 00:14:39.120 --> 00:14:42.000 Stemming is the older, rougher method. It operates 00:14:42.000 --> 00:14:43.879 like a butcher knife, just chopping the ends 00:14:43.879 --> 00:14:46.919 off words based on rigid rules, so closing becomes 00:14:46.919 --> 00:14:49.179 close. Sometimes it chops off so much you are 00:14:49.179 --> 00:14:51.379 left with a non -word. Which isn't ideal. Not 00:14:51.379 --> 00:14:53.919 at all. Lemmatization is much smarter. It acts 00:14:53.919 --> 00:14:56.320 more like a librarian. It actually references 00:14:56.320 --> 00:14:58.779 a comprehensive dictionary to analyze the word's 00:14:58.779 --> 00:15:01.279 morphology and return the true base word, which 00:15:01.279 --> 00:15:03.519 is called the lemma. It knows that the lemma 00:15:03.519 --> 00:15:06.720 of better is good. Oh, wow. Then you have WordSense 00:15:06.799 --> 00:15:11.759 This is the classic problem of words having multiple 00:15:11.759 --> 00:15:14.620 meanings. If I say book a flight versus read 00:15:14.620 --> 00:15:17.320 a book, the machine has to look at the surrounding 00:15:17.320 --> 00:15:20.139 context to know if book is functioning as an 00:15:20.139 --> 00:15:23.059 action verb or a physical noun. Exactly. And 00:15:23.059 --> 00:15:26.820 that leads directly into Named Entity Recognition, 00:15:27.240 --> 00:15:30.860 or NER. This is how a system figures out if a 00:15:30.860 --> 00:15:33.820 word is a proper name, a location, or an organization. 00:15:34.469 --> 00:15:36.690 You might think the monrain could just look for 00:15:36.690 --> 00:15:38.549 the capital letters. Which works great in English, 00:15:38.590 --> 00:15:41.789 but not globally. Right. In German... Every single 00:15:41.789 --> 00:15:43.870 noun is capitalized, whether it's a specific 00:15:43.870 --> 00:15:46.830 person's name or just the word for apple. Arabic 00:15:46.830 --> 00:15:49.389 doesn't use capitalization at all. And in Spanish 00:15:49.389 --> 00:15:51.490 and French, they don't capitalize names when 00:15:51.490 --> 00:15:53.629 they are serving as adjectives. Wow. I didn't 00:15:53.629 --> 00:15:55.730 even think of that. Yeah. So the machine cannot 00:15:55.730 --> 00:15:58.769 rely on punctuation. It has to infer the entity 00:15:58.769 --> 00:16:01.570 entirely from the complex mathematical context 00:16:01.570 --> 00:16:03.529 surrounding it. And then there's discourse and 00:16:03.529 --> 00:16:05.570 corpherence resolution. The sort text gives this 00:16:05.570 --> 00:16:07.649 great example. He entered John's house through 00:16:07.649 --> 00:16:09.899 the front door. The system has to know that the 00:16:09.899 --> 00:16:12.279 front door specifically acts as a bridge to John's 00:16:12.279 --> 00:16:14.139 house, not just some random door down the street. 00:16:14.580 --> 00:16:16.480 Exactly. What does this all mean? Correference 00:16:16.480 --> 00:16:19.200 resolution sounds exactly like trying to keep 00:16:19.200 --> 00:16:21.860 track of all the characters and their varying 00:16:21.860 --> 00:16:25.379 nicknames in a dense thousand page Russian novel. 00:16:25.500 --> 00:16:28.159 That is a perfect way to describe it. Humans 00:16:28.159 --> 00:16:30.559 use these bridging relationships effortlessly. 00:16:30.840 --> 00:16:32.720 We know the front door belongs to the house that 00:16:32.720 --> 00:16:35.759 was just mentioned. But for a machine, every 00:16:35.759 --> 00:16:40.120 single pronoun, every he, she, it or thee is 00:16:40.120 --> 00:16:42.919 a puzzle piece. It's just a blank. Yeah. It has 00:16:42.919 --> 00:16:45.019 to be mathematically resolved against the local 00:16:45.019 --> 00:16:47.860 text. It's an intricate web of dependencies. 00:16:48.159 --> 00:16:50.919 And once you resolve all those entities and pronouns, 00:16:51.019 --> 00:16:53.879 you can do incredibly useful things like sentiment 00:16:53.879 --> 00:16:56.539 analysis. That's where you categorize the emotional 00:16:56.539 --> 00:16:59.480 tone of text as positive, negative, or neutral. 00:16:59.820 --> 00:17:02.759 It is exactly what companies use to automatically 00:17:02.759 --> 00:17:05.400 analyze millions of customer reviews to see if 00:17:05.400 --> 00:17:07.259 people actually like your new product or hate 00:17:07.259 --> 00:17:09.559 it. Right. But understanding all that context 00:17:09.559 --> 00:17:12.359 is just half the battle. Once you crack the code 00:17:12.359 --> 00:17:14.440 of meaning and context, the machine doesn't just 00:17:14.440 --> 00:17:16.980 sit there processing language quietly. You start 00:17:16.980 --> 00:17:19.720 generating it at scale. And the evolution of 00:17:19.720 --> 00:17:22.380 text generation is just fascinating to look at. 00:17:22.640 --> 00:17:25.640 Back in 1984, a rule -based system named Ractor 00:17:25.640 --> 00:17:28.720 generated an entire book called The Policeman's 00:17:28.720 --> 00:17:31.940 Beard. is half constructed. Catchy title. Right. 00:17:32.180 --> 00:17:34.400 But it was entirely nonsensical, just stringing 00:17:34.400 --> 00:17:36.819 words together based on those rigid grammar rules 00:17:36.819 --> 00:17:39.880 we talked about earlier. Decades later in 2018, 00:17:40.259 --> 00:17:43.099 a neural network generated a novel called One 00:17:43.099 --> 00:17:46.819 the Road, which was 60 million words long. But 00:17:46.819 --> 00:17:49.220 again, it was basically semantics free. It sounded 00:17:49.220 --> 00:17:51.500 vaguely like a road trip novel, but it meant 00:17:51.500 --> 00:17:54.000 nothing. It lacked a coherent narrative arc. 00:17:54.190 --> 00:17:57.950 But by 2019, things shifted dramatically. A machine 00:17:57.950 --> 00:18:01.109 generated a science book called Lithium Ion Batteries, 00:18:01.390 --> 00:18:04.390 published by Springer. Oh, that was a huge milestone. 00:18:04.710 --> 00:18:06.990 Yeah. Unlike the previous examples, this wasn't 00:18:06.990 --> 00:18:09.210 hallucinated nonsense. It was grounded in actual 00:18:09.210 --> 00:18:12.059 factual text summarization. The machine read 00:18:12.059 --> 00:18:14.599 vast amounts of chemistry research and synthesized 00:18:14.599 --> 00:18:17.319 it into a coherent, structured book. We are moving 00:18:17.319 --> 00:18:19.940 past just parsing sentences into something that 00:18:19.940 --> 00:18:22.420 looks a lot like cognition. We are trying to 00:18:22.420 --> 00:18:25.240 emulate intelligent comprehension. The source 00:18:25.240 --> 00:18:27.920 dives into George Lakoff's theory of conceptual 00:18:27.920 --> 00:18:31.220 metaphor, which argues that to truly master language, 00:18:31.579 --> 00:18:34.119 a computer must understand intent. For instance, 00:18:34.119 --> 00:18:36.839 if I say that is a big tree, I am clearly talking 00:18:36.839 --> 00:18:40.119 about physical size. Right. But if I say tomorrow's 00:18:40.119 --> 00:18:43.339 a big day, I am talking about importance. But 00:18:43.339 --> 00:18:45.880 wait, I have to challenge this. If a computer 00:18:45.880 --> 00:18:48.339 doesn't actually have a physical body, it doesn't 00:18:48.339 --> 00:18:50.920 experience time, and it has never felt anxiety 00:18:50.920 --> 00:18:53.559 about an upcoming event, how can it ever truly 00:18:53.559 --> 00:18:56.240 understand the importance of a big day? It's 00:18:56.240 --> 00:18:58.609 just lines of code. This raises an important 00:18:58.609 --> 00:19:01.609 question, and the answer lies entirely in mathematics. 00:19:02.250 --> 00:19:04.750 The algorithm doesn't feel anxiety or importance. 00:19:05.190 --> 00:19:07.869 Instead, the source text outlines a formula for 00:19:07.869 --> 00:19:10.029 this called the Relative Measure of Meaning, 00:19:10.490 --> 00:19:14.069 or RMM. It calculates the intent mathematically 00:19:14.069 --> 00:19:16.230 by looking at the mathematical neighborhood of 00:19:16.230 --> 00:19:18.769 the words. Okay, how does that mathematical neighborhood 00:19:18.769 --> 00:19:21.339 actually work? It takes the probable measure 00:19:21.339 --> 00:19:23.700 of meaning based on the massive data sets it 00:19:23.700 --> 00:19:26.720 was trained on and multiplies it by a probability 00:19:26.720 --> 00:19:29.420 function that scans the specific tokens located 00:19:29.420 --> 00:19:31.559 immediately before and after the phrase being 00:19:31.559 --> 00:19:35.359 analyzed. So if big is next today, the algorithm 00:19:35.359 --> 00:19:37.839 scans the wider sentence and might see words 00:19:37.839 --> 00:19:42.440 like wedding, graduation, or nervous. It calculates 00:19:42.440 --> 00:19:44.619 that in this specific mathematical neighborhood, 00:19:44.839 --> 00:19:47.640 the word big lives much closer to the concept 00:19:47.640 --> 00:19:50.539 of important event, rather than physically giant 00:19:50.539 --> 00:19:54.799 24 hours. By assigning these relative measures 00:19:54.799 --> 00:19:57.900 of meaning, the algorithm uses complex probability 00:19:57.900 --> 00:20:00.559 to mathematically approximate human cognitive 00:20:00.559 --> 00:20:03.059 intuition. It's just staggering to think about. 00:20:03.099 --> 00:20:06.339 We've gone from the naive optimism of those 1950 00:20:06.339 --> 00:20:09.299 scientists handwriting rigid algebra rules and 00:20:09.299 --> 00:20:12.140 getting crushed by the LPAC report to feeding 00:20:12.140 --> 00:20:14.799 mundane Canadian Parliament records into early 00:20:14.799 --> 00:20:17.240 statistical models. Quite the leap. And all the 00:20:17.240 --> 00:20:19.400 way to massive deep neural networks that turn 00:20:19.400 --> 00:20:22.539 words into multidimensional vectors. just to 00:20:22.539 --> 00:20:24.299 calculate the mathematical weight of a human 00:20:24.299 --> 00:20:26.680 metaphor. It's a testament to the sheer power 00:20:26.680 --> 00:20:29.859 of massive data, computational scaling, and the 00:20:29.859 --> 00:20:31.920 willingness of the scientific community to abandon 00:20:31.920 --> 00:20:33.900 their old assumptions when they hit a wall. I 00:20:33.900 --> 00:20:35.480 really want to thank you for joining us on this 00:20:35.480 --> 00:20:38.039 deep dive into the source material. We interact 00:20:38.039 --> 00:20:40.660 with this invisible machinery constantly. And 00:20:40.660 --> 00:20:42.779 I hope the next time your smartphone automatically 00:20:42.779 --> 00:20:45.500 filters out a spam email or correctly guesses 00:20:45.500 --> 00:20:48.160 the end of your text message, you have a newfound 00:20:48.160 --> 00:20:51.359 appreciation for the decades of complex mathematics. 00:20:51.210 --> 00:20:54.390 making it happen. It is incredible. But as we 00:20:54.390 --> 00:20:56.170 close, I want to leave you with one final thing 00:20:56.170 --> 00:20:59.130 to ponder. Ooh, lay it on us. We've seen how 00:20:59.130 --> 00:21:02.049 computers have completely abandoned human -made 00:21:02.049 --> 00:21:04.769 chromatical rules. They aren't learning syntax 00:21:04.769 --> 00:21:07.309 trees. They are now mastering language purely 00:21:07.309 --> 00:21:09.869 through statistical patterns hidden in massive, 00:21:10.150 --> 00:21:12.970 uninterpretable neural networks. So when that 00:21:12.970 --> 00:21:15.410 machine accurately summarizes a medical record 00:21:15.410 --> 00:21:17.630 or writes a coherent science book, does it actually 00:21:17.630 --> 00:21:20.390 understand us? Or have we simply built a perfect 00:21:20.430 --> 00:21:23.029 mathematical mirror, one that brilliantly reflects 00:21:23.029 --> 00:21:25.109 our own thoughts back at us without a single 00:21:25.109 --> 00:21:26.630 drop of real comprehension.