WEBVTT 00:00:00.000 --> 00:00:02.060 What if I told you the math equation deciding 00:00:02.060 --> 00:00:06.820 whether you get a mortgage or a job or even medical 00:00:06.820 --> 00:00:09.560 care might be actively prejudiced against you? 00:00:09.699 --> 00:00:11.619 Yeah. I mean, it's a really comforting expectation, 00:00:11.740 --> 00:00:14.859 right? We treat computers like perfect objective 00:00:14.859 --> 00:00:17.120 calculators. Exactly. You punch in a complex 00:00:17.120 --> 00:00:20.199 equation, hit equals, and the machine just gives 00:00:20.199 --> 00:00:23.879 you this pristine mathematical answer. We desperately 00:00:23.879 --> 00:00:26.550 want systems that are binary. Yeah. Systems stripped 00:00:26.550 --> 00:00:30.250 of any messy human feelings or, you know, historical 00:00:30.250 --> 00:00:32.570 baggage. But then you step into the actual world 00:00:32.570 --> 00:00:34.810 of modern algorithms and suddenly that perfect 00:00:34.810 --> 00:00:36.890 calculator is just, well, it's fundamentally 00:00:36.890 --> 00:00:39.570 broken. It really is. It's a murky digital landscape. 00:00:39.770 --> 00:00:42.229 Right. And it's inheriting our very human flaws. 00:00:43.770 --> 00:00:47.070 So welcome to today's Deep Dive. We are thrilled 00:00:47.070 --> 00:00:49.530 you could join us because we are exploring a 00:00:49.530 --> 00:00:51.929 comprehensive Wikipedia article on the mechanics 00:00:51.929 --> 00:00:55.009 of algorithmic bias. It is such a crucial topic 00:00:55.009 --> 00:00:57.090 to dig into. It really is. And if you're sitting 00:00:57.090 --> 00:00:58.689 there right now wondering why you should care 00:00:58.689 --> 00:01:01.090 about this, well, algorithms are the invisible 00:01:01.090 --> 00:01:03.149 architects of your modern life. They really are 00:01:03.149 --> 00:01:05.689 everywhere. Yeah, they curate the news you read. 00:01:05.890 --> 00:01:08.109 They screen your resume before a human ever even 00:01:08.109 --> 00:01:10.469 sees it. They decide your credit limit. We tend 00:01:10.469 --> 00:01:12.769 to think of them as flawless math. But today, 00:01:13.030 --> 00:01:15.290 we are short -cutting through the academic jargon 00:01:15.290 --> 00:01:17.890 to understand how they actually work. OK, let's 00:01:17.890 --> 00:01:20.450 unpack this. So to really understand algorithmic 00:01:20.450 --> 00:01:23.670 bias, we have to start by entirely dismantling 00:01:23.670 --> 00:01:26.590 the idea that computers are perfectly neutral. 00:01:26.870 --> 00:01:29.510 Right. Getting rid of that myth. Exactly. And 00:01:29.510 --> 00:01:32.049 this isn't a new revelation born from modern 00:01:32.049 --> 00:01:33.790 artificial intelligence. We actually have to 00:01:33.790 --> 00:01:37.450 go back to 1976. Oh, wow. The era of room -sized 00:01:37.450 --> 00:01:39.989 mainframe computers and punch cards. The very 00:01:39.989 --> 00:01:42.290 same. Yeah. So artificial intelligence pioneer 00:01:42.290 --> 00:01:44.750 Joseph Weissenbaum wrote this foundational book 00:01:44.750 --> 00:01:47.549 called Computer Power and Human Reason. OK. And 00:01:47.549 --> 00:01:49.670 he explained that early computer programs were 00:01:49.670 --> 00:01:51.790 essentially designed to mimic human reasoning. 00:01:52.510 --> 00:01:55.890 Weissenbaum warned that code at its core embodies 00:01:55.890 --> 00:01:59.989 law. Embodies law, meaning it enforces a specific 00:01:59.989 --> 00:02:02.870 worldview. Precisely. A program is just a sequence 00:02:02.870 --> 00:02:05.510 of rules created by a human being. So it naturally 00:02:05.510 --> 00:02:08.250 enforces a programmer -specific way of solving 00:02:08.250 --> 00:02:10.370 a problem. Right, which includes their imagination 00:02:10.370 --> 00:02:13.530 of how the world works, their expectations, and 00:02:13.530 --> 00:02:16.150 I guess their blind spots. Yeah, their blind 00:02:16.150 --> 00:02:18.229 spots are baked right in. It's kind of like blindly 00:02:18.229 --> 00:02:21.689 following a GPS. Actually, the source mentions 00:02:21.689 --> 00:02:24.689 Weizenbaum compared trusting a computer you don't 00:02:24.689 --> 00:02:27.110 understand to a tourist flipping a coin to find 00:02:27.110 --> 00:02:29.759 their hotel. That is a great way to put it. Right. 00:02:30.000 --> 00:02:32.340 Like, you might arrive at the destination, but 00:02:32.340 --> 00:02:34.539 it doesn't mean your method was actually smart. 00:02:35.180 --> 00:02:37.759 Yet we trust the computer's turn left command 00:02:37.759 --> 00:02:39.900 just because a glowing screen tells us to do 00:02:39.900 --> 00:02:42.319 it. What's fascinating here is the psychology 00:02:42.319 --> 00:02:44.639 behind why we do that. Sociologists actually 00:02:44.639 --> 00:02:48.039 call this automation bias. Automation bias. Yeah, 00:02:48.479 --> 00:02:51.879 or author Clay Shirky describes it as algorithmic 00:02:51.879 --> 00:02:54.639 authority. It's our dangerous human tendency 00:02:54.639 --> 00:02:57.819 to treat unmanaged algorithms as absolute truth. 00:02:58.159 --> 00:03:00.400 So we just assign them more authority than human 00:03:00.400 --> 00:03:02.759 experts simply because the output looks formal 00:03:02.759 --> 00:03:05.860 and mathematical. Exactly. We basically delegate 00:03:05.860 --> 00:03:08.259 our critical thinking to the machine. And we 00:03:08.259 --> 00:03:10.659 started doing that decades ago with some pretty 00:03:10.659 --> 00:03:14.159 chilling results. Take the computer guidance 00:03:14.159 --> 00:03:16.500 assessment system used for admissions at St. 00:03:16.620 --> 00:03:18.219 George's Hospital Medical School. This was in 00:03:18.219 --> 00:03:20.860 the UK from 1982 to 1986. I think it's a really 00:03:20.860 --> 00:03:23.479 early example. Yeah, and this algorithm actively 00:03:23.479 --> 00:03:26.939 denied entry to women and people with foreign 00:03:26.939 --> 00:03:29.639 -sounding names. And it wasn't some glitch in 00:03:29.639 --> 00:03:32.319 the matrix either. The algorithm was built to 00:03:32.319 --> 00:03:35.180 replicate past human decisions. Right, because 00:03:35.180 --> 00:03:38.060 the programmers literally fed the machine historical 00:03:38.060 --> 00:03:40.699 admissions data. The machine looked at the data 00:03:40.699 --> 00:03:43.800 and deduced a pattern. A highly biased pattern. 00:03:44.020 --> 00:03:46.360 Exactly. It saw that the Human Admissions Board 00:03:46.360 --> 00:03:49.419 historically rejected those specific demographics. 00:03:49.960 --> 00:03:52.919 Therefore, the machine assumed being male and 00:03:52.919 --> 00:03:56.219 having a traditional British name were like required 00:03:56.219 --> 00:03:58.759 variables for a successful candidate. So it simply 00:03:58.759 --> 00:04:01.300 automated and accelerated the human prejudices 00:04:01.300 --> 00:04:02.979 that already existed in the department. Yep. 00:04:03.020 --> 00:04:05.180 It just scaled up the bias. Now, we also see 00:04:05.180 --> 00:04:07.099 this with legal frameworks getting trapped in 00:04:07.099 --> 00:04:10.400 code like the 1981 British Nationality Act program. 00:04:10.460 --> 00:04:12.620 Oh yeah, that's a classic example of pre -existing 00:04:12.620 --> 00:04:15.680 bias. Right, the program essentially inscribed 00:04:15.680 --> 00:04:18.959 a highly sexist legal definition of parenthood 00:04:18.959 --> 00:04:22.589 into a permanent algorithm. But wait, hold on. 00:04:22.990 --> 00:04:25.290 I can understand how a rudimentary system from 00:04:25.290 --> 00:04:27.970 the 1980s might accidentally copy past human 00:04:27.970 --> 00:04:30.870 mistakes or bad laws, but what if a programmer 00:04:30.870 --> 00:04:33.470 goes in today with the absolute best intentions? 00:04:33.569 --> 00:04:34.589 Okay, I see what you're going to do. Like, what 00:04:34.589 --> 00:04:36.769 if they are actively trying to help marginalized 00:04:36.769 --> 00:04:38.990 groups? You're telling me the math can still 00:04:38.990 --> 00:04:41.850 get it wrong? It absolutely can. And it frequently 00:04:41.850 --> 00:04:44.170 happens through a mechanism called label choice 00:04:44.170 --> 00:04:47.029 bias. Label choice bias. How does that work? 00:04:47.310 --> 00:04:50.870 So this occurs when programmers use proxy measures 00:04:50.870 --> 00:04:53.310 to train algorithms. Basically because the actual 00:04:53.310 --> 00:04:55.230 thing they want to measure is impossible to quantify. 00:04:55.569 --> 00:04:57.410 Okay, give me an example. Well, there was a widely 00:04:57.410 --> 00:04:59.689 used healthcare algorithm designed to predict 00:04:59.689 --> 00:05:02.050 which patients had the most complex healthcare 00:05:02.050 --> 00:05:04.870 needs. The idea was so the hospital could allocate 00:05:04.870 --> 00:05:07.329 extra resources and proactive care to help them. 00:05:07.490 --> 00:05:10.029 Which sounds like a genuinely noble goal. On 00:05:10.029 --> 00:05:13.250 paper, yes, totally. But you cannot mathematically 00:05:13.250 --> 00:05:16.860 quantify the abstract concept of need. So the 00:05:16.860 --> 00:05:19.639 algorithm used healthcare costs as a proxy for 00:05:19.639 --> 00:05:22.579 need. Oh, I see. The underlying logic was straightforward. 00:05:22.920 --> 00:05:25.100 Right. Right. The more money spent on your healthcare 00:05:25.100 --> 00:05:27.939 in the past, the sicker you must be. But that 00:05:27.939 --> 00:05:30.360 completely ignores the entire reality of how 00:05:30.360 --> 00:05:33.139 healthcare access actually works. Precisely the 00:05:33.139 --> 00:05:36.740 fatal flaw. Due to systemic societal inequities, 00:05:37.139 --> 00:05:40.220 lack of access, and, you know, distrust in the 00:05:40.220 --> 00:05:43.199 medical system. Black patients historically incurred 00:05:43.199 --> 00:05:45.920 far fewer health care costs than white patients. 00:05:46.180 --> 00:05:48.079 Even when they were suffering from the exact 00:05:48.079 --> 00:05:51.360 same severe chronic illnesses. Exactly. So because 00:05:51.360 --> 00:05:54.319 they spent less money in the past, the algorithm's 00:05:54.319 --> 00:05:57.180 proxy logic falsely categorized them as healthier. 00:05:57.399 --> 00:05:59.620 And inadvertently denied them the proactive care. 00:06:00.439 --> 00:06:02.519 The machine functioned perfectly according to 00:06:02.519 --> 00:06:05.019 its instructions, but the proxy was fundamentally 00:06:05.019 --> 00:06:07.790 broken. Wow. Yeah. When researchers eventually 00:06:07.790 --> 00:06:09.930 audited the system and adjusted the target variable 00:06:09.930 --> 00:06:12.310 from cost to actual health variables like active 00:06:12.310 --> 00:06:14.470 chronic conditions, almost double the number 00:06:14.470 --> 00:06:17.310 of blank. Black patients were suddenly flagged 00:06:17.310 --> 00:06:19.889 for the extra help program. That is wild. Just 00:06:19.889 --> 00:06:23.209 a complete failure of the proxy. And sometimes 00:06:23.209 --> 00:06:26.009 bias sneaks in, not through bad proxies, but 00:06:26.009 --> 00:06:28.449 through literal technical constraints in the 00:06:28.449 --> 00:06:30.990 software architecture itself. Oh, for sure. Technical 00:06:30.990 --> 00:06:33.889 limitations are huge. Look at Turnitin, the widely 00:06:33.889 --> 00:06:36.899 used plagiarism detection software. It looks 00:06:36.899 --> 00:06:39.980 for copied work by comparing long, unbroken strings 00:06:39.980 --> 00:06:43.060 of text to online sources. But because of how 00:06:43.060 --> 00:06:45.959 the code technically matches those specific long 00:06:45.959 --> 00:06:49.060 strings, it is far more likely to flag non -native 00:06:49.060 --> 00:06:51.160 English speakers. Right, because of vocabulary 00:06:51.160 --> 00:06:53.879 differences. Exactly. Native speakers naturally 00:06:53.879 --> 00:06:56.560 possess the vocabulary to easily swap in synonyms, 00:06:56.860 --> 00:06:59.060 alter sentence structures, and break up the text 00:06:59.060 --> 00:07:01.259 to evade the string matching detection. So the 00:07:01.259 --> 00:07:03.360 technical constraint essentially punishes non 00:07:03.360 --> 00:07:05.480 -native speakers for their limited vocabulary. 00:07:05.449 --> 00:07:08.410 while letting native speakers cheat successfully. 00:07:08.490 --> 00:07:10.930 It's incredibly unfair. This raises an important 00:07:10.930 --> 00:07:13.250 question though. What happens when an algorithm 00:07:13.250 --> 00:07:16.120 which is just a fixed set of rigid instructions 00:07:16.120 --> 00:07:19.980 encounters shifting cultural norms or unexpected 00:07:19.980 --> 00:07:22.579 context. Right, when the world changes but the 00:07:22.579 --> 00:07:24.860 code doesn't. Exactly. We call this emergent 00:07:24.860 --> 00:07:27.779 bias. This is the wild west of machine learning, 00:07:28.199 --> 00:07:30.319 where artificial intelligence develops biases 00:07:30.319 --> 00:07:33.579 entirely on its own when it interacts with the 00:07:33.579 --> 00:07:36.740 messy, unpredictable real world. Because the 00:07:36.740 --> 00:07:39.040 machine is trying to apply static rules to a 00:07:39.040 --> 00:07:42.019 society that never stops changing. Yeah. Take 00:07:42.019 --> 00:07:45.439 the 1990 National Residency Match Program, or 00:07:45.439 --> 00:07:48.819 the NRMP. This is the centralized software used 00:07:48.819 --> 00:07:51.180 to place graduating US medical students into 00:07:51.180 --> 00:07:53.800 hospitals. When the algorithm was originally 00:07:53.800 --> 00:07:57.279 built, very few married couples applied for residencies 00:07:57.279 --> 00:07:59.139 together. Right, the demographics were different 00:07:59.139 --> 00:08:01.339 back then. So the system was designed to optimize 00:08:01.339 --> 00:08:04.240 for single individuals. But over time, as more 00:08:04.240 --> 00:08:06.379 women entered medical school, the demographics 00:08:06.379 --> 00:08:08.899 shifted, and more students began requesting to 00:08:08.899 --> 00:08:11.259 be placed alongside their partners in the same 00:08:11.259 --> 00:08:13.860 geographic region. And the rigid code just couldn't 00:08:13.860 --> 00:08:17.199 handle the new social dynamic. The architecture 00:08:17.199 --> 00:08:19.779 simply lacked the parameters to solve for two 00:08:19.779 --> 00:08:23.129 linked individuals simultaneously. If a married 00:08:23.129 --> 00:08:25.829 couple applied, the algorithm just weighed the 00:08:25.829 --> 00:08:28.329 location choices of a higher -rated partner first, 00:08:28.910 --> 00:08:30.949 placed them, and then looked at whatever scraps 00:08:30.949 --> 00:08:33.129 were left over in that city for the second partner. 00:08:33.230 --> 00:08:36.149 Oh, that's awful. It systematically penalized 00:08:36.149 --> 00:08:38.669 the lower -rated partner by sending them to highly 00:08:38.669 --> 00:08:41.029 -unpreferred placements rather than searching 00:08:41.029 --> 00:08:43.809 for a fair, optimized compromise for both of 00:08:43.809 --> 00:08:46.269 them as a unit. It just could not adapt to the 00:08:46.269 --> 00:08:48.980 new context. And sometimes the context the machine 00:08:48.980 --> 00:08:52.399 misses is incredibly dangerous. There was a medical 00:08:52.399 --> 00:08:55.159 triage bot designed to sort patients by risk, 00:08:55.500 --> 00:08:57.559 specifically looking at pneumonia. Oh, I know 00:08:57.559 --> 00:08:59.740 this one. It's terrifying. It really is. The 00:08:59.740 --> 00:09:01.799 algorithm reviewed millions of historical health 00:09:01.799 --> 00:09:03.720 records and concluded that it should give lower 00:09:03.720 --> 00:09:06.000 triage priority to asthmatics who came in with 00:09:06.000 --> 00:09:09.000 pneumonia. Which is medically terrifying. Asthmatics 00:09:09.000 --> 00:09:11.500 who contract pneumonia are at massive risk of 00:09:11.500 --> 00:09:13.940 severe complications. Right. But the machine 00:09:13.940 --> 00:09:16.059 just looked at the historical spreadsheet and 00:09:16.059 --> 00:09:18.019 the data showed that asthmatics with pneumonia 00:09:18.019 --> 00:09:21.200 had incredibly high survival rates. Because human 00:09:21.200 --> 00:09:24.460 doctors know they're high risk. Historically, 00:09:24.899 --> 00:09:27.200 the moment an asthmatic with pneumonia walked 00:09:27.200 --> 00:09:29.539 through the doors, human doctors rushed them 00:09:29.539 --> 00:09:32.700 to the absolute best, most immediate intensive 00:09:32.700 --> 00:09:35.600 care. Exactly. It completely missed the causality. 00:09:35.799 --> 00:09:38.000 That's like a doctor taking away a kid's bicycle 00:09:38.000 --> 00:09:40.320 helmet, because the data shows kids wearing helmets 00:09:40.320 --> 00:09:43.200 survive crashes better. That is the perfect analogy. 00:09:43.480 --> 00:09:45.620 The algorithm completely missed that the helmet 00:09:45.620 --> 00:09:48.039 caused the survival, just like the asthmatic 00:09:48.039 --> 00:09:50.179 survived because they were rushed to VIP care. 00:09:50.360 --> 00:09:53.620 The algorithm just saw the data point, high survival 00:09:53.620 --> 00:09:56.379 rate, completely stripped of its context, and 00:09:56.379 --> 00:09:58.440 bumped the most vulnerable patients to the back 00:09:58.440 --> 00:10:01.139 of the line. The machine finds correlations without 00:10:01.139 --> 00:10:03.779 understanding the mechanisms behind them. And 00:10:03.779 --> 00:10:06.740 this gets exponentially more complex when emergent 00:10:06.740 --> 00:10:09.559 bias creates feedback loops. They literally become 00:10:09.559 --> 00:10:12.059 self -fulfilling prophecies. Like the predictive 00:10:12.059 --> 00:10:14.320 policing software used in Oakland, California. 00:10:14.480 --> 00:10:17.240 Yes, a system called PredPol. In simulations, 00:10:17.399 --> 00:10:19.460 the software directed a disproportionate amount 00:10:19.460 --> 00:10:22.580 of police patrols to historically black neighborhoods. 00:10:22.820 --> 00:10:25.460 And why was that? Because the machine was generating 00:10:25.460 --> 00:10:28.399 its predictions based on public crime reports 00:10:28.399 --> 00:10:31.860 rather than actual crime rates. The simulation 00:10:31.860 --> 00:10:35.120 revealed a glaring human variable. The public 00:10:35.120 --> 00:10:37.639 often reported suspicious activity simply based 00:10:37.639 --> 00:10:40.480 on the sight of police cars, regardless of what 00:10:40.480 --> 00:10:43.559 the police were actually doing. So, wait. The 00:10:43.559 --> 00:10:46.279 algorithm sends a police car to a specific neighborhood. 00:10:46.980 --> 00:10:49.379 Residents see the police car and report a crime. 00:10:49.539 --> 00:10:52.159 Yep. And then the algorithm absorbs that new 00:10:52.159 --> 00:10:54.679 report as fresh data, decides the neighborhood 00:10:54.679 --> 00:10:57.919 is indeed a high crime hotspot, and sends more 00:10:57.919 --> 00:11:00.240 police cars. Leading to more sightings, more 00:11:00.240 --> 00:11:03.059 reports, and an endless accelerating feedback 00:11:03.059 --> 00:11:05.919 loop. The software ends up predicting its own 00:11:05.919 --> 00:11:08.580 policing patterns entirely divorced from the 00:11:08.580 --> 00:11:10.620 reality of actual crime. Here's where it gets 00:11:10.620 --> 00:11:12.720 really interesting because these aren't just 00:11:12.720 --> 00:11:15.460 isolated glitches in a university lab. This is 00:11:15.460 --> 00:11:17.759 impacting your daily reality, your career, your 00:11:17.759 --> 00:11:20.080 privacy. The commercial impacts of these blind 00:11:20.080 --> 00:11:23.080 spots are staggering and usually completely invisible 00:11:23.080 --> 00:11:25.240 to the user. Absolutely invisible. Like this 00:11:25.240 --> 00:11:27.519 goes way back. American Airlines was actually 00:11:27.519 --> 00:11:29.740 found to be manipulating slight results in their 00:11:29.740 --> 00:11:32.000 booking systems back in the 80s to favor their 00:11:32.000 --> 00:11:36.019 own flights. And more recently, take Amazon's 00:11:36.019 --> 00:11:39.200 experimental AI hiring tool. They wanted to automate 00:11:39.200 --> 00:11:41.600 RISM screening to find the best tech talent. 00:11:42.019 --> 00:11:44.960 So they trained the AI on 10 years of resumes 00:11:44.960 --> 00:11:47.419 submitted to the company. But the tech industry 00:11:47.419 --> 00:11:49.700 has historically been heavily male dominated. 00:11:50.159 --> 00:11:53.000 So the training data was overwhelmingly male. 00:11:53.179 --> 00:11:55.399 So the machine learned that male -associated 00:11:55.399 --> 00:11:58.080 traits were a proxy for success. It didn't just 00:11:58.080 --> 00:12:01.240 look for coding skills, it actively began penalizing 00:12:01.240 --> 00:12:03.340 resumes that included the word women's, like 00:12:03.340 --> 00:12:06.399 women's chess club captain. Wow. It also downgraded 00:12:06.399 --> 00:12:09.360 graduates of two all -women's colleges. The system 00:12:09.360 --> 00:12:11.600 had to be completely shut down because the historical 00:12:11.600 --> 00:12:13.940 bias was baked so deeply into the mathematical 00:12:13.940 --> 00:12:16.139 weights. And we see this on a smaller scale, 00:12:16.399 --> 00:12:18.539 too. LinkedIn used to suggest the name Andrew 00:12:18.539 --> 00:12:20.639 when users explicitly searched for the female 00:12:20.639 --> 00:12:23.480 name, Andra. Because statistically, the algorithm 00:12:23.480 --> 00:12:26.320 saw more Andrews in high -ranking professional 00:12:26.320 --> 00:12:29.379 profiles. It's wild. And the predictive power 00:12:29.379 --> 00:12:32.679 of these models also crosses into severe privacy 00:12:32.679 --> 00:12:36.919 violations. Retailers like Target have notoriously 00:12:36.919 --> 00:12:40.039 utilized purchasing data to infer incredibly 00:12:40.039 --> 00:12:42.940 intimate details about customers. Oh, the pregnancy 00:12:42.940 --> 00:12:45.940 prediction story. Yes. By tracking the study 00:12:45.940 --> 00:12:49.000 shift from scented to unscented lotions combined 00:12:49.000 --> 00:12:51.259 with the purchase of specific calcium supplements, 00:12:51.980 --> 00:12:54.259 Target's algorithm could predict when female 00:12:54.259 --> 00:12:56.659 customers were pregnant, even if they hadn't 00:12:56.659 --> 00:12:58.820 announced it to their own families. And because 00:12:58.820 --> 00:13:00.960 the algorithm mathematically predicted the pregnancy 00:13:00.960 --> 00:13:03.259 based on lotion purchases, rather than the user 00:13:03.259 --> 00:13:06.080 explicitly reporting their medical status, the 00:13:06.080 --> 00:13:08.440 retailer had no legal obligation to protect that 00:13:08.440 --> 00:13:10.960 privacy. They could freely share that inferred 00:13:10.960 --> 00:13:13.080 profile with marketing partners. Some of the 00:13:13.080 --> 00:13:15.559 most bizarre logic failures happen when companies 00:13:15.559 --> 00:13:18.019 try to automate complex language and context, 00:13:18.379 --> 00:13:21.519 like in content moderation. Take Facebook's 2017 00:13:21.519 --> 00:13:23.240 hate speech algorithm. OK, what happened there? 00:13:23.370 --> 00:13:26.289 The engineers designed the rules to protect broad 00:13:26.289 --> 00:13:29.269 categories of people based on protected classes 00:13:29.269 --> 00:13:32.990 like race and gender. But the rules explicitly 00:13:32.990 --> 00:13:35.809 did not protect subsets of those categories. 00:13:36.049 --> 00:13:37.769 How does that actually play out in the code though? 00:13:37.929 --> 00:13:41.190 Well, it creates a massive logic failure. The 00:13:41.190 --> 00:13:44.600 AI actively blocked hate speech directed at white 00:13:44.600 --> 00:13:47.440 men because white is a broad racial category 00:13:47.440 --> 00:13:50.659 and men is a broad gender category. Both triggered 00:13:50.659 --> 00:13:52.700 the protection protocols. Right. But the exact 00:13:52.700 --> 00:13:55.159 same algorithm permitted severe hate speech directed 00:13:55.159 --> 00:13:57.720 against black children. While black triggered 00:13:57.720 --> 00:14:00.139 the race protection, children was categorized 00:14:00.139 --> 00:14:03.179 merely as an age subset, not a broad protected 00:14:03.179 --> 00:14:06.340 class in their system. The Boolean logic completely 00:14:06.340 --> 00:14:09.120 overrode the reality of the situation. This blind 00:14:09.120 --> 00:14:11.259 spot in categorization doesn't just cause headaches 00:14:11.259 --> 00:14:13.539 on social media platforms either. When these 00:14:13.539 --> 00:14:15.679 same rigid definitions are handed over to state 00:14:15.679 --> 00:14:18.200 militaries and global governments, the consequences 00:14:18.200 --> 00:14:20.500 become a matter of life and death. They absolutely 00:14:20.500 --> 00:14:23.070 do. For instance, strictly looking at the reports 00:14:23.070 --> 00:14:26.110 on this, large language models, the tech behind 00:14:26.110 --> 00:14:28.970 conversational tools, exhibit severe dialect 00:14:28.970 --> 00:14:32.169 prejudice. Studies show these models harbor covert 00:14:32.169 --> 00:14:34.870 racism against speakers of African American English, 00:14:35.289 --> 00:14:37.269 assigning more negative stereotypes to those 00:14:37.269 --> 00:14:39.570 prompts than recorded human biases do. Yeah, 00:14:39.590 --> 00:14:41.990 the training data reflects societal prejudices. 00:14:42.190 --> 00:14:44.250 Additionally, a recent study found that several 00:14:44.250 --> 00:14:47.970 major LLMs showed persistent anti -Israel bias 00:14:47.970 --> 00:14:51.159 in their generated responses. It just impartially 00:14:51.159 --> 00:14:53.879 scales whatever biases exist in the data it's 00:14:53.879 --> 00:14:55.720 fed. And the application of rigid algorithms 00:14:55.720 --> 00:14:57.879 extends directly into military profiling too. 00:14:58.120 --> 00:15:00.779 The U .S. military utilized AI facial recognition 00:15:00.779 --> 00:15:03.919 systems to ban fertilizer purchases by Afghan 00:15:03.919 --> 00:15:06.100 nationals. Because of explosives, right. Right. 00:15:06.100 --> 00:15:08.240 Improvised explosive devices used against U .S. 00:15:08.240 --> 00:15:10.700 soldiers often contain nitrates derived from 00:15:10.700 --> 00:15:13.519 fertilizer. So the AI enforced a purchasing ban 00:15:13.519 --> 00:15:16.600 by identifying traits mapping to a blanket description 00:15:16.600 --> 00:15:18.879 of Afghan nationals. So it just banned everyone 00:15:18.879 --> 00:15:22.210 who fit the profile. Exactly. As a result, local 00:15:22.210 --> 00:15:25.009 farmers whose entire livelihood depended on seasonal 00:15:25.009 --> 00:15:27.730 agriculture were automatically denied the ability 00:15:27.730 --> 00:15:30.730 to buy basic supplies. Similarly, the Chinese 00:15:30.730 --> 00:15:33.649 government uses artificial intelligence to heavily 00:15:33.649 --> 00:15:36.629 restrict Uyghur minorities in the Xinjiang region. 00:15:37.309 --> 00:15:39.929 The system defines suspicious behavior mathematically 00:15:39.929 --> 00:15:43.169 and automatically denies Uyghurs the ability 00:15:43.169 --> 00:15:45.730 to purchase everyday household commodities, such 00:15:45.730 --> 00:15:48.710 as kitchen knives, unless they pass strict security 00:15:48.710 --> 00:15:51.730 protocols. It's incredibly Yeah, this even includes 00:15:51.730 --> 00:15:54.490 a mandate to etch a personalized algorithmic 00:15:54.490 --> 00:15:57.169 barcode of trustworthiness directly onto the 00:15:57.169 --> 00:15:59.210 blade of the knife itself. These are instances 00:15:59.210 --> 00:16:02.789 of AI applying rigid profiling on a massive structural 00:16:02.789 --> 00:16:05.950 scale. But even on a consumer level, these rigid 00:16:05.950 --> 00:16:08.590 profiles lead to hidden exclusions every single 00:16:08.590 --> 00:16:11.620 day. Like facial recognition software unexpectedly 00:16:11.620 --> 00:16:14.019 suspending transgender Uber drivers because their 00:16:14.019 --> 00:16:16.399 appearance changes during transition. Yes, or 00:16:16.399 --> 00:16:18.500 Grindr being linked to sex offender apps because 00:16:18.500 --> 00:16:21.100 of shared data profiling. And people with disabilities 00:16:21.100 --> 00:16:23.720 are frequently completely ignored by smart AI 00:16:23.720 --> 00:16:25.500 assistants. Because they are missing from the 00:16:25.500 --> 00:16:27.659 training data, right? Mm -hmm. Disabilities are 00:16:27.659 --> 00:16:31.019 incredibly diverse. They can be temporary, situational, 00:16:31.460 --> 00:16:34.299 highly personalized. Because there is a lack 00:16:34.299 --> 00:16:37.519 of explicit categorized data available, partly 00:16:37.519 --> 00:16:40.500 due to the entirely valid privacy fears of disclosing 00:16:40.500 --> 00:16:43.220 a medical disability to a tech company, the machine 00:16:43.220 --> 00:16:45.519 learning models are trained inequitably. That 00:16:45.519 --> 00:16:48.019 makes sense. If people with atypical speech patterns 00:16:48.019 --> 00:16:50.480 aren't heavily included in the data set's training 00:16:50.480 --> 00:16:53.080 voice control features, the AI simply cannot 00:16:53.080 --> 00:16:55.500 understand them. They are mathematically rendered 00:16:55.500 --> 00:16:58.620 invisible. So, what does this all mean? With 00:16:58.620 --> 00:17:01.960 all this damage, from biased hiring tools to 00:17:01.960 --> 00:17:04.900 healthcare proxies to massive global profiling, 00:17:05.599 --> 00:17:06.740 I know you're probably sitting there wondering, 00:17:06.940 --> 00:17:09.680 why can't we just hit undo? Why can't the programmers 00:17:09.680 --> 00:17:11.839 just go in and fix the broken code? It comes 00:17:11.839 --> 00:17:14.380 down to a concept the sociologist Bruno Latour 00:17:14.380 --> 00:17:17.539 calls blackboxing. Blackboxing. Yeah. He argues 00:17:17.539 --> 00:17:20.079 that when scientific or technical work is highly 00:17:20.079 --> 00:17:22.359 successful, it becomes completely invisible. 00:17:22.640 --> 00:17:25.279 We only look at the inputs and the outputs and 00:17:25.279 --> 00:17:28.039 we ignore the dizzying internal complexity Oh 00:17:28.039 --> 00:17:30.160 modern machine learning algorithms are incredibly 00:17:30.160 --> 00:17:34.700 complex black boxes in 2013 Facebook's newsfeed 00:17:34.700 --> 00:17:36.960 used a hundred thousand different data points 00:17:36.960 --> 00:17:39.440 just to determine the layout of your screen a 00:17:39.440 --> 00:17:41.440 hundred thousand Just for the layout just for 00:17:41.440 --> 00:17:44.259 the layout today's models use billions of parameters 00:17:44.259 --> 00:17:47.299 large teams of programmers work in isolated silos 00:17:47.299 --> 00:17:50.079 They borrow old code libraries from previous 00:17:50.079 --> 00:17:53.490 decades. The truth is Often no single human being 00:17:53.490 --> 00:17:55.829 actually understands how the entire algorithm 00:17:55.829 --> 00:17:58.690 operates from end to end. Wait, if we can't understand 00:17:58.690 --> 00:18:00.910 the whole black box, can't we just mandate a 00:18:00.910 --> 00:18:03.509 new rule at the top? Just program the AI with 00:18:03.509 --> 00:18:05.769 a strict definition of fairness? That brings 00:18:05.769 --> 00:18:08.670 us to the ultimate fairness paradox. Defining 00:18:08.670 --> 00:18:10.910 fairness is actually mathematically impossible 00:18:10.910 --> 00:18:13.250 for a machine. Mathematically impossible? Yes, 00:18:13.470 --> 00:18:14.869 because the different definitions of fairness 00:18:14.869 --> 00:18:17.470 contradict each other. Imagine a bank using an 00:18:17.470 --> 00:18:19.930 algorithm to approve loans. You have two groups 00:18:19.930 --> 00:18:23.190 of applicants. Due to historical societal factors, 00:18:23.470 --> 00:18:25.569 group A has a higher average income than group 00:18:25.569 --> 00:18:28.410 B. Okay, following so far. If you want to program 00:18:28.410 --> 00:18:31.349 the machine for equality of treatment, you tell 00:18:31.349 --> 00:18:33.670 it to use the exact same credit score threshold 00:18:33.670 --> 00:18:36.809 for everyone, ignoring race or background. Sounds 00:18:36.809 --> 00:18:39.549 fair. But because of that historical income gap, 00:18:39.759 --> 00:18:42.380 Group A might end up getting 80 % of the approved 00:18:42.380 --> 00:18:45.460 loans. So instead, you tell the machine to optimize 00:18:45.460 --> 00:18:48.579 for equality of outcome. You mandate that loan 00:18:48.579 --> 00:18:51.500 approvals must be a 50 -50 split between the 00:18:51.500 --> 00:18:53.019 two groups. I think just balance the scales. 00:18:53.400 --> 00:18:56.220 Exactly. But to mathematically achieve that 50 00:18:56.220 --> 00:18:59.119 -50 split, the algorithm has to lower the credit 00:18:59.119 --> 00:19:01.759 score threshold for Group B while keeping it 00:19:01.759 --> 00:19:05.819 high for Group A. You have now violated equality 00:19:05.819 --> 00:19:09.059 of treatment. Oh, wow. I see it now. You cannot 00:19:09.059 --> 00:19:11.359 simultaneously have equal outcomes and identical 00:19:11.359 --> 00:19:13.640 treatment. They literally cancel each other out. 00:19:14.200 --> 00:19:16.019 And often forcing either version of fairness 00:19:16.019 --> 00:19:18.440 comes at the direct expense of the model's overall 00:19:18.440 --> 00:19:20.519 predictive accuracy. If we connect this to the 00:19:20.519 --> 00:19:22.519 bigger picture, are we just doomed to be ruled 00:19:22.519 --> 00:19:26.039 by biased black box machines? What are the actual 00:19:26.039 --> 00:19:28.640 solutions here? Well, there's a massive push 00:19:28.640 --> 00:19:31.279 in the computer science world right now for explainable 00:19:31.279 --> 00:19:33.500 AI. What does that look like? These are systems 00:19:33.500 --> 00:19:36.200 intentionally designed so that a human auditor 00:19:36.359 --> 00:19:39.279 can actually trace the logic path and understand 00:19:39.279 --> 00:19:42.819 why the AI made a specific decision. We are also 00:19:42.819 --> 00:19:46.019 seeing the introduction of model cards. Model 00:19:46.019 --> 00:19:48.700 cards? Think of them as standardized nutritional 00:19:48.700 --> 00:19:51.940 warning labels for AI, summarizing the model's 00:19:51.940 --> 00:19:54.839 intended uses, the exact data sets it was trained 00:19:54.839 --> 00:19:57.900 on, and its known limitations or biases. That 00:19:57.900 --> 00:20:00.569 sounds super helpful. But a huge part of the 00:20:00.569 --> 00:20:02.710 solution also has to be addressing who is actually 00:20:02.710 --> 00:20:05.970 building these things, right? Only 12 % of machine 00:20:05.970 --> 00:20:07.890 learning engineers are women. Yes, there is a 00:20:07.890 --> 00:20:10.250 massive diversity crisis in the field. If the 00:20:10.250 --> 00:20:12.829 creators all share the exact same socioeconomic 00:20:12.829 --> 00:20:15.470 background, the exact same blind spots, the machine 00:20:15.470 --> 00:20:17.670 will inevitably absorb and scale those blind 00:20:17.670 --> 00:20:20.130 spots. And while the engineers scramble to improve 00:20:20.130 --> 00:20:22.410 transparency, global governments are finally 00:20:22.410 --> 00:20:24.750 waking up and attempting to regulate. The regulatory 00:20:24.750 --> 00:20:26.690 landscape is definitely fragmented right now. 00:20:26.789 --> 00:20:29.430 In Europe, the GDPR has a specific specific clause, 00:20:29.769 --> 00:20:32.450 Article 22, addressing automated decision making. 00:20:32.569 --> 00:20:35.230 Right, the human in the loop clause. Exactly. 00:20:35.470 --> 00:20:37.529 It gives citizens the right to a human in the 00:20:37.529 --> 00:20:40.549 loop. If an algorithm makes a decision that significantly 00:20:40.549 --> 00:20:43.390 impacts your life, like denying you a mortgage, 00:20:44.190 --> 00:20:46.730 you have the legal right to demand a real human 00:20:46.730 --> 00:20:49.430 being review that automated decision. Meanwhile, 00:20:49.769 --> 00:20:51.930 the U .S. approach is happening entirely at the 00:20:51.930 --> 00:20:55.130 local level. New York City, for example, implemented 00:20:55.130 --> 00:20:58.210 a law requiring employers who use automated AI 00:20:58.210 --> 00:21:01.190 hiring tools to conduct independent bias audits 00:21:01.190 --> 00:21:04.170 every year and publicly post the results. And 00:21:04.170 --> 00:21:06.710 India has proposed a personal data bill that 00:21:06.710 --> 00:21:09.390 explicitly defines discriminatory treatment by 00:21:09.390 --> 00:21:12.049 an algorithm as a specific source of legal harm, 00:21:12.569 --> 00:21:15.309 giving citizens a path to recourse. The era of 00:21:15.309 --> 00:21:17.210 letting the calculator run on its own without 00:21:17.210 --> 00:21:19.920 oversight is ending. And that brings us right 00:21:19.920 --> 00:21:22.759 back to you listening right now. We've journeyed 00:21:22.759 --> 00:21:25.319 from the mainframe computers of the 1970s to 00:21:25.319 --> 00:21:27.680 the broken healthcare proxies to the invisible 00:21:27.680 --> 00:21:30.039 resumes and social media logic gates of today. 00:21:30.299 --> 00:21:32.369 We've covered a lot of ground. We really have. 00:21:32.730 --> 00:21:35.549 And you are now equipped to question the algorithmic 00:21:35.549 --> 00:21:38.150 authority in your daily life. The next time a 00:21:38.150 --> 00:21:39.890 streaming service aggressively hides a certain 00:21:39.890 --> 00:21:42.509 type of artist from you, or you are mysteriously 00:21:42.509 --> 00:21:45.109 denied a credit limit increase, or you see a 00:21:45.109 --> 00:21:47.809 very specific polarizing news feed, you know 00:21:47.809 --> 00:21:50.029 the invisible gears turning behind the screen. 00:21:50.130 --> 00:21:52.910 You understand label choice bias, proxy failures, 00:21:53.230 --> 00:21:55.789 and emergent feedback loops. Absolutely. I want 00:21:55.789 --> 00:21:57.730 to leave you with one final thought to mull over. 00:21:58.029 --> 00:22:00.670 We spent this time seeing how AI is ultimately 00:22:00.670 --> 00:22:04.569 just a mirror. It is reflecting society's historical 00:22:04.569 --> 00:22:07.769 prejudices, our flawed logic, and our deepest 00:22:07.769 --> 00:22:10.609 blind spots. A very clear mirror. Yeah. But perhaps 00:22:10.609 --> 00:22:12.730 the struggle to fix algorithmic bias will be 00:22:12.730 --> 00:22:15.730 the exact mechanism that finally forces us to 00:22:15.730 --> 00:22:18.509 confront and fix the hidden biases within ourselves. 00:22:19.269 --> 00:22:21.230 What if the machine doesn't destroy our morality? 00:22:21.440 --> 00:22:23.819 but actually forces us to perfect it. We started 00:22:23.819 --> 00:22:26.000 this conversation by imagining a calculator, 00:22:26.420 --> 00:22:29.200 perfectly clean and objective. But it turns out 00:22:29.200 --> 00:22:31.599 the algorithm isn't a calculator at all. It's 00:22:31.599 --> 00:22:34.480 an x -ray. It's taking a deeply uncomfortable 00:22:34.480 --> 00:22:36.980 picture of exactly who we are, broken bones and 00:22:36.980 --> 00:22:39.200 all. And now that we can clearly see the fracture 00:22:39.200 --> 00:22:41.700 on the screen, it's on us to set it right. Thank 00:22:41.700 --> 00:22:43.460 you so much for joining us on this deep dive.