WEBVTT 00:00:00.000 --> 00:00:04.160 Hi, my name is Seth Ariel Green and I'm glad 00:00:04.160 --> 00:00:07.540 to be here on The Vegan Report. I'm a researcher 00:00:07.540 --> 00:00:09.839 at the Humane and Sustainable Food Lab at Stanford 00:00:09.839 --> 00:00:13.000 University. We're a research group at the medical 00:00:13.000 --> 00:00:15.679 school there aimed at accelerating society's 00:00:15.679 --> 00:00:19.620 transition away from factory farming. Today we're 00:00:19.620 --> 00:00:22.519 looking at what gets people to cut back on consumption 00:00:22.519 --> 00:00:26.519 of meat and animal products, specifically persuasion, 00:00:26.859 --> 00:00:30.219 choice architecture, psychological appeals, and 00:00:30.219 --> 00:00:32.759 adding new plant -based meat analogues to menus. 00:00:33.719 --> 00:00:35.920 I'll be discussing two papers that I'm a co -author 00:00:35.920 --> 00:00:39.340 on. The first is called, Meaningfully Reducing 00:00:39.340 --> 00:00:41.820 Consumption of Meat and Animal Products is an 00:00:41.820 --> 00:00:45.000 Unsolved Problem, a Meta -Analysis. The second 00:00:45.000 --> 00:00:48.140 is a preprint called, Taking a Bite Out of Meat 00:00:48.140 --> 00:00:50.799 or Just Giving Fresh Veggies the Boot. Plant 00:00:50.799 --> 00:00:52.979 -based meats did not reduce meat purchasing in 00:00:52.979 --> 00:00:55.179 a randomized controlled menu intervention. If 00:00:55.179 --> 00:00:57.539 you're interested in this work, you can check 00:00:57.539 --> 00:01:02.280 out the lab I work at at foodlabstanford .com. 00:01:02.520 --> 00:01:06.620 That's foodlabstanford .com. We have all of our 00:01:06.620 --> 00:01:08.480 publications there, including the two papers 00:01:08.480 --> 00:01:11.420 that I'm about to discuss. And I also write a 00:01:11.420 --> 00:01:14.420 sub -stack called regression to the meat. That's 00:01:14.420 --> 00:01:18.159 regression to the M -E -A -T. The lab is run 00:01:18.159 --> 00:01:20.579 by Maya Mathur, who is a professor at Stanford. 00:01:21.219 --> 00:01:24.140 And I am a research scientist there. You can 00:01:24.140 --> 00:01:26.819 also find new episodes every Wednesday on this 00:01:26.819 --> 00:01:29.560 podcast. Okay, so we're going to go as follows. 00:01:30.760 --> 00:01:32.859 The meta -analysis, the first paper, is the main 00:01:32.859 --> 00:01:35.000 paper I'm going to be talking about. But because 00:01:35.000 --> 00:01:37.159 meta -analysis is a whole complicated thing, 00:01:37.719 --> 00:01:42.280 I'm going to describe the method in general and 00:01:42.280 --> 00:01:46.060 like why one would do this kind of paper. And 00:01:46.060 --> 00:01:47.640 then we're going to look at the paper itself. 00:01:48.269 --> 00:01:50.810 Next, I'll turn to the second paper. I'll start 00:01:50.810 --> 00:01:53.290 with its setup, and I'll then move to its results. 00:01:54.189 --> 00:01:56.209 And finally, we'll conclude with general lessons 00:01:56.209 --> 00:01:58.390 from the two. So that's five things I hope to 00:01:58.390 --> 00:02:03.790 cover. Maybe six. Okay, so let's start out with 00:02:03.790 --> 00:02:06.010 why meta -analysis, or rather, what is the whole 00:02:06.010 --> 00:02:08.770 point of this thing? Basically, a meta -analysis 00:02:08.770 --> 00:02:10.710 is a paper where you take the results of other 00:02:10.710 --> 00:02:14.110 papers and you amalgamate them into a conclusion 00:02:14.110 --> 00:02:17.590 about a literature as a whole. So, Let's say 00:02:17.590 --> 00:02:21.509 like in the idealized case, you have 10 papers 00:02:21.509 --> 00:02:24.689 that all look at one thing. Let's say it's the 00:02:24.689 --> 00:02:29.289 effects of insecticide treated malaria nets on 00:02:29.289 --> 00:02:32.449 infant mortality. If you have those 10 papers, 00:02:32.810 --> 00:02:34.930 meta -analysis is pretty easy. Basically what 00:02:34.930 --> 00:02:39.210 you do is you put them all together, you look 00:02:39.210 --> 00:02:42.389 at the outcomes, and then you come up with a 00:02:42.389 --> 00:02:47.439 weighted average of those outcomes. where more 00:02:47.439 --> 00:02:50.960 weight goes to more precise estimates. And in 00:02:50.960 --> 00:02:53.020 practice what this means is that bigger papers, 00:02:53.099 --> 00:02:56.159 those with more subjects, get more weight in 00:02:56.159 --> 00:02:58.900 the resulting average. Okay so that's the idealized 00:02:58.900 --> 00:03:03.479 case. Like the intervention is one thing and 00:03:03.479 --> 00:03:06.319 the outcome is one well -defined thing and you 00:03:06.319 --> 00:03:10.020 just give a weighted average of all the 10 interventions 00:03:10.020 --> 00:03:12.539 on the one thing and see like what the overall 00:03:12.539 --> 00:03:16.400 effect is. And the idea here is that If you look 00:03:16.400 --> 00:03:19.599 at 10 studies from 10 different locations, you're 00:03:19.599 --> 00:03:22.199 going to have a better sense of what the so -called 00:03:22.199 --> 00:03:25.219 true underlying population effect is than you 00:03:25.219 --> 00:03:29.479 would if you just chose one study. I think this 00:03:29.479 --> 00:03:31.740 is pretty intuitive. It's the same way that, 00:03:32.460 --> 00:03:33.879 like, if you want to figure out what Americans 00:03:33.879 --> 00:03:35.800 think about politics, you do a random sample 00:03:35.800 --> 00:03:38.719 of 100 people, not just 10 people or not just 00:03:38.719 --> 00:03:42.180 one person. Like, you want just more samples, 00:03:43.080 --> 00:03:46.349 or a larger sample rather. Okay, that's like 00:03:46.349 --> 00:03:51.469 the best -case scenario My lab and well my work 00:03:51.469 --> 00:03:54.750 more generally we're in the behavioral sciences 00:03:54.750 --> 00:03:58.310 and Meta analysis can be a little trickier than 00:03:58.310 --> 00:04:02.389 that The first is that like let's say you wanted 00:04:02.389 --> 00:04:04.370 to ask a question. This is the question that 00:04:04.370 --> 00:04:08.830 my paper actually asks What gets people to eat? 00:04:09.889 --> 00:04:16.430 less meat and animal products okay, so It's pretty 00:04:16.430 --> 00:04:17.730 clear what the outcome is. We're going to look 00:04:17.730 --> 00:04:20.149 at reductions in meat and animal products, which 00:04:20.149 --> 00:04:22.810 I'm henceforth just going to call MAP. So MAP 00:04:22.810 --> 00:04:28.110 is meat and animal products. But it's not at 00:04:28.110 --> 00:04:30.449 all clear that the research field as a whole 00:04:30.449 --> 00:04:32.509 agrees on what the right way to approach that 00:04:32.509 --> 00:04:35.569 is. So you could theoretically write this paper 00:04:35.569 --> 00:04:39.610 and look at thousands of different approaches. 00:04:41.490 --> 00:04:46.670 And in that case, it's not totally clear what 00:04:46.670 --> 00:04:49.589 any weighted average you come up with actually 00:04:49.589 --> 00:04:53.290 means. So like let's say we're looking at two 00:04:53.290 --> 00:04:58.370 papers. We want to meta -analyze them and one 00:04:58.370 --> 00:05:00.730 is a choice architecture intervention. That's 00:05:00.730 --> 00:05:02.529 the kind of thing where you manipulate something 00:05:02.529 --> 00:05:04.750 about an environment where someone eats meat 00:05:04.750 --> 00:05:09.189 and then sees see how that affects how much meat 00:05:09.189 --> 00:05:11.329 someone eats, right? So that could be a good 00:05:11.329 --> 00:05:14.089 example is like making the spoons with which 00:05:14.089 --> 00:05:17.449 people at cafeterias serve meat literally smaller, 00:05:18.050 --> 00:05:22.290 or like putting the plant -based option at eye 00:05:22.290 --> 00:05:24.589 level to try to guide people unconsciously towards 00:05:24.589 --> 00:05:26.529 that. Okay, so you have one study that's like 00:05:26.529 --> 00:05:28.990 that, and the other study is you make people 00:05:28.990 --> 00:05:31.689 watch Dominion, which is this really upsetting 00:05:31.689 --> 00:05:36.470 documentary of how animals are treated on factory 00:05:36.470 --> 00:05:38.709 farms. Okay, and then you average those two things 00:05:38.709 --> 00:05:42.009 together, and you get a number, but like, What 00:05:42.009 --> 00:05:47.589 does that number mean? What is the average of 00:05:47.589 --> 00:05:49.910 changing the spoon size and watching Dominion? 00:05:50.350 --> 00:05:53.569 In some sense, it could be said to represent 00:05:53.569 --> 00:05:57.170 the literature's best efforts. Like if you said, 00:05:57.350 --> 00:05:59.870 this is what our field is doing, this is how 00:05:59.870 --> 00:06:02.610 we're doing, this is our number. But in another 00:06:02.610 --> 00:06:05.269 sense, that average doesn't really correspond 00:06:05.269 --> 00:06:08.689 to anything you can... point to in the real world 00:06:08.689 --> 00:06:10.410 because these two things are just not alike. 00:06:10.550 --> 00:06:15.029 You cannot average the apple -ness of an apple 00:06:15.029 --> 00:06:18.509 and an orange. And then the other side of this 00:06:18.509 --> 00:06:20.730 problem is that the same thing is true of outcomes. 00:06:21.110 --> 00:06:25.170 So if one study reports food frequency questionnaires, 00:06:25.209 --> 00:06:28.649 which are just, you're supposed to tell a researcher 00:06:28.649 --> 00:06:32.050 what you ate over the past 24 hours, and the 00:06:32.050 --> 00:06:38.319 other is watching what people do and don't eat 00:06:38.319 --> 00:06:41.279 at a dining hall, it's also not clear what the 00:06:41.279 --> 00:06:43.980 average of these two outcomes actually signifies. 00:06:44.420 --> 00:06:47.060 Okay, so this is why meta -analysis is sometimes 00:06:47.060 --> 00:06:50.240 taken as like the definitive answer on what is 00:06:50.240 --> 00:06:52.600 the effect of X on Y, but when you read these 00:06:52.600 --> 00:06:58.259 papers, the hard question to ask is like, are 00:06:58.259 --> 00:07:02.160 these inputs even similar enough that we think 00:07:02.160 --> 00:07:06.259 they can be averaged, like conceptually? And 00:07:06.259 --> 00:07:10.240 likewise, are these outcomes all pointing at 00:07:10.240 --> 00:07:13.980 the same target? And if the answer to those questions 00:07:13.980 --> 00:07:19.180 is I'm not sure or I don't know, then you can 00:07:19.180 --> 00:07:22.259 come up with an answer to your question, but 00:07:22.259 --> 00:07:26.060 it's just not clear that it means anything. Okay, 00:07:26.160 --> 00:07:30.639 so that's why meta -analysis is hard. Nevertheless, 00:07:31.519 --> 00:07:34.660 I did one with my co -authors Benny Smith and 00:07:34.660 --> 00:07:38.939 Maya Mathur. on the question of what gets people 00:07:38.939 --> 00:07:42.399 to eat less meat and animal products map, or 00:07:42.399 --> 00:07:44.620 which class of interventions is most effective. 00:07:47.300 --> 00:07:50.560 The way that we handle this problem of, wow, 00:07:50.560 --> 00:07:52.980 studies are so crazily diverse and we can't really 00:07:52.980 --> 00:07:57.720 make sense of everything, is like so. First, 00:07:57.759 --> 00:08:00.620 we only looked at studies that met a pretty high 00:08:00.620 --> 00:08:05.860 bar for research validity. And that meant a few 00:08:05.860 --> 00:08:10.639 different things. First, they actually had to 00:08:10.639 --> 00:08:14.060 measure consumption, not behavioral intentions 00:08:14.060 --> 00:08:17.720 or not attitudes. Pausing here to say they can 00:08:17.720 --> 00:08:19.860 also measure behavioral intentions and attitudes, 00:08:20.160 --> 00:08:22.639 but they had to measure consumption behavior, 00:08:22.839 --> 00:08:24.720 and we only looked at that outcome. So at least 00:08:24.720 --> 00:08:29.220 we're comparing like to like basically for the 00:08:29.220 --> 00:08:32.889 results we're looking at within papers. And the 00:08:32.889 --> 00:08:35.529 second criteria was that they needed to have 00:08:35.529 --> 00:08:38.590 enough people in their samples that we thought 00:08:38.590 --> 00:08:40.610 that we could say something meaningful. So that 00:08:40.610 --> 00:08:43.429 was at least 25 people in treatment and 25 in 00:08:43.429 --> 00:08:49.370 control. And okay, I've got a caveat here, something 00:08:49.370 --> 00:08:55.330 complicated. Not every intervention is assigned 00:08:55.330 --> 00:08:58.509 at the level of individual. Some things are like 00:08:58.509 --> 00:09:01.549 on some days cafeterias do something and then 00:09:01.549 --> 00:09:03.769 not on other days And it's not totally clear 00:09:03.769 --> 00:09:06.610 how many people came in and out So for those 00:09:06.610 --> 00:09:08.990 kinds of studies studies where treatment was 00:09:08.990 --> 00:09:12.169 administered at the level of day or place We 00:09:12.169 --> 00:09:16.149 just we had to say we said at least five Units 00:09:16.149 --> 00:09:18.610 in treatment and control and typically in all 00:09:18.610 --> 00:09:20.850 of those cases. There were like hundreds of people 00:09:20.850 --> 00:09:26.299 actually coming through Okay Third criteria it 00:09:26.299 --> 00:09:28.919 had to be a randomized control trial not any 00:09:28.919 --> 00:09:31.779 other kinds of design. This is just like for 00:09:31.779 --> 00:09:37.440 Getting causal estimates that is like X caused 00:09:37.440 --> 00:09:43.340 Y RCTs Trump other designs and Fourth they had 00:09:43.340 --> 00:09:48.940 to measure outcomes at least a single day after 00:09:48.940 --> 00:09:51.860 Treatment was first administered and the idea 00:09:51.860 --> 00:09:56.039 here was You might have an intervention that 00:09:56.039 --> 00:09:58.340 really really effectively reduces meat for like 00:09:58.340 --> 00:10:01.480 one meal I mean theoretically you might just 00:10:01.480 --> 00:10:04.960 like not serve But you might make it like really 00:10:04.960 --> 00:10:07.620 hard or difficult to get meat like what if you 00:10:07.620 --> 00:10:09.340 had a what if you went into a restaurant? And 00:10:09.340 --> 00:10:11.480 if you ordered a burger, they gave you an electric 00:10:11.480 --> 00:10:13.940 shock and you would be like, oh my god Why'd 00:10:13.940 --> 00:10:16.419 you do that? But that would definitely have an 00:10:16.419 --> 00:10:18.820 effect but then like what happens when you go 00:10:18.820 --> 00:10:22.419 home you probably Hate the people who did this 00:10:22.419 --> 00:10:25.169 to you and eat like twice as many burgers. So 00:10:25.169 --> 00:10:29.509 we only look at studies that measure a delayed 00:10:29.509 --> 00:10:32.570 effect of some sort to try to get at like how 00:10:32.570 --> 00:10:34.470 do people deal with the intervention in their 00:10:34.470 --> 00:10:37.570 lives, like how do they compensate later. Okay 00:10:37.570 --> 00:10:44.389 so that's the setup and when we did that we did 00:10:44.389 --> 00:10:47.549 this like long thorough literature review and 00:10:47.549 --> 00:10:52.679 we came up with about uh that 35 papers met all 00:10:52.679 --> 00:10:55.500 our criteria and that was 41 interventions and 00:10:55.500 --> 00:10:59.539 a hundred and 41 sorry studies 112 interventions 00:10:59.539 --> 00:11:02.120 so this literature when we first started looking 00:11:02.120 --> 00:11:05.019 at it it's like thousands or maybe tens of thousands 00:11:05.019 --> 00:11:08.240 of papers i mean broadly speaking not all of 00:11:08.240 --> 00:11:10.179 them look strictly at like reductions in meat 00:11:10.179 --> 00:11:12.059 and animal products but some of them might look 00:11:12.059 --> 00:11:15.559 at like you know trying to get kids to drink 00:11:15.559 --> 00:11:19.509 like oat milk or even like you know, just cut 00:11:19.509 --> 00:11:21.909 back on fatty foods in general. It's a lot of 00:11:21.909 --> 00:11:24.690 different things here, but it started out as 00:11:24.690 --> 00:11:29.090 a huge literature, ended up with 35 papers. Most 00:11:29.090 --> 00:11:32.529 of those papers have been published since 2020, 00:11:32.529 --> 00:11:35.450 so this is a pretty modern literature. Okay, 00:11:35.629 --> 00:11:38.029 so 35 papers that meet all our criteria, which 00:11:38.029 --> 00:11:39.830 again are about quality, and they actually have 00:11:39.830 --> 00:11:43.049 to be trying to get people to eat less meat and 00:11:43.049 --> 00:11:46.590 or animal products, and those papers fell into 00:11:46.590 --> 00:11:50.539 four categories. Those were persuasion, choice 00:11:50.539 --> 00:11:53.039 architecture, psychology, and then these kind 00:11:53.039 --> 00:11:56.440 of combo persuasion and psychology papers. Going 00:11:56.440 --> 00:12:00.019 through each in turn. Persuasion is what it sounds 00:12:00.019 --> 00:12:02.259 like, is trying to get people to eat less meat 00:12:02.259 --> 00:12:04.960 by talking to them about it, or otherwise using 00:12:04.960 --> 00:12:09.000 information to consciously persuade them. Persuasion 00:12:09.000 --> 00:12:12.700 efforts can come in three broad categories, broadly 00:12:12.700 --> 00:12:14.919 speaking, and those are, I bet you can see this 00:12:14.919 --> 00:12:17.879 coming, health, the environment, and animal welfare. 00:12:18.220 --> 00:12:21.980 So an environmental intervention, a good example 00:12:21.980 --> 00:12:25.419 is a paper by Andrew Jalil and others from 2023 00:12:25.419 --> 00:12:29.340 that replaced an intro economics course with 00:12:29.340 --> 00:12:34.000 just one lecture of it with a lecture about the 00:12:34.000 --> 00:12:37.120 global warming consequences of meat. It also 00:12:37.120 --> 00:12:38.899 covered a little bit about the health into stuff. 00:12:40.299 --> 00:12:45.860 The health version of this is aimed at mostly 00:12:45.860 --> 00:12:48.179 talking about red and processed meat and how 00:12:48.179 --> 00:12:50.679 that's not good for you. But there are some health 00:12:50.679 --> 00:12:52.820 interventions that just broadly make the point 00:12:52.820 --> 00:12:55.059 that, like, you should be eating more vegetables. 00:12:55.159 --> 00:12:57.460 So we looked at those. And then the last one, 00:12:57.539 --> 00:13:00.500 the animal welfare stuff, this is like pamphlets 00:13:00.500 --> 00:13:05.379 and videos and op -eds. A lot of these studies, 00:13:05.480 --> 00:13:08.159 the animal welfare stuff, was run by animal welfare 00:13:08.159 --> 00:13:12.139 organizations. Okay. Those are the persuasion 00:13:12.139 --> 00:13:14.470 approaches. Then there is the choice architecture, 00:13:14.509 --> 00:13:16.230 which is, I've talked a little bit about earlier, 00:13:16.330 --> 00:13:19.529 but to rehash, that's changing the aspects of 00:13:19.529 --> 00:13:22.230 the context in which food is selected and consumed. 00:13:22.669 --> 00:13:26.330 So a good example, Anderson and Nelander, 2021, 00:13:26.990 --> 00:13:32.090 they, as I said earlier, just put the vegetarian 00:13:32.090 --> 00:13:35.389 option at a university cafe for the day at eye 00:13:35.389 --> 00:13:38.870 level or not. So that just changes how you see 00:13:38.870 --> 00:13:42.450 it or not. And then psychology studies are really 00:13:42.509 --> 00:13:48.950 interesting broad category here basically they 00:13:48.950 --> 00:13:53.269 want to manipulate how you think and feel about 00:13:53.269 --> 00:13:55.830 meat without necessarily giving you like direct 00:13:55.830 --> 00:13:58.590 reasons that you should eat it so it's not like 00:13:58.590 --> 00:14:00.750 meat is bad for you so you should eat less meat 00:14:00.750 --> 00:14:03.850 it's more like oh did you know that plant -based 00:14:03.850 --> 00:14:07.210 options are increasingly popular and cool among 00:14:07.210 --> 00:14:10.090 your peers that last example that's like increasingly 00:14:10.090 --> 00:14:13.500 popular and cool this is the norms approach. 00:14:13.879 --> 00:14:16.080 And most psychology studies we looked at were 00:14:16.080 --> 00:14:19.360 norms -based interventions and those basically 00:14:19.360 --> 00:14:23.919 try to make plant -based options seem like either 00:14:23.919 --> 00:14:26.720 normal or desirable. Those are two different 00:14:26.720 --> 00:14:29.399 kinds of norms interventions. And further, you 00:14:29.399 --> 00:14:32.700 might say that plant -based options are normal 00:14:32.700 --> 00:14:34.980 today or you might do this other thing called 00:14:34.980 --> 00:14:39.059 dynamic norms where you say plant -based options 00:14:39.059 --> 00:14:42.919 are increasing in popularity. And the idea is 00:14:42.919 --> 00:14:44.740 there that people will do this pre -conformity 00:14:44.740 --> 00:14:48.279 thing where they want to go to where people are 00:14:48.279 --> 00:14:52.299 going in the future. And finally, we have a couple 00:14:52.299 --> 00:14:56.919 of papers that combine persuasion and psychological 00:14:56.919 --> 00:15:02.279 messages. One example is from Piester et al. 00:15:02.379 --> 00:15:07.389 P -I -E -S -T -E -R 2020. They put up messages 00:15:07.389 --> 00:15:09.549 around a dining hall, which I'm pretty sure was 00:15:09.549 --> 00:15:12.309 UVA, but it's like some school in South Central 00:15:12.309 --> 00:15:17.009 America, or the central region of the South in 00:15:17.009 --> 00:15:23.009 America. And they say, veggie burgers are a tasty, 00:15:23.409 --> 00:15:27.059 healthy choice. they're good for the environment, 00:15:27.139 --> 00:15:30.340 they're sustainable. Also, 95 % of your peers 00:15:30.340 --> 00:15:32.659 agree that veggie burgers are the tasty choice. 00:15:32.980 --> 00:15:35.059 So that's doing some things to try to tell you 00:15:35.059 --> 00:15:37.080 why you should eat them, and it's doing some 00:15:37.080 --> 00:15:39.100 things that are trying to tell you how you should 00:15:39.100 --> 00:15:41.860 think about or feel about the broader question 00:15:41.860 --> 00:15:47.100 of eating plant -based options. Okay, so those 00:15:47.100 --> 00:15:50.879 are our big approaches. We've got persuasion, 00:15:51.340 --> 00:15:54.529 choice architecture, psychology, and combo of 00:15:54.529 --> 00:15:58.149 persuasion and psychology. What are our results? 00:15:58.769 --> 00:16:02.830 Doing a little drumroll here. Overall we find 00:16:02.830 --> 00:16:06.710 nothing very impressive. In meta -analysis it's 00:16:06.710 --> 00:16:09.549 common to standardize everything by turning it 00:16:09.549 --> 00:16:14.610 into hundredths of a standard deviation, so it's 00:16:14.610 --> 00:16:19.049 called Cohen's d, and a d of 1 is one standard 00:16:19.049 --> 00:16:21.450 deviation of change. A way to think about this 00:16:21.450 --> 00:16:25.669 is Like, if I did an intervention on children's 00:16:25.669 --> 00:16:29.570 SAT scores, and I achieve the coincity of 1, 00:16:29.809 --> 00:16:31.830 that would be equivalent to 100 points each on 00:16:31.830 --> 00:16:36.309 the verbal and math increase. Because the standard 00:16:36.309 --> 00:16:38.429 deviation of the SAT is 100 points per section. 00:16:39.990 --> 00:16:41.690 In this context, it's a little less clear what 00:16:41.690 --> 00:16:43.370 a standard deviation means, but like, just go 00:16:43.370 --> 00:16:46.049 with it. It means like, a D of 1 is like, oh, 00:16:46.049 --> 00:16:52.629 you really got the job done. Our overall... Cohen's 00:16:52.629 --> 00:16:57.490 d is 0 .07. That means 7 one hundredths of a 00:16:57.490 --> 00:17:00.649 standard deviation. In general, this corresponds 00:17:00.649 --> 00:17:03.669 to like a few percentage points change or up 00:17:03.669 --> 00:17:06.109 to like five percentage points in some cases, 00:17:06.369 --> 00:17:09.069 or maybe like a, you know, notable reduction 00:17:09.069 --> 00:17:13.450 that levels out over time. Basically, we put 00:17:13.450 --> 00:17:15.569 everything together and we do not come up with 00:17:15.569 --> 00:17:17.210 the kind of transformational changes that we 00:17:17.210 --> 00:17:20.450 were looking for. That's why the paper is titled 00:17:20.759 --> 00:17:22.839 meaningfully reducing meat consumption is an 00:17:22.839 --> 00:17:29.400 unsolved problem. Now let me harken back to the 00:17:29.400 --> 00:17:31.799 problem I was saying earlier about how it's not 00:17:31.799 --> 00:17:33.299 totally clear that when you put all these studies 00:17:33.299 --> 00:17:35.680 together that it means anything because I don't 00:17:35.680 --> 00:17:37.859 know what the average of a psychology study at 00:17:37.859 --> 00:17:40.099 a choice architecture study actually indicates. 00:17:40.579 --> 00:17:44.960 To deal with that we then separated each approach 00:17:44.960 --> 00:17:47.180 into its own theoretical category and we looked 00:17:47.180 --> 00:17:49.839 at those numbers by themselves. Basically here's 00:17:49.839 --> 00:17:52.990 what we found. Choice architecture studies have 00:17:52.990 --> 00:17:55.490 the largest effect on average, about 0 .2, which 00:17:55.490 --> 00:17:57.390 is again about a fifth of a standard deviation 00:17:57.390 --> 00:17:59.309 of change. And that's, you know, in some contexts 00:17:59.309 --> 00:18:02.569 that could be meaningful. But we only had two 00:18:02.569 --> 00:18:06.049 studies that met that. Meaning that almost every 00:18:06.049 --> 00:18:08.009 choice architecture study, like hundreds of them, 00:18:08.390 --> 00:18:11.470 did not qualify. Typically that's because a lot 00:18:11.470 --> 00:18:14.250 of these choice architecture things either manipulate 00:18:14.250 --> 00:18:17.549 hypothetical, environments though it'll like 00:18:17.549 --> 00:18:20.230 take you through an online grocery store and 00:18:20.230 --> 00:18:22.369 then like you know make the Plant -based option 00:18:22.369 --> 00:18:24.329 more prominent, but you don't actually eat anything 00:18:24.329 --> 00:18:27.509 Or choose anything and the other reason is that 00:18:27.509 --> 00:18:29.950 everything is very immediate. So a good example 00:18:29.950 --> 00:18:35.269 is in the defaults literature It's a good example 00:18:35.269 --> 00:18:41.250 The paradigmatic defaults intervention would 00:18:41.250 --> 00:18:44.509 be let's say you were hosting an academic conference 00:18:47.479 --> 00:18:49.680 and you were asked in advance what kind of meal 00:18:49.680 --> 00:18:53.980 you wanted and your choices were vegetarian and 00:18:53.980 --> 00:18:57.000 non -vegetarian. In the default intervention 00:18:57.000 --> 00:19:01.079 version, your choice would just say, you will 00:19:01.079 --> 00:19:03.440 be getting a vegetarian meal. Please email us 00:19:03.440 --> 00:19:06.259 if you don't want that. That's great. It seems 00:19:06.259 --> 00:19:08.059 to have worked very well. My co -author, Maya, 00:19:08.079 --> 00:19:09.680 has a couple of really strong papers in this 00:19:09.680 --> 00:19:13.019 category, but we don't know what people do when 00:19:13.019 --> 00:19:15.480 they go home. or when they like leave the conference. 00:19:16.119 --> 00:19:20.240 So yes, there's like some evidence that just 00:19:20.240 --> 00:19:22.880 making it a little annoying to get meat, like 00:19:22.880 --> 00:19:24.619 you have to write someone an email to say, yeah, 00:19:24.619 --> 00:19:27.720 I want the beef, reduces the amount of meat people 00:19:27.720 --> 00:19:30.880 order. But whether that has any lasting effect 00:19:30.880 --> 00:19:35.299 on people's behavior, I don't know. Okay, so 00:19:35.299 --> 00:19:38.619 that's choice architecture. I would say psychology 00:19:38.619 --> 00:19:42.059 is the second most or psychological interventions. 00:19:42.700 --> 00:19:44.859 They have like about a tenth of a standard deviation 00:19:44.859 --> 00:19:47.259 of change. This dynamic norm stuff, if you want 00:19:47.259 --> 00:19:49.059 to read more about it, there's lots of cool papers. 00:19:49.339 --> 00:19:51.960 If you search like dynamic norms that meet, you'll 00:19:51.960 --> 00:19:55.740 probably encounter papers by Greg Sparkman, who's 00:19:55.740 --> 00:19:58.559 a professor at BC. I would say his lab is like 00:19:58.559 --> 00:20:02.500 the main center of this kind of research. It's 00:20:02.500 --> 00:20:04.500 really hard to use these interventions to get 00:20:04.500 --> 00:20:07.880 people to do things. Or you can get them to change 00:20:07.880 --> 00:20:10.799 a little bit, but it's like it works at a classic 00:20:10.799 --> 00:20:13.750 paper with Greg and co -authors. finds good changes 00:20:13.750 --> 00:20:16.710 in one setting, no changes in another, and like 00:20:16.710 --> 00:20:19.130 a backlash in the third. And when you look at 00:20:19.130 --> 00:20:23.690 this all together, it's really unclear why it 00:20:23.690 --> 00:20:26.049 worked in one place and not in another. We don't 00:20:26.049 --> 00:20:29.829 know. Like one explanation is that maybe there's 00:20:29.829 --> 00:20:31.390 something going on that we don't know about why 00:20:31.390 --> 00:20:33.289 it works and better in some context and others, 00:20:33.349 --> 00:20:35.650 not others, but the other possibility is that 00:20:35.650 --> 00:20:39.269 it's just really noisy. Meaning like the true 00:20:39.269 --> 00:20:44.059 answer is zero, or pretty close to zero, but 00:20:44.059 --> 00:20:47.519 there's spread around that just because of random 00:20:47.519 --> 00:20:51.160 variation, so it's going to look sometimes like 00:20:51.160 --> 00:20:53.640 you get a positive result, but it's just noise. 00:20:54.740 --> 00:20:56.940 Okay, but I don't really know. That's sort of 00:20:56.940 --> 00:21:00.619 an ongoing area of research. In general, persuasion 00:21:00.619 --> 00:21:03.799 efforts, we get a D of 0 .07, which is the same 00:21:03.799 --> 00:21:06.839 as our overall, and that masks a little bit of 00:21:06.839 --> 00:21:09.970 heterogeneity. In general, animal welfare appeals 00:21:09.970 --> 00:21:14.369 don't seem to be working. We get a D of 0 .03 00:21:14.369 --> 00:21:18.549 for all animal welfare appeals. That's not enough 00:21:18.549 --> 00:21:22.490 to do anything with. It's 0 .09 for the environment 00:21:22.490 --> 00:21:25.109 and 0 .08 for health. These are, again, all I'm 00:21:25.109 --> 00:21:26.950 just saying about these is that they're really 00:21:26.950 --> 00:21:31.730 small numbers. For animal welfare, not even a 00:21:31.730 --> 00:21:33.710 percentage point change on average. And for health 00:21:33.710 --> 00:21:35.990 and environment, maybe a couple percentage points 00:21:35.990 --> 00:21:39.480 changes on average. And persuasion and psychology 00:21:39.480 --> 00:21:45.460 combos are like a D of .11, which is about as 00:21:45.460 --> 00:21:49.339 effective as psychology on its own. Okay, so 00:21:49.339 --> 00:21:58.180 what do we make of this? I still don't know how 00:21:58.180 --> 00:22:01.839 to digest these results in, I wrote this stupid 00:22:01.839 --> 00:22:08.549 paper, this great paper. Do we just stop? I mean, 00:22:08.549 --> 00:22:12.690 that's very discouraging, but I would say that 00:22:12.690 --> 00:22:15.369 it's discouraging to me that if you look at like, 00:22:15.549 --> 00:22:17.710 there have been some previous meta -analyses 00:22:17.710 --> 00:22:22.109 that found big results and what we think basically 00:22:22.109 --> 00:22:24.390 happened, or I'm just going to say me, what I 00:22:24.390 --> 00:22:27.970 think happened, is that those papers hoovered 00:22:27.970 --> 00:22:31.450 in studies with much weaker designs and measurement 00:22:31.450 --> 00:22:36.339 strategies and weaker designs are just more likely 00:22:36.339 --> 00:22:40.539 to produce stronger effects for complicated reasons. 00:22:41.480 --> 00:22:45.119 This is what the online writer Guern, G -W -E 00:22:45.119 --> 00:22:48.319 -R -N, calls the stainless steel law of evaluation, 00:22:48.519 --> 00:22:50.259 which is that the better designed the impact 00:22:50.259 --> 00:22:52.559 assessment of a social program, the more likely 00:22:52.559 --> 00:22:55.160 is the resulting estimate of net impact to be 00:22:55.160 --> 00:22:58.079 zero. So we only look at like what I consider 00:22:58.079 --> 00:23:01.000 pretty good studies and we found basically no 00:23:01.000 --> 00:23:03.700 effect. That's, I mean, so in some ways it's 00:23:03.839 --> 00:23:07.440 That's discouraging. On the other hand, I would 00:23:07.440 --> 00:23:10.180 say this literature is getting better and better, 00:23:10.359 --> 00:23:12.119 like in terms of its methodological quality. 00:23:12.259 --> 00:23:14.220 As I said, most of the papers we looked at are 00:23:14.220 --> 00:23:16.819 from pretty recently. That's great. We still 00:23:16.819 --> 00:23:19.160 think it's a lot of untapped ground. And also 00:23:19.160 --> 00:23:21.420 there's a lot of theoretical stuff that can be 00:23:21.420 --> 00:23:26.779 done that isn't in our data set. So for instance, 00:23:27.059 --> 00:23:29.339 what about like extended contact with farm animals? 00:23:29.519 --> 00:23:32.930 Does that change anything? manipulations to the 00:23:32.930 --> 00:23:35.950 price of meat itself, like little stimulated 00:23:35.950 --> 00:23:38.730 small -scale meat taxes. None of those studies 00:23:38.730 --> 00:23:40.569 have been tested in a way that meets our criteria. 00:23:42.089 --> 00:23:45.769 There was a dearth of disgust studies, like, 00:23:45.769 --> 00:23:48.410 ew, oh my god, how much... I had no idea that 00:23:48.410 --> 00:23:51.109 so much milk and blood and pus was allowed in 00:23:51.109 --> 00:23:54.009 milk. Almost nothing that evaluated that was 00:23:54.009 --> 00:23:57.470 done rigorously. I think there's some pretty 00:23:57.470 --> 00:23:59.809 good potential for fiction to be rigorously evaluated, 00:23:59.829 --> 00:24:02.779 like... go ahead and show a lot of people like 00:24:02.779 --> 00:24:05.180 Lisa the Vegetarian, which is a Simpsons episode, 00:24:05.440 --> 00:24:09.119 or Babe, which made its farmer, the guy who played 00:24:09.119 --> 00:24:11.539 the farmer, James Cromwell, he went vegan after 00:24:11.539 --> 00:24:15.119 being in that movie, or Okja, which is like explicitly 00:24:15.119 --> 00:24:18.839 an animal rights parable. So I still think there's 00:24:18.839 --> 00:24:23.619 a lot of room here to go. Now, the next thing 00:24:23.619 --> 00:24:28.200 is that some people might say, what about like 00:24:28.200 --> 00:24:30.490 fake meat options? Isn't that the thing? Doesn't 00:24:30.490 --> 00:24:33.390 everybody love Beyond Meat or Impossible Meat? 00:24:33.410 --> 00:24:36.109 You haven't talked about that. Why not? Which 00:24:36.109 --> 00:24:40.670 gets us to our next paper, which, again, is called 00:24:40.670 --> 00:24:43.569 Taking a Bite Out of Meat or Just Giving Fresh 00:24:43.569 --> 00:24:46.230 Veggies the Boot. Plant -based meals do not reduce 00:24:46.230 --> 00:24:48.710 meat purchasing in a randomized, controlled menu 00:24:48.710 --> 00:24:51.410 intervention. Now, that title basically tells 00:24:51.410 --> 00:24:53.269 you what the whole paper is about, but I'll give 00:24:53.269 --> 00:24:58.960 you the setup. So it's an online survey. We get 00:24:58.960 --> 00:25:02.200 people into the survey and we give them questions 00:25:02.200 --> 00:25:05.220 about like choosing pens, choosing t -shirts. 00:25:05.599 --> 00:25:07.900 It's like, you know, we tell them it's a consumer 00:25:07.900 --> 00:25:10.720 choice survey. And then one of those things is 00:25:10.720 --> 00:25:14.660 they choose a taco filling from a menu that looks 00:25:14.660 --> 00:25:17.160 a lot like Chipotle's. And they were randomized 00:25:17.160 --> 00:25:21.559 into three treatments. The first is the only 00:25:21.559 --> 00:25:24.730 plant -based option is veggies and guac. and 00:25:24.730 --> 00:25:27.210 you got a bunch of meat options like beef barbacoza 00:25:27.210 --> 00:25:31.309 or barbacoa i think and like carnitas and chicken 00:25:31.309 --> 00:25:34.950 and maybe and steak i think okay that's the first 00:25:34.950 --> 00:25:40.690 treatment arm the second is you got the veggies 00:25:40.690 --> 00:25:42.809 and guac but you also have sofritas which is 00:25:42.809 --> 00:25:45.029 this thing that chipotle has it's like a you 00:25:45.029 --> 00:25:48.309 know pork analog tofu based thing if you've been 00:25:48.309 --> 00:25:50.970 to chipotle i think it's pretty good and the 00:25:50.970 --> 00:25:54.289 third option is this hypothetical thing we invented 00:25:54.289 --> 00:25:56.829 and showed people called chickenetus, which is 00:25:56.829 --> 00:25:59.930 like a fake chicken plant -based analog. And 00:25:59.930 --> 00:26:03.930 the idea is just seeing more plant -based options, 00:26:04.250 --> 00:26:06.309 or does having more plant -based options available 00:26:06.309 --> 00:26:08.690 make you more likely to select plant -based? 00:26:09.470 --> 00:26:14.630 Uh, the answer is kind of, like a little, but 00:26:14.630 --> 00:26:17.029 not much and not enough to be statistically significant. 00:26:18.490 --> 00:26:21.609 adding sofritas reduced demand for meat by 1 00:26:21.609 --> 00:26:25.329 .14 percentage points relative to having just 00:26:25.329 --> 00:26:29.170 chips and guac or veggies and guac and adding 00:26:29.170 --> 00:26:31.470 sofritas and chickenitas together reduced demand 00:26:31.470 --> 00:26:38.289 for meat by 2 .14 percentage points so you know 00:26:38.289 --> 00:26:41.529 two percentage points is something if we could 00:26:41.529 --> 00:26:43.930 like do that at all the Chipotle stores in America 00:26:43.930 --> 00:26:45.970 that would probably be a big something But we 00:26:45.970 --> 00:26:48.390 were aiming for five percentage points change, 00:26:48.430 --> 00:26:50.430 which we think is basically like the smallest 00:26:50.430 --> 00:26:53.410 amount we could feasibly tell Chipotle, hey, 00:26:53.509 --> 00:26:56.470 look, there's unmet demand for plant -based options. 00:26:56.589 --> 00:27:02.529 So you should add another one. We also noted 00:27:02.529 --> 00:27:06.569 that as we added more and more veggie options, 00:27:07.009 --> 00:27:10.509 demand for chip for the veggies and guac options 00:27:10.509 --> 00:27:18.579 decreased. So it started out. in the no no plant 00:27:18.579 --> 00:27:23.039 -based meat analogs options it was 9 .2 percent 00:27:23.039 --> 00:27:25.279 and then in the arm with sofritas it goes to 00:27:25.279 --> 00:27:28.759 6 .6 percent and the arm with sofritas and chickenitas 00:27:28.759 --> 00:27:32.420 it goes to 5 .7 percent so one unfortunate conclusion 00:27:32.420 --> 00:27:35.839 to this is that when you add a new plant -based 00:27:35.839 --> 00:27:39.700 meat option meat analog option to your menu at 00:27:39.700 --> 00:27:42.940 least some of that demand is coming from vegetarians 00:27:42.940 --> 00:27:47.799 or people who would otherwise be eating vegetarian. 00:27:50.019 --> 00:27:52.380 The paper has some other results that are kind 00:27:52.380 --> 00:27:54.880 of like just confirming that this is basically 00:27:54.880 --> 00:27:58.339 legit. For instance, men eat more meat than women. 00:27:58.759 --> 00:28:01.779 Republicans eat more meat than Democrats. People 00:28:01.779 --> 00:28:04.099 without a four -year college degree eat more 00:28:04.099 --> 00:28:07.940 meat than people who do or choose more meat in 00:28:07.940 --> 00:28:12.869 this context. When I put all these results together, 00:28:13.769 --> 00:28:17.410 it sure might sound like I'm a huge hater. Like 00:28:17.410 --> 00:28:19.609 I'm saying, oh, nothing works, sorry, go home. 00:28:22.170 --> 00:28:24.269 Sometimes I do feel that way. This is a really 00:28:24.269 --> 00:28:26.750 tough problem we're working on. I think that's 00:28:26.750 --> 00:28:30.009 overall the conclusion that I would give from 00:28:30.009 --> 00:28:33.309 these papers is that it's really hard to get 00:28:33.309 --> 00:28:35.309 people to eat less meat and animal products. 00:28:36.029 --> 00:28:39.529 Anyone who has told you otherwise, my honest 00:28:39.529 --> 00:28:44.279 -to -god opinion, is that they probably weren't 00:28:44.279 --> 00:28:46.299 evaluating the intervention very rigorously. 00:28:47.619 --> 00:28:50.900 And I don't, that's like nothing special about 00:28:50.900 --> 00:28:53.299 our field, this is just true about social science 00:28:53.299 --> 00:28:57.660 research. Again, the better design and evaluation, 00:28:57.839 --> 00:29:00.259 the more likely the resulting estimate of net 00:29:00.259 --> 00:29:03.660 impact is to be zero. On the other hand, I still 00:29:03.660 --> 00:29:04.980 think there are lots of cool things that should 00:29:04.980 --> 00:29:07.640 be tried and can be tested rigorously. And what 00:29:07.640 --> 00:29:09.920 I'm personally interested at the moment is something 00:29:09.920 --> 00:29:12.670 closer to like Mental categorization so like 00:29:12.670 --> 00:29:14.670 how do vegetarians think about the question of 00:29:14.670 --> 00:29:17.609 eating meat versus eating eggs things like that 00:29:17.609 --> 00:29:25.009 and Lastly I would say If and when plant -based 00:29:25.009 --> 00:29:32.750 meat is as good as or even cheaper than Meat 00:29:32.750 --> 00:29:36.730 that's gonna be a game changer, but I just wish 00:29:36.730 --> 00:29:39.599 to caveat that by saying that There does not 00:29:39.599 --> 00:29:41.640 seem to be any evidence of that happening anytime 00:29:41.640 --> 00:29:46.180 soon. So, again, sorry to be the bearer of bad 00:29:46.180 --> 00:29:48.519 news. What I'm saying is basically that this 00:29:48.519 --> 00:29:53.019 is a hard problem, and we just need to keep cracking 00:29:53.019 --> 00:29:57.980 at it. Thanks for listening to my bucket of cold 00:29:57.980 --> 00:30:02.700 water of a podcast recording. If you have any 00:30:02.700 --> 00:30:05.700 questions, you can find me at SethArielGreen 00:30:05.700 --> 00:30:09.509 .com. Green like the color. You can find our 00:30:09.509 --> 00:30:13.130 lab at foodlabstanford .com. You can find my 00:30:13.130 --> 00:30:18.670 newsletter at regressiontothemeat .substack .com. 00:30:19.109 --> 00:30:21.410 Thanks very much for listening and I look forward 00:30:21.410 --> 00:30:22.630 to hearing from you if you have any questions. 00:30:23.609 --> 00:30:23.890 Thanks.