Hi, my name is Seth Ariel Green and I'm glad
to be here on The Vegan Report. I'm a researcher
at the Humane and Sustainable Food Lab at Stanford
University. We're a research group at the medical
school there aimed at accelerating society's
transition away from factory farming. Today we're
looking at what gets people to cut back on consumption
of meat and animal products, specifically persuasion,
choice architecture, psychological appeals, and
adding new plant -based meat analogues to menus.
I'll be discussing two papers that I'm a co -author
on. The first is called, Meaningfully Reducing
Consumption of Meat and Animal Products is an
Unsolved Problem, a Meta -Analysis. The second
is a preprint called, Taking a Bite Out of Meat
or Just Giving Fresh Veggies the Boot. Plant
-based meats did not reduce meat purchasing in
a randomized controlled menu intervention. If
you're interested in this work, you can check
out the lab I work at at foodlabstanford .com.
That's foodlabstanford .com. We have all of our
publications there, including the two papers
that I'm about to discuss. And I also write a
sub -stack called regression to the meat. That's
regression to the M -E -A -T. The lab is run
by Maya Mathur, who is a professor at Stanford.
And I am a research scientist there. You can
also find new episodes every Wednesday on this
podcast. Okay, so we're going to go as follows.
The meta -analysis, the first paper, is the main
paper I'm going to be talking about. But because
meta -analysis is a whole complicated thing,
I'm going to describe the method in general and
like why one would do this kind of paper. And
then we're going to look at the paper itself.
Next, I'll turn to the second paper. I'll start
with its setup, and I'll then move to its results.
And finally, we'll conclude with general lessons
from the two. So that's five things I hope to
cover. Maybe six. Okay, so let's start out with
why meta -analysis, or rather, what is the whole
point of this thing? Basically, a meta -analysis
is a paper where you take the results of other
papers and you amalgamate them into a conclusion
about a literature as a whole. So, Let's say
like in the idealized case, you have 10 papers
that all look at one thing. Let's say it's the
effects of insecticide treated malaria nets on
infant mortality. If you have those 10 papers,
meta -analysis is pretty easy. Basically what
you do is you put them all together, you look
at the outcomes, and then you come up with a
weighted average of those outcomes. where more
weight goes to more precise estimates. And in
practice what this means is that bigger papers,
those with more subjects, get more weight in
the resulting average. Okay so that's the idealized
case. Like the intervention is one thing and
the outcome is one well -defined thing and you
just give a weighted average of all the 10 interventions
on the one thing and see like what the overall
effect is. And the idea here is that If you look
at 10 studies from 10 different locations, you're
going to have a better sense of what the so -called
true underlying population effect is than you
would if you just chose one study. I think this
is pretty intuitive. It's the same way that,
like, if you want to figure out what Americans
think about politics, you do a random sample
of 100 people, not just 10 people or not just
one person. Like, you want just more samples,
or a larger sample rather. Okay, that's like
the best -case scenario My lab and well my work
more generally we're in the behavioral sciences
and Meta analysis can be a little trickier than
that The first is that like let's say you wanted
to ask a question. This is the question that
my paper actually asks What gets people to eat?
less meat and animal products okay, so It's pretty
clear what the outcome is. We're going to look
at reductions in meat and animal products, which
I'm henceforth just going to call MAP. So MAP
is meat and animal products. But it's not at
all clear that the research field as a whole
agrees on what the right way to approach that
is. So you could theoretically write this paper
and look at thousands of different approaches.
And in that case, it's not totally clear what
any weighted average you come up with actually
means. So like let's say we're looking at two
papers. We want to meta -analyze them and one
is a choice architecture intervention. That's
the kind of thing where you manipulate something
about an environment where someone eats meat
and then sees see how that affects how much meat
someone eats, right? So that could be a good
example is like making the spoons with which
people at cafeterias serve meat literally smaller,
or like putting the plant -based option at eye
level to try to guide people unconsciously towards
that. Okay, so you have one study that's like
that, and the other study is you make people
watch Dominion, which is this really upsetting
documentary of how animals are treated on factory
farms. Okay, and then you average those two things
together, and you get a number, but like, What
does that number mean? What is the average of
changing the spoon size and watching Dominion?
In some sense, it could be said to represent
the literature's best efforts. Like if you said,
this is what our field is doing, this is how
we're doing, this is our number. But in another
sense, that average doesn't really correspond
to anything you can... point to in the real world
because these two things are just not alike.
You cannot average the apple -ness of an apple
and an orange. And then the other side of this
problem is that the same thing is true of outcomes.
So if one study reports food frequency questionnaires,
which are just, you're supposed to tell a researcher
what you ate over the past 24 hours, and the
other is watching what people do and don't eat
at a dining hall, it's also not clear what the
average of these two outcomes actually signifies.
Okay, so this is why meta -analysis is sometimes
taken as like the definitive answer on what is
the effect of X on Y, but when you read these
papers, the hard question to ask is like, are
these inputs even similar enough that we think
they can be averaged, like conceptually? And
likewise, are these outcomes all pointing at
the same target? And if the answer to those questions
is I'm not sure or I don't know, then you can
come up with an answer to your question, but
it's just not clear that it means anything. Okay,
so that's why meta -analysis is hard. Nevertheless,
I did one with my co -authors Benny Smith and
Maya Mathur. on the question of what gets people
to eat less meat and animal products map, or
which class of interventions is most effective.
The way that we handle this problem of, wow,
studies are so crazily diverse and we can't really
make sense of everything, is like so. First,
we only looked at studies that met a pretty high
bar for research validity. And that meant a few
different things. First, they actually had to
measure consumption, not behavioral intentions
or not attitudes. Pausing here to say they can
also measure behavioral intentions and attitudes,
but they had to measure consumption behavior,
and we only looked at that outcome. So at least
we're comparing like to like basically for the
results we're looking at within papers. And the
second criteria was that they needed to have
enough people in their samples that we thought
that we could say something meaningful. So that
was at least 25 people in treatment and 25 in
control. And okay, I've got a caveat here, something
complicated. Not every intervention is assigned
at the level of individual. Some things are like
on some days cafeterias do something and then
not on other days And it's not totally clear
how many people came in and out So for those
kinds of studies studies where treatment was
administered at the level of day or place We
just we had to say we said at least five Units
in treatment and control and typically in all
of those cases. There were like hundreds of people
actually coming through Okay Third criteria it
had to be a randomized control trial not any
other kinds of design. This is just like for
Getting causal estimates that is like X caused
Y RCTs Trump other designs and Fourth they had
to measure outcomes at least a single day after
Treatment was first administered and the idea
here was You might have an intervention that
really really effectively reduces meat for like
one meal I mean theoretically you might just
like not serve But you might make it like really
hard or difficult to get meat like what if you
had a what if you went into a restaurant? And
if you ordered a burger, they gave you an electric
shock and you would be like, oh my god Why'd
you do that? But that would definitely have an
effect but then like what happens when you go
home you probably Hate the people who did this
to you and eat like twice as many burgers. So
we only look at studies that measure a delayed
effect of some sort to try to get at like how
do people deal with the intervention in their
lives, like how do they compensate later. Okay
so that's the setup and when we did that we did
this like long thorough literature review and
we came up with about uh that 35 papers met all
our criteria and that was 41 interventions and
a hundred and 41 sorry studies 112 interventions
so this literature when we first started looking
at it it's like thousands or maybe tens of thousands
of papers i mean broadly speaking not all of
them look strictly at like reductions in meat
and animal products but some of them might look
at like you know trying to get kids to drink
like oat milk or even like you know, just cut
back on fatty foods in general. It's a lot of
different things here, but it started out as
a huge literature, ended up with 35 papers. Most
of those papers have been published since 2020,
so this is a pretty modern literature. Okay,
so 35 papers that meet all our criteria, which
again are about quality, and they actually have
to be trying to get people to eat less meat and
or animal products, and those papers fell into
four categories. Those were persuasion, choice
architecture, psychology, and then these kind
of combo persuasion and psychology papers. Going
through each in turn. Persuasion is what it sounds
like, is trying to get people to eat less meat
by talking to them about it, or otherwise using
information to consciously persuade them. Persuasion
efforts can come in three broad categories, broadly
speaking, and those are, I bet you can see this
coming, health, the environment, and animal welfare.
So an environmental intervention, a good example
is a paper by Andrew Jalil and others from 2023
that replaced an intro economics course with
just one lecture of it with a lecture about the
global warming consequences of meat. It also
covered a little bit about the health into stuff.
The health version of this is aimed at mostly
talking about red and processed meat and how
that's not good for you. But there are some health
interventions that just broadly make the point
that, like, you should be eating more vegetables.
So we looked at those. And then the last one,
the animal welfare stuff, this is like pamphlets
and videos and op -eds. A lot of these studies,
the animal welfare stuff, was run by animal welfare
organizations. Okay. Those are the persuasion
approaches. Then there is the choice architecture,
which is, I've talked a little bit about earlier,
but to rehash, that's changing the aspects of
the context in which food is selected and consumed.
So a good example, Anderson and Nelander, 2021,
they, as I said earlier, just put the vegetarian
option at a university cafe for the day at eye
level or not. So that just changes how you see
it or not. And then psychology studies are really
interesting broad category here basically they
want to manipulate how you think and feel about
meat without necessarily giving you like direct
reasons that you should eat it so it's not like
meat is bad for you so you should eat less meat
it's more like oh did you know that plant -based
options are increasingly popular and cool among
your peers that last example that's like increasingly
popular and cool this is the norms approach.
And most psychology studies we looked at were
norms -based interventions and those basically
try to make plant -based options seem like either
normal or desirable. Those are two different
kinds of norms interventions. And further, you
might say that plant -based options are normal
today or you might do this other thing called
dynamic norms where you say plant -based options
are increasing in popularity. And the idea is
there that people will do this pre -conformity
thing where they want to go to where people are
going in the future. And finally, we have a couple
of papers that combine persuasion and psychological
messages. One example is from Piester et al.
P -I -E -S -T -E -R 2020. They put up messages
around a dining hall, which I'm pretty sure was
UVA, but it's like some school in South Central
America, or the central region of the South in
America. And they say, veggie burgers are a tasty,
healthy choice. they're good for the environment,
they're sustainable. Also, 95 % of your peers
agree that veggie burgers are the tasty choice.
So that's doing some things to try to tell you
why you should eat them, and it's doing some
things that are trying to tell you how you should
think about or feel about the broader question
of eating plant -based options. Okay, so those
are our big approaches. We've got persuasion,
choice architecture, psychology, and combo of
persuasion and psychology. What are our results?
Doing a little drumroll here. Overall we find
nothing very impressive. In meta -analysis it's
common to standardize everything by turning it
into hundredths of a standard deviation, so it's
called Cohen's d, and a d of 1 is one standard
deviation of change. A way to think about this
is Like, if I did an intervention on children's
SAT scores, and I achieve the coincity of 1,
that would be equivalent to 100 points each on
the verbal and math increase. Because the standard
deviation of the SAT is 100 points per section.
In this context, it's a little less clear what
a standard deviation means, but like, just go
with it. It means like, a D of 1 is like, oh,
you really got the job done. Our overall... Cohen's
d is 0 .07. That means 7 one hundredths of a
standard deviation. In general, this corresponds
to like a few percentage points change or up
to like five percentage points in some cases,
or maybe like a, you know, notable reduction
that levels out over time. Basically, we put
everything together and we do not come up with
the kind of transformational changes that we
were looking for. That's why the paper is titled
meaningfully reducing meat consumption is an
unsolved problem. Now let me harken back to the
problem I was saying earlier about how it's not
totally clear that when you put all these studies
together that it means anything because I don't
know what the average of a psychology study at
a choice architecture study actually indicates.
To deal with that we then separated each approach
into its own theoretical category and we looked
at those numbers by themselves. Basically here's
what we found. Choice architecture studies have
the largest effect on average, about 0 .2, which
is again about a fifth of a standard deviation
of change. And that's, you know, in some contexts
that could be meaningful. But we only had two
studies that met that. Meaning that almost every
choice architecture study, like hundreds of them,
did not qualify. Typically that's because a lot
of these choice architecture things either manipulate
hypothetical, environments though it'll like
take you through an online grocery store and
then like you know make the Plant -based option
more prominent, but you don't actually eat anything
Or choose anything and the other reason is that
everything is very immediate. So a good example
is in the defaults literature It's a good example
The paradigmatic defaults intervention would
be let's say you were hosting an academic conference
and you were asked in advance what kind of meal
you wanted and your choices were vegetarian and
non -vegetarian. In the default intervention
version, your choice would just say, you will
be getting a vegetarian meal. Please email us
if you don't want that. That's great. It seems
to have worked very well. My co -author, Maya,
has a couple of really strong papers in this
category, but we don't know what people do when
they go home. or when they like leave the conference.
So yes, there's like some evidence that just
making it a little annoying to get meat, like
you have to write someone an email to say, yeah,
I want the beef, reduces the amount of meat people
order. But whether that has any lasting effect
on people's behavior, I don't know. Okay, so
that's choice architecture. I would say psychology
is the second most or psychological interventions.
They have like about a tenth of a standard deviation
of change. This dynamic norm stuff, if you want
to read more about it, there's lots of cool papers.
If you search like dynamic norms that meet, you'll
probably encounter papers by Greg Sparkman, who's
a professor at BC. I would say his lab is like
the main center of this kind of research. It's
really hard to use these interventions to get
people to do things. Or you can get them to change
a little bit, but it's like it works at a classic
paper with Greg and co -authors. finds good changes
in one setting, no changes in another, and like
a backlash in the third. And when you look at
this all together, it's really unclear why it
worked in one place and not in another. We don't
know. Like one explanation is that maybe there's
something going on that we don't know about why
it works and better in some context and others,
not others, but the other possibility is that
it's just really noisy. Meaning like the true
answer is zero, or pretty close to zero, but
there's spread around that just because of random
variation, so it's going to look sometimes like
you get a positive result, but it's just noise.
Okay, but I don't really know. That's sort of
an ongoing area of research. In general, persuasion
efforts, we get a D of 0 .07, which is the same
as our overall, and that masks a little bit of
heterogeneity. In general, animal welfare appeals
don't seem to be working. We get a D of 0 .03
for all animal welfare appeals. That's not enough
to do anything with. It's 0 .09 for the environment
and 0 .08 for health. These are, again, all I'm
just saying about these is that they're really
small numbers. For animal welfare, not even a
percentage point change on average. And for health
and environment, maybe a couple percentage points
changes on average. And persuasion and psychology
combos are like a D of .11, which is about as
effective as psychology on its own. Okay, so
what do we make of this? I still don't know how
to digest these results in, I wrote this stupid
paper, this great paper. Do we just stop? I mean,
that's very discouraging, but I would say that
it's discouraging to me that if you look at like,
there have been some previous meta -analyses
that found big results and what we think basically
happened, or I'm just going to say me, what I
think happened, is that those papers hoovered
in studies with much weaker designs and measurement
strategies and weaker designs are just more likely
to produce stronger effects for complicated reasons.
This is what the online writer Guern, G -W -E
-R -N, calls the stainless steel law of evaluation,
which is that the better designed the impact
assessment of a social program, the more likely
is the resulting estimate of net impact to be
zero. So we only look at like what I consider
pretty good studies and we found basically no
effect. That's, I mean, so in some ways it's
That's discouraging. On the other hand, I would
say this literature is getting better and better,
like in terms of its methodological quality.
As I said, most of the papers we looked at are
from pretty recently. That's great. We still
think it's a lot of untapped ground. And also
there's a lot of theoretical stuff that can be
done that isn't in our data set. So for instance,
what about like extended contact with farm animals?
Does that change anything? manipulations to the
price of meat itself, like little stimulated
small -scale meat taxes. None of those studies
have been tested in a way that meets our criteria.
There was a dearth of disgust studies, like,
ew, oh my god, how much... I had no idea that
so much milk and blood and pus was allowed in
milk. Almost nothing that evaluated that was
done rigorously. I think there's some pretty
good potential for fiction to be rigorously evaluated,
like... go ahead and show a lot of people like
Lisa the Vegetarian, which is a Simpsons episode,
or Babe, which made its farmer, the guy who played
the farmer, James Cromwell, he went vegan after
being in that movie, or Okja, which is like explicitly
an animal rights parable. So I still think there's
a lot of room here to go. Now, the next thing
is that some people might say, what about like
fake meat options? Isn't that the thing? Doesn't
everybody love Beyond Meat or Impossible Meat?
You haven't talked about that. Why not? Which
gets us to our next paper, which, again, is called
Taking a Bite Out of Meat or Just Giving Fresh
Veggies the Boot. Plant -based meals do not reduce
meat purchasing in a randomized, controlled menu
intervention. Now, that title basically tells
you what the whole paper is about, but I'll give
you the setup. So it's an online survey. We get
people into the survey and we give them questions
about like choosing pens, choosing t -shirts.
It's like, you know, we tell them it's a consumer
choice survey. And then one of those things is
they choose a taco filling from a menu that looks
a lot like Chipotle's. And they were randomized
into three treatments. The first is the only
plant -based option is veggies and guac. and
you got a bunch of meat options like beef barbacoza
or barbacoa i think and like carnitas and chicken
and maybe and steak i think okay that's the first
treatment arm the second is you got the veggies
and guac but you also have sofritas which is
this thing that chipotle has it's like a you
know pork analog tofu based thing if you've been
to chipotle i think it's pretty good and the
third option is this hypothetical thing we invented
and showed people called chickenetus, which is
like a fake chicken plant -based analog. And
the idea is just seeing more plant -based options,
or does having more plant -based options available
make you more likely to select plant -based?
Uh, the answer is kind of, like a little, but
not much and not enough to be statistically significant.
adding sofritas reduced demand for meat by 1
.14 percentage points relative to having just
chips and guac or veggies and guac and adding
sofritas and chickenitas together reduced demand
for meat by 2 .14 percentage points so you know
two percentage points is something if we could
like do that at all the Chipotle stores in America
that would probably be a big something But we
were aiming for five percentage points change,
which we think is basically like the smallest
amount we could feasibly tell Chipotle, hey,
look, there's unmet demand for plant -based options.
So you should add another one. We also noted
that as we added more and more veggie options,
demand for chip for the veggies and guac options
decreased. So it started out. in the no no plant
-based meat analogs options it was 9 .2 percent
and then in the arm with sofritas it goes to
6 .6 percent and the arm with sofritas and chickenitas
it goes to 5 .7 percent so one unfortunate conclusion
to this is that when you add a new plant -based
meat option meat analog option to your menu at
least some of that demand is coming from vegetarians
or people who would otherwise be eating vegetarian.
The paper has some other results that are kind
of like just confirming that this is basically
legit. For instance, men eat more meat than women.
Republicans eat more meat than Democrats. People
without a four -year college degree eat more
meat than people who do or choose more meat in
this context. When I put all these results together,
it sure might sound like I'm a huge hater. Like
I'm saying, oh, nothing works, sorry, go home.
Sometimes I do feel that way. This is a really
tough problem we're working on. I think that's
overall the conclusion that I would give from
these papers is that it's really hard to get
people to eat less meat and animal products.
Anyone who has told you otherwise, my honest
-to -god opinion, is that they probably weren't
evaluating the intervention very rigorously.
And I don't, that's like nothing special about
our field, this is just true about social science
research. Again, the better design and evaluation,
the more likely the resulting estimate of net
impact is to be zero. On the other hand, I still
think there are lots of cool things that should
be tried and can be tested rigorously. And what
I'm personally interested at the moment is something
closer to like Mental categorization so like
how do vegetarians think about the question of
eating meat versus eating eggs things like that
and Lastly I would say If and when plant -based
meat is as good as or even cheaper than Meat
that's gonna be a game changer, but I just wish
to caveat that by saying that There does not
seem to be any evidence of that happening anytime
soon. So, again, sorry to be the bearer of bad
news. What I'm saying is basically that this
is a hard problem, and we just need to keep cracking
at it. Thanks for listening to my bucket of cold
water of a podcast recording. If you have any
questions, you can find me at SethArielGreen
.com. Green like the color. You can find our
lab at foodlabstanford .com. You can find my
newsletter at regressiontothemeat .substack .com.
Thanks very much for listening and I look forward
to hearing from you if you have any questions.
Thanks.