You know, usually when we think about changing
our minds, we view it as this kind of moment
of defeat. Right, like a structural failure or
something. Yeah, exactly. You build a belief
system, a crack appears in the foundation, and
you just assume the whole house is going to come
down. And it feels incredibly destabilizing.
I mean, human beings are fundamentally wired
to treat our convictions as absolute destinations
rather than provisional rest stops. But what
if changing your mind wasn't a collapse at all?
What if it was actually like a mathematical upgrade?
Oh, I like that framing. Think of a really good
detective working a complex case. A savvy detective
doesn't just look at a new clue and a total vacuum,
right? No, of course not. They take that brand
new fingerprint or that fresh alibi and they
weigh it against everything they already knew
about the case. Exactly. They update their work
in Peary. And today we're taking a deep dive
into the very math of doing exactly that. Yes,
we are. We've got this massive comprehensive
Wikipedia article sitting in front of us. It
covers the history. the dense mathematics, and
the real -world applications of a concept called
Bayesian inference. one of the most powerful
mental models you can acquire. And our mission
for you today is to basically demystify it. We
want to translate this heavy statistical framework
into a mental shortcut that you can actually
use. Right, because the goal is to walk away
from this deep dive as a sharper, more adaptable
thinker, especially in a world where we are constantly
bombarded with, you know, conflicting information.
So to set the stage here, Bayesian inference
isn't just some dry statistical formula to be
memorized in a universal lecture and then forgotten.
No, not at all. It is a foundational framework
for human reasoning. Its roots actually trace
all the way back to a mathematician and minister
named Thomas Bayes in the 1700s. The 1700s, yeah.
Yeah, and it was later rigorously developed by
Pierre -Simon Laplace, and what these two managed
to do was build this unprecedented bridge. The
bridge between what and what? Well, they connected
pure objective mathematics with human subjective
belief. They essentially wrote the formula for
how to rationally update what you believe when
new information finally arrives. To understand
how that mathematical bridge is actually built
and, you know, how it's currently being used
to do wild things like map the cosmos or train
AI, we first need to strip the complexity all
the way down. We definitely do. So we need to
look at a highly relatable thought experiment
from our source material. It involves cookies.
It is remarkable how often high -level mathematics
relies on baked goods to make a point. It really
grounds the abstract ideas. So I want you to
picture two identical bowls of cookies sitting
on a table in front of you. Let's call it Fred's
Cookie Bowls. OK, I'm picturing it. Two bowls.
Right. Bowl number one has 10 chocolate chip
cookies and 30 plain cookies. So 40 total, but
mostly plain. Got it. Bowl number two has 20
chocolate chip and 20 plain. A perfect 50 -50
split. The setup is clear. Two bowls, different
ratios. Now, our friend Fred blindfolds you,
shuffles the two bowls around on the table so
you totally lose track of them, and asks you
to pick a bowl at random. Okay. Then, from that
random bowl, you reach in and pick a cookie at
random. You take off the blindfold and look at
your hand. It is a plain cookie. Right, a plain
cookie. Question BayesianInference asks is this,
what is the probability that you just pulled
that plain cookie from bowl number one. And this
is where human intuition usually jumps in, far
ahead of the actual math. Oh, totally. Our intuition
immediately screams that it's definitely more
than a 50 % chance, because bowl number one has
way more plain cookies in it. Right. You're holding
a plain cookie so it feels obvious. But Bayesian
inference gives us the precise language and the
precise mathematical engine to actually prove
it. So let's lay out the formal vocabulary. First,
we have what is called the prior. The prior,
yes. Before you even looked at the cookie in
your hand, what was your chance of picking bowl
number one? It was 50 -50. You picked a bowl
entirely at random while blindfolded. Exactly.
The prior probability is your baseline estimate
of a hypothesis before any new evidence is observed.
It's just your starting point. Then we introduce
the likelihood. This measures the compatibility
of the new evidence with the hypothesis. Right,
so the likelihood of pulling a plain cookie from
bowl 1 is 30 out of 40, which is 75%. And the
likelihood of pulling a plain cookie from bowl
2 is 20 out of 40, or 50%. Finally, we take those
numbers and calculate the posterior. The posterior.
That is your updated mathematically sound belief
after incorporating the new evidence. Yeah. When
you plug your 50 % prior and those likelihoods
into Bayes' theorem, the posterior calculates
out to exactly 60%. There's a 60 % chance you
pick from bowl number one. Synthesizing that
into the core formula basically looks like this.
The posterior is proportional to the likelihood
multiplied by the prior. Or, formulated another
way, posterior equals likelihood times prior
divided by the total evidence. Exactly. And the
reason this matters for you listening to this
right now is because it mathematically prevents
the human tendency to overreact to new data.
Oh, that makes sense. It forces us to anchor
our new observations to our existing knowledge
base. So because you saw a plain cookie, your
brain might want to jump to being like 80 or
90 percent sure it was bowl one. Right. But the
math restrains you to 60 percent because it forces
you to remember that your initial chance of picking
bowl one was only a coin toss. It's like a built
-in shock absorber for your opinions. It's a
great way to put it. But that brings up a really
fascinating friction point. Because the poster
your belief, your final conclusion, relies so
heavily on your initial prior belief. What happens
if your starting belief is just incredibly stubborn?
Ah, yeah. That leads us straight into a vital
mathematical principle known as Cromwell's Rule.
Right, Cromwell's Rule. It states that if your
prior probability for a model, your initial foundational
belief, is exactly zero or exactly one, then
absolutely no amount of new evidence can ever
change your mind. And the reason is purely mechanical.
Mathematically, Multiplying any number by zero
always yields zero. Right, it's just basic multiplication.
Exactly. If you assign a prior of zero to a hypothesis,
meaning you believe it is utterly impossible,
it doesn't matter if the new evidence has a massive
undeniable likelihood. The posterior will remain
zero. You are multiplying the near evidence by
zero. Let me play devil's advocate here, though,
because I think a lot of people might push back
on that being a bad thing. We often admire people
with strong, unwavering convictions, right? We
do, yeah. Being 100 % sure of your principles
is frequently framed as a strong leadership trait.
Socially, unwavering conviction might be rewarded.
But mathematically, it's a dead end. It is a
complete cognitive trap. Think of a heavy lock
door. If you approach that door and you're 100
% certain that it is locked, you won't even bother
to try the handle. No, why would you? Right.
Even if someone is standing right next to you,
holding a key, explicitly telling you, hey, I
just unlocked this, your absolute certainty prevents
you from verifying the new evidence. You just
turn around and walk away. That perfectly illustrates
the danger. If we pull back to look at the larger
implications, Cromwell's rule mathematically
reveals why hard fundamentalist convictions are
entirely insensitive to counter -evidence. Wow.
Absolute certainty is the enemy of discovery.
Yes. If you want to be a rational, adaptable
thinker, a turbasion, you must always leave a
tiny fraction of a percentage open to being wrong.
Even just a sliver. Even if your prior is 0 .0001%,
that sliver of doubt is required. If you don't
leave that sliver, the entire mathematical engine
of learning breaks down. You literally cannot
update a belief if you start at exactly 100 %
or exactly 0%. It's mechanically impossible to
learn anything new if you already know everything.
Precisely. So, we've seen how a single piece
of evidence, like one cookie, or one locked door,
updates a single belief. But real life is rarely
that clean, right? Very rarely. How does this
scale up? What does Bayesian inference look like
when we are accumulating evidence over a long
period of time in deeply complex scenarios? To
see that scaling effect, we can examine the medieval
archaeology example from our source text. It
beautifully demonstrates how Bayes' theorem handles
sequential compounding data. Oh, I loved this
part. Imagine an archaeologist is digging at
a newly discovered site. They know from the general
region that the site is from the medieval period,
somewhere between the 11th and 16th centuries.
But they have no idea exactly when, in that 500
-year span, the site was actually inhabited.
Right. So as they begin to excavate, they find
50 fragments of broken pottery. Some of the pottery
is glazed and some is decorated. And the archaeologist
has historical data to compare this against.
They know that if a site is from the early medieval
period, say the 11th century, only about 1 %
of the pottery would typically be glazed and
50 % would be decorated. Okay, so mostly decorated,
barely any glaze. Exactly, but if it's from the
late medieval period, the 16th century, the manufacturing
techniques changed. By then, 81 % would be glazed
and only 5 % decorated. So the evidence is trickling
in one single fragment at a time. How does the
Bayesian math actually process a slow trickle
of clues? It starts with setting what statisticians
call a uniform prior. Meaning they don't have
a favorite century yet. Right. Because the archaeologist
has no initial reason to favor one specific century
over another, they assign a 20 % probability
to each of the five centuries, an equal unbiased
spread. OK, fair enough. Then they dig up the
first pottery fragment. Let's say it's highly
glazed. They apply Bayes' theorem and the probability
shift. The likelihood of the 16th century goes
up a bit, and the likelihood of the 11th century
drops. Because 16th century pots were way more
likely to be glazed. Yes. But here is the critical
mechanism. That new updated probability, the
posterior, now becomes the prior for the second
fragment. Today's posterior is tomorrow's prior.
You just carry the updated math forward to the
next clue. That's the compounding power of it.
This process repeats 50 times. Every fragment
updates the belief. And by the time the 50th
fragment is pulled from the dirt and analyzed,
the math has converged to a highly specific,
dominant conclusion. It has. The calculations
reveal a 63 % chance the site is from the 14th
century. A 36 % chance it's from the 15th century,
and practically zero chance it's from the 11th
or 12th. And there is a mathematical guarantee
backing this process up, known as the Bernstein
von Mises theorem. Yes, quite a mouthful. Without
getting bogged down in the dense academic terminology,
what this theorem essentially proves is that
if you collect enough independent data, your
final conclusion will eventually converge on
the truth, regardless of what your initial prior
was. The sheer volume of incoming data will eventually
wash away the initial bias. That is so cool.
It is. If our archaeologists had started out
completely stubbornly convinced that the site
was from the 11th century, assigning it a 90
% prior, those 50 fragments of highly glazed
pottery would have relentlessly dragged the math
back to the 14th century anyway. The data always
wins in the end, as long as you keep updating.
But hold on. Let me push back on the premise
here. Sure. If the initial prior can just be
whatever the researcher wants it to be, a uniform
prior, or just a subjective expert opinion. How
can Bayesian inference be considered hard objective
science? That is the million dollar question.
Aren't we basically letting personal bias slip
through the back door of the scientific method?
That exact critique completely fractured the
statistical world in the 20th century. After
the 1920s, a rival school of thought called frequentist
statistics actually dominated the field. Frequentist
statistics. Right. Frequentists rely entirely
on objective sampling data. They detest the concept
of a prior because they view it as inherently
unscientific and subjective. They only want to
look at the frequency of events as they appear
in the data itself. To put an analogy on it,
a frequentist is like someone trying to predict
if it will rain tomorrow by pulling up 50 years
of almanacs, seeing that it rained on 12 % of
this specific calendar date historically, and
concluding the chance of rain is exactly 12%.
Because they only look at the historical frequency.
Right. A Bayesian is the person who looks at
that same almanac, but then walks outside, sees
massive black thunder clouds rolling over the
horizon, and says, I'm updating that 12 % prior
to a 90 % posterior because of the new evidence
right in front of me. They incorporate the present
state into the historical data. Exactly. That
is a highly effective way to separate the two.
And the Bayesian approach, relying on inverse
probability to infer backwards from effects to
causes, using a subjective belief, it absolutely
infuriated many traditional scientists. Really?
Like who? Well, the famous philosopher of science,
Karl Popper, explicitly rejected Bayesian rationalism.
Popper argued that using Bayes' rule creates
a vicious circle. How so? He said it presupposes
what it attempts to justify because you are baking
your own beliefs into the starting formula. So
Popper is essentially saying you can't use your
own opinion as part of the math to prove your
own opinion. Exactly. Which sounds like a completely
fair critique. Okay. How did the Beijing camp
defend against that? The defense relies on radical
transparency. Bayesians argue that everyone has
biases and assumptions when they look at data.
That's true. The frequentist approach often hides
those assumptions behind complex sampling models
and arbitrary confidence intervals. The Bayesian
approach, conversely, forces you to explicitly
state your bias out loud as a mathematical prior.
Oh, I see. You have to write down exactly what
your assumptions are before you start calculating.
Precisely. Furthermore, a statistician named
Abraham Wald later proved mathematically that
every admissible statistical decision procedure
is, at its core, either a Bayesian procedure
or a limit of one. Wait, really? Yeah, it turns
out to be the unavoidable foundational math of
making decisions under uncertainty. So instead
of pretending we don't have biases, we put a
hard number on them, put them out in the open,
and let the incoming data correct us over time.
Exactly. And despite those massive philosophical
debates, the Bayesian approach came roaring back
to dominance in the 1980s. It experienced a massive
resurgence, largely thanks to the rise of powerful
computers. Right. Yes. Bayesian math can get
computationally overwhelming very quickly. But
with the invention of Markov chain Monte Carlo
methods, The real -world applications exploded.
We see that term Markov chain Monte Carlo or
MCMC all over the source material. But how do
we actually compute this when the math gets insane?
How does MCMC actually work? Think of it like
a blindfolded hiker trying to find the highest
peak in a massive sprawling mountain range. Okay,
blindfolded hiker. The hiker can't see the whole
map to calculate where the peak is, but they
can take a step, feel if the ground is sloping
up or down, and make a decision to stay or move
higher. Just feeling their way up. Right. Markov
chain Monte Carlo algorithms do something similar
in a landscape of complex probabilities. They
wander around, taking steps, testing the math
locally, and eventually they map a the shape
of the mountain finding the highest probability
without needing to calculate every single inch
of the infinite terrain. That computational power
brings us to where this subjective math is actually
operating in your daily life right now. Yeah.
Let's trace this from your inbox to the literal
edge of the universe. Let's do it. If you've
ever checked your email spam folder, you are
looking at Bayesian inference at work. Yes, specifically
naive Bayes classifiers. The software starts
with a prior belief about what a normal email
looks like. Then it looks at the new evidence,
the specific words in the incoming email. If
it sees the word lottery or Prince, the algorithm
notes that the likelihood of those words appearing
in a legitimate email is very low, but incredibly
high for spam. The posterior probability updates
and the emails aggressively filtered out. The
algorithm learns over time based on past data.
But how does a computer which only understands
ones and zeros possess a subjective bias? It's
because the programmers initially feed it a massive
data set of human -labeled spam to establish
its prior. Right. Humans give it the bias. From
tech, we jump to cosmology. Scientists use Bayesian
inference to literally map the universe. They
do. Our sources point out that researchers use
Bayesian model comparison on the cosmic microwave
background data, the Planck's 2018 data, to fit
the parameters of the standard model of the Big
Bang, the Lambda CDM model. Cosmology is the
perfect application for Bayes' theorem because
of a fundamental physical limitation. We only
have one universe. Yeah, we can't exactly run
a control group on the big bang. Exactly. A frequentist
would prefer to sample multiple universes to
find an average frequency of how big bangs usually
play out. But we can't do that. We only have
one set of cosmic microwave background data.
So what do they do? Cosmologists set prior parameters
based on existing physics, run the data through
those Markov chain Monte Carlo simulations we
just talked about, and update their beliefs about
the fundamental structure of reality. So it filters
our spam and it measures the Big Bang. But here
is where this beautiful mathematical framework
hits a very messy, very human limit. The courtroom.
Oh, this is a fascinating example in the UK.
There was a legal case known as RV Adams The
defense actually brought in an expert to explain
Bayes' theorem to the jury, hoping to help them
combine different pieces of conflicting evidence
mathematically. And the court's reaction to that
was aggressively hostile. Really hostile. The
Court of Appeal ultimately ruled that introducing
Bayes' theorem plunges the jury into inappropriate
and unnecessary realms of theory and complexity,
deflecting them from their proper task. It really
makes you wonder. Are human brains just not wired
for this type of rigorous explicit accumulation
of evidence? It's tough. We naturally crave cohesive
stories and emotional narratives, not probability
matrices. There is a profound friction between
mathematical logic and human nature. The source
text highlights a fascinating counter -argument
regarding this friction by a theorist named Gardner
-Medwin. was his argument. He argues that juries
shouldn't just be calculating the probability
of guilt, they should be looking at the probability
of the evidence given that the defendant is innocent.
Okay, let's break down what that actually means
for a regular person sitting on a jury. If you
are going to use Bayes' theorem to compute a
final posterior probability of someone's guilt,
the math demands that you start with a prior
probability of guilt. But what is the prior probability
that a random person sitting on trial committed
a crime before you even look at the DNA or the
witnesses? How do you even set that number? If
100 ,000 people live in the city where the crime
happened, is the prior one in 100 ,000? Or do
you base it on the defendant's past criminal
record? Yeah. Suddenly you are asking a jury.
to assign a mathematical number to a person's
underlying suspiciousness before looking at the
facts of the case. Which is exactly the problem.
The math works perfectly, but the act of setting
a prior incidence of guilt feels fundamentally
opposed to the legal concept of innocent until
proven guilty. It totally clashes. It requires
knowing a baseline probability of criminality
that is highly controversial to establish in
a specific individual trial. It encapsulates
why this is such a powerful yet demanding mental
model. So to wrap up our deep dive today, what
are the actionable takeaways? I think the main
one is transparency. Yeah. Thinking like a Bajan
means explicitly acknowledging that you have
priors, you have initial biases, you have starting
assumptions, and that is perfectly OK as long
as you make them transparent. Exactly. It means
you have to remain genuinely open to new evidence
and let that evidence iteratively update your
beliefs over time. And above all, avoid 100 %
certainty. Leave the door unlocked, even if it's
just a fraction of a percent. Before we sign
off, I want to leave you with one final, truly
mind -bending concept from the source material.
It's a theory called Solomonov's inductive inference.
Okay, take us down the rabbit hole. Ray Salomonash
mathematically combined Bayesian statistics with
Occam's razor, which is the philosophical principle
that the simplest explanation is usually the
correct one. Right. He used these two concepts
to theorize a framework for universal prediction.
The underlying idea is that if our environment,
our physical reality follows any unknown but
computable probability distribution, then Bayesian
updating could theoretically be used to predict
the yet unseen parts of the universe in an optimal
perfect - fashion. Wait so if the universe operates
on some kind of underlying computable code Bayes
theorem is the literal key to predicting the
future. Given enough incoming data and the correct
universal prior to start with you could hypothetically
predict the unseen sequence of reality perfectly.
Every random event would just be another data
point updating the master equation. Look around
the room you're in right now. It really asks
you to ponder whether everything in your life,
the traffic on your commute, the decisions of
the people around you, the seemingly random chaos
of your day, is just a computable sequence. It's
a wild thought. It might all just be waiting
for the right prior and enough data to be perfectly
mathematically predicted. It brings us right
back to our savvy detective at the start of the
show. It really does. Because in the end, we're
all just detectives trying to weigh the clues
of today against the mysteries of tomorrow. making
sure we never ever lock the door on the truth.