You know, we usually assume more power is always
better. It just like makes intuitive sense. Yeah.
You buy the fastest processor, you get the smartest
AI, you just throw that massive brain at every
single problem you encounter. Right. It's our
default setting. But what if that's entirely
wrong? What if using our absolute smartest tools,
you know, less? is actually the secret to better
results. It sounds completely counterintuitive
at first. I mean, we're totally conditioned to
want the absolute best all the time. Exactly.
Welcome to the Deep Dive. Today, we're exploring
Anthropics Advisor Strategy. Glad to be here.
We're going to deconstruct why defaulting to
the smartest AI model for every single step is
a mistake. And we're going to look at the exact
cost math, too. The token economics here are
honestly fascinating. Yeah, they really are.
We'll also untangle the confusion between the
Cloud Messages API and Cloud Code. Right, because
there's a huge shift happening right now. Oh,
massive. Right. We're moving from picking the
smartest model to designing the smartest tasks.
It completely changes how you build systems.
So let's unpack the core philosophy here. Anthropic
treats intelligence as a managed resource. And
I have to admit something right here. I still
wrestle with burning budget on simple tasks myself.
Oh, you're definitely not alone in that trap.
It's just so incredibly tempting to pick the
strongest model in the dropdown and just hit
go. Yeah. Most people build their AI workflows
exactly that way. They grab the most capable.
model available, then they just wire it into
every single node of their system. They essentially
treat maximum intelligence as like an insurance
policy against failure. But using an insurance
policy for everyday driving is incredibly inefficient.
Exactly. It wastes an enormous amount of money.
It burns through your API rate limits instantly.
And fundamentally, well, it solves the wrong
problem. It ignores how workflows actually function.
Break down what you mean by solving the wrong
problem. Think about a compound agentic workflow.
Most tasks aren't just one monolithic thought.
Right, they have pieces. Yeah, they're a sequence
of mixed steps. Maybe one step requires intense,
multi -step logical reasoning. But the next four
steps, they might just involve basic tool use
or, you know, fetching a specific document from
a database. Or just writing a three -sentence
summary of a text file. Exactly. So the advisor
strategy is basically a structural fix for that
inefficiency. Let me define that jargon for you
real quick. A cheaper AI does basic work, calling
smarter AI for art parts. That is the perfect
distillation of it. Yeah. Inside the system,
you pair two models together. Okay. A fast, highly
efficient model acts as the frontline worker.
It handles all the routing, the searching, the
basic parsing. Right. A much stronger, more expensive
model sits in the background as an advisor. The
frontline model only calls the advisor when it
hits a wall. It makes me think of a high -end
software engineering team. Imagine hiring a top
-tier, world -renowned senior systems architect.
You pay them an absolute fortune, but then you
force them to sit there and fix basic markdown
typos in your documentation all day? Right, and
they'd probably quit out of sheer boredom. Exactly.
You'd let a junior developer handle the typos.
You only tap the senior architect when the server
suddenly crashes. Or when the database architecture
needs a fundamental redesign. Yeah, exactly.
The analogy tracks perfectly. AI models don't
have egos, obviously, but the economic principle
is identical. Right. You simply don't need top
-level intelligence from start to finish. If
90 % of a task stays on the cheaper default path,
your whole system runs lighter. It runs drastically
faster. And obviously it costs a fraction of
the price. Which brings up a crucial operational
question for you. How often do these mixed workflows
actually need to escalate to the smarter model?
Honestly, it's far less often than you'd intuitively
expect. A massive chunk of daily computational
work is just moving data around, which rarely
requires deep reasoning. So basic tasks rarely
escalate, keeping the whole system incredibly
efficient. Yeah. Once you genuinely internalize
intelligence as a managed resource, you stop
asking which single AI should do everything.
You start asking how to divide the labor. Yeah.
Philosophically, putting a junior developer on
typo duty makes perfect sense. But AI isn't human.
Right. Does this theory actually survive contact
with real world math? What happens when Anthropic
forces these models to team up in a lab? That's
where the benchmark data becomes incredibly relevant.
Yeah. Anthropic ran this through S2BE Bench.
And that's a notoriously difficult software engineering
benchmark, right? Yeah, it is. Let's clarify
what SWE Bench actually tests. It's not just
answering trivia. No, it tests complex, agentic
tasks. The AI is given a real -world GitHub issue
and a massive code base. Okay. It has to navigate
the files, find the bug. write the patch, and
ensure it passes tests. So it's heavily reliant
on multi -step execution. Highly reliant. For
this test, they used Clod 3 .5 Sonnet as the
frontline model, but they gave it Clod 3 Opus
as an advisor. Opus is their most capable, deepest
reasoning model. What were the actual results
of pairing them up? The Sonnet and Opus combination
improved the overall score by 2 .7 percentage
points compared to Sonnet working entirely alone.
Okay, so the output quality went up. But what
happened to the actual cost of running that test?
That's the fascinating part. Adding the smarter,
more expensive model actually cut the cost per
task by almost 12%. Wait, really? Adding an expensive
model lowered the total bill. Yes, because Sonnet
was doing all the tedious, time -consuming file
searching. It only invoked Opus for the final
critical code generation. Wow. And they ran another
test that's even more dramatic. They used BrowseComp,
a web browsing benchmark. And they used Haiku
for that one, right? Their absolute fastest,
cheapest model. Exactly. Haiku alone scored a
19 .7 % success rate. Okay. But when they gave
Haiku access to Opus as an advisor, the score
shot up to 41 .2%. It more than doubled the performance.
It doubled the performance. And crucially, it
still costs significantly less than if they had
just forced Opus to do the entire browsing task
from scratch. B, I want to dive into the specific
token economics here because this explains the
why behind those savings. Yeah, let's do it.
Let's look at the actual pricing. Opus is $5
per million input tokens. But it's $25 per million
output tokens. Right. And Sonnet is $3 for input,
$15 for output. Haiku is just $1 for input and
$5 for output. The pattern is glaring. Output
tokens cost five times as much as input tokens
across the board. Yeah, they do. Why is generating
text so much more expensive than reading it?
It comes down to how the underlying transformer
architecture actually processes data. Okay. When
you feed a model a massive document, Kodo, The
EPUs can process those tokens in parallel. Right.
They crunch the whole block of text simultaneously.
It's highly efficient. But generation is different.
Generation is autoressive. It happens sequentially.
The model has to calculate the probability of
the first word. Then it has to read that first
word to calculate the second word. It's a continuous
computationally heavy loop. So if you force an
expensive model like Opus to output long strings
of basic text, you're just setting money on fire.
Precisely. You're paying a premium compute penalty
for incredibly trivial generation. I need to
push back on this a bit, though. Sure. Benchmarks
are tightly controlled laboratory tests. The
prompts are clean. The environments are static.
True. How does this actually hold up in messy
production environments with real unpredictable
users? That's a very fair critique. Production
environments are inherently chaotic. Yeah. Users
will submit bizarre edge cases that benchmarks
simply don't account for. The savings aren't
an automatic guarantee. So does saving money
this way mean we're sacrificing overall output
quality? Not if your escalation logic is tight.
You are forcing a weak model to guess. You're
just stopping an expensive model from overworking.
No, you just stop paying a premium for work that
requires zero deep reasoning. Exactly. You protect
your budget for the moments that actually require
maximum intelligence. The economic value is clearly
proven, but we really need to clarify where the
strategy actually lives. Yeah, we do. Anthropix
toolset is expanding rapidly, and it's causing
a lot of friction and confusion for users. The
naming conventions definitely don't help. People
conflate these distinct tools constantly. Let's
define the core infrastructure first. What is
the Cloud Messages API? The raw building blocks
developers use to create custom AI apps. That's
the perfect way to frame it. The advisor strategy
is a design pattern that lives inside the Cloud
Messages API. Okay. It is strictly for developers.
It's for engineering teams writing their own
code to dictate exactly how an AI system behaves.
But then you have Cloud Code. People hear about
this advisor strategy and assume it's just baked
into everything Anthropic releases. That's the
biggest misconception right now. Cloud Code is
a completely different beast. Right. It's a finished
product. It's a standalone. terminal coding assistant
that you install and use. It does not automatically
use this intricate advisor routing setup under
the hood. Wait, so this intelligent routing isn't
just a toggle switch in the settings? Not at
all. That routing is a highly specific capability.
It has to be manually built and tuned by developers
using the Messages API. You have to write the
code that tells the models how to talk to each
other. Let's try an analogy. Think of the Messages
API. like buying raw engine components. You're
buying the pistons, the crankshaft, and the spark
plugs to build your own custom race car from
scratch. That's the API. But Cloud Code is like
walking onto a dealership lot and buying a finished
sedan. You just turn the key and drive. You can't
suddenly swap the engine out while you're on
the highway. That perfectly captures the dynamic.
The Messages API gives you immense granular control
over model interaction. Cloud Code prioritizes...
instant frictionless usability for a single user.
There are other layers to this too, right? Like
the agent SDK. Yeah, the agent SDK and managed
agents are another layer of abstraction. They're
designed to help teams package agentic behavior
into their software more easily. Gotcha. But
again, if you want to orchestrate this highly
specific multi -model advisor routing, you are
working directly in the API. Let me make sure
this is crystal clear for you. If I am just using
cloud code, Do I have this routing built in automatically?
You don't. You're simply relying on whichever
single model you selected for that specific coding
session. No, this exact automated routing strategy
requires building it yourself in the API. That's
right. You have to intentionally design the handoff
logic yourself. And building that custom logic
takes incredible focus and reliable infrastructure.
We're going to take a quick pause here for our
sponsor. We'll be right back to talk about how
you actually orchestrate this. We are back. So
assuming we have our infrastructure sorted, let's
explore the actual skill required to make this
API routing work. It all comes down to orchestration
and rigorous testing. This is where the engineering
gets genuinely fascinating. Routing the simple,
obvious tasks is an easy win. Sure. The real
challenge is managing the transition point on
the harder tasks. Why is the transition so difficult
to manage? Because the core challenge isn't just
solving a complex task. The system's primary
hurdle is the frontline model actually noticing
that the task is too hard. Yeah. It has to possess
the self -awareness to recognize it needs the
advisor. Whoa, imagine the system organically
realizing it is out of its depth and calling
for help. It's an incredible mechanism. You're
essentially prompting a model. to evaluate its
own limitations in real time. That's wild. It
has to pause mid -workflow and say, my confidence
score on this is dropping. I cannot execute this
reliably alone. That means the escalation logic
is literally a component of the system's overall
intelligence. Precisely. You have to give the
cheaper model a specific tool, like an escalate
to expert function. Right. But a weaker model
might get lucky and guess the right answer without
using the tool. Or worse, it might fail to notice
the complexity entirely. I have to ask about
a very specific risk here. Is there a danger
of an AI version of the Dunning -Kruger effect?
Expand on that. How do you mean? Well, the Dunning
-Kruger effect in humans is when someone incompetent
vastly overestimates their own ability. Right.
Isn't a cheaper, less capable model. inherently
worse at judging its own competence, that feels
like a paradox. Yeah. What if it confidently
hallucinates a totally wrong answer instead of
escalating? That paradox is the exact danger
you have to engineer against. A cheaper model
doesn't inherently know its own blind spots.
So you can't just deploy this. No, you can't
just deploy the strategy blindly and assume it
works. You have to explicitly teach it how to
doubt itself. Yes. You might write a meta prompt
saying, if you cannot extract the exact variable
in three attempts, or if the request involves
multivariable calculus, you must use the escalate
tool. You have to test those boundaries rigorously.
Because if it fails quietly and confidently,
the user experience is destroyed. If the haiku
model misses critical context but confidently
proceeds, the cost savings are entirely useless.
If it escalates too late, You've already burned
tokens on a doomed path. You're no longer just
evaluating the AI's final text output. You are
evaluating the system's internal judgment. So
what is the main point of failure when designing
this system? It's almost exclusively the routing
logic. The cheaper model simply fails to recognize
the edge of its own capabilities. Weak routing
logic failing to escalate when the task suddenly
becomes truly complex. It's the biggest trap
developers fall into. They see the 12 % cost
savings on a spreadsheet and rush the implementation
without tuning the thresholds. Two sec silence.
Let's scale this back from the developer level
for a moment. Let's talk about how you, the listener,
can manually apply this mindset today. How can
we integrate this philosophy into our everyday
personal workflows? The underlying logic absolutely
applies to the everyday user, even if you're
just typing prompts into the standard web interface.
Right. Your deliberate choice of model dictates
your daily budget and your time efficiency. So
how should we structure our own daily projects?
Let's say I'm writing a massive research report.
You should use Opus for the initial planning
step. Then let Sonnet handle the heavy execution.
Plan with Opus, execute with Sonnet. That's a
remarkably clean framework. It's simple. But
it fundamentally shifts how you work. Planning
and execution require entirely different cognitive
muscles from the AI. Explain the difference in
those cognitive muscles. Planning requires high
leverage holistic judgment. When you're outlining
a research report. You need the AI to spot hidden
logical gaps. You need it to synthesize disparate
ideas into a cohesive structural vision. You
want it to deeply understand the root problem.
Yeah. That deep architectural synthesis is exactly
where Opus shines. It's the architect. Opus is
the architect drawing the intricate blueprints
for the skyscraper. Yes. But once those blueprints
are locked in, the nature of the work changes
completely. How so? The execution phase, actually
drafting the sections, is much longer. It's highly
repetitive. It's essentially just following instructions.
So Sonnet is the highly capable builder hammering
the nails according to the blueprint. Exactly.
Sonnet is brilliant at following a well -defined,
strict plan. It writes well, it's fast, and it
doesn't need to invent the core structure. And
remember, you also have Haiku in your toolkit.
Right. Where does Haiku fit into that research
report workflow? Haiku is brilliant for the administrative
microtasks. You need to quickly reformat a list
of citations. Use haiku. Okay. You need to summarize
a 50 -page PDF to see if it's worth reading.
Use haiku. Exploring a bunch of random tangential
ideas very quickly. Haiku is perfect for that.
You really are acting as the orchestrator of
your own manual workflow. You are. You're internalizing
the advisor strategy and applying it to your
own digital life. But does the execution phase
really require less intelligence than the planning
phase? Almost universally, yes. Following a beautifully
detailed map requires significantly less genius
than charting the unknown territory to draw the
map in the first place. Yes. Once the exact path
is clear, steady execution is usually perfectly
fine. That's why you place your strongest, most
expensive model... at the point of maximum leverage.
Let the cheaper, faster models handle the steady,
repetitive execution. Beat, we've covered a massive
amount of technical and philosophical ground
today. Let's bring it all together for you. We
need to recap the central big idea here. We are
sitting in the middle of a massive paradigm shift
in how computing is structured. AI system design
is moving completely away from brute force model
selection. Two years ago, developers just asked,
What is the smartest model I can possibly afford?
That question is officially obsolete. Now the
entire industry is pivoting to task design. The
new foundational question is, what is the most
elegant way to structure this specific task?
System orchestration is quickly becoming the
single most valuable skill in software engineering.
Yeah. You have to know how to aggressively deconstruct
work into micro components. You have to know
exactly which model fits which specific fragmented
piece. It's not about throwing maximum compute
at every minor inconvenience anymore. It's about
precision. It's about systemic elegance. The
companies that win the next decade won't just
be the ones paying for the most expensive API
call. They'll be the ones who engineer systems
that know exactly when not to use them. This
leaves us with a really fascinating thought to
end on. I want you to actively think about this
as you go about your work today. Look closely
at your own daily workflow. Look at how you deploy
your own human brain. How do you mean? Exactly.
How much of your actual workday requires your
personal opus level deep reasoning? Think about
the specific tasks that demand your absolute
best unfiltered cognitive effort. The deep strategy.
The complex problem solving. And then look at
everything else. How much of your day is just
haiku level mindless administrative execution?
Answering basic emails. Formatting spreadsheets.
Exactly. What if you started fiercely preserving
your own mental bandwidth? That is a phenomenal
way to frame personal productivity. What if you
actively managed your human energy the exact
same way we are teaching these APIs to manage
their compute cycles? Save your deep reasoning
for the blueprints. Keep questioning. Keep exploring.
Catch you next time on the Deep Dive.