welcome to money is freedom
a podcast
exploring how finance and freedom connect in our lives
created from thoughtful research
and narrated by Notebook L m
a I this series brings you clear
meaningful insights into finance and beyond
welcome to episode 41 where we explore how AI
agents are evolving from isolated tools
into structured interoperable systems
what some are calling the emerging agent stack
this shift is reshaping the way we build and
interact with AI
and it's happening fast
we're drawing inspiration from a fantastic
blog post by John Turau
the Agent Stack Emerges which maps out the core layers
from memory and planning to orchestration
that are forming the backbone of next gen AI systems
if you're building in the AI space
or just trying to make sense of the chaos
this episode is for you let's dive in
welcome to the deep dive
today we're uh
diving into something really moving fast
the infrastructure behind AI agents
yes it's evolving incredibly quickly
we got some great material that shows how the actual
you know foundations for these AI
apps are being radically rethought
not just built but redesigned exactly
it's a real shift we're moving away from well
what felt like a lot of one off experiments
just a short while ago right
and now now we're seeing um
much more structure clearer layers are forming
it's not just about speed increases either
it's yeah
it's making people fundamentally rethink
software development itself
that's really the core of it
isn't it these foundational shifts
yeah AI agents aren't just some SCI fi concept anymore
they're actually pushing changes in infrastructure now
right now so
we're digging into this whole emerging AI
agent infrastructure stack
what we hope you'll get is a clearer picture
of how fast this is moving
and maybe surprisingly
the patterns that are actually starting to stick
and the growth speed is well
it's pretty staggering honestly
companies really leaning into AI
agents are scaling in ways that just look different
totally different I mean
look at cursor they hit what
a hundred million dollar a r r yeah
in just 21 months with a tiny team like 20 people
and then there's bolt and lovable right
$10 million basically overnight
couple months with 15 people each
it really makes you question
the old rules about scaling a business
doesn't it absolutely
is that sustainable
or is it just a new kind of economic model emerging
here that's the million dollar question
maybe billion dollar but what it definitely shows is
they're breaking free from old constraints
it's not just you know
shaving off a few percentage points of inefficiency
no it feels bigger
it's using AI for massive operational leverage
and the infrastructure has to scramble to keep up
to support this kind of hyper growth
it's funny thinking back like less than a year ago
June 2024
people were saying this space was pretty empty
oh yeah very DIY
everyone building their own thing from scratch
yeah except little labs
exactly lots of experiments
which is great but also loads of duplicated effort
because there weren't really established ways of doing
things and now boom
just a few months later the pictures way clearer
and we have actual data showing this speed up
yeah like the neon report
the serverless postgress folks right
what did they find they're seeing
AI agents creating databases on their platform
more than four times faster than human developers
four times that's huge
it's not just a little bump
it accelerates their core metric across the board
what does that even mean for the future
back ends becoming super dynamic maybe yeah
maybe databases just spin up and down instantly
based on AI needs it definitely points that way
more fluid
more responsive infrastructure driven by the AI itself
and it's not just the back end stuff
no definitely not
look at browser bases stage hand library
that's for UI automation
letting agents actually use websites like we do
ah yeah
clicking buttons filling forms
exactly and it just crossed
half a million monthly installs on npm wow
500 k installs a month
that signals massive demand for letting AI
interact with the visual web right
absolutely for a specialized library like that
it really underlines how intense the need is
it almost makes you think
if you're starting a company now
are you already late huh
you could think that but what's weird is we're
still seeing brand new companies founded
even this year getting real traction
so why is that do you think
well
I guess it means the ground is still shifting so fast
new needs new niches pop up constantly
the core tech is still being figured out
so there's room for fresh ideas
so
we're past the initial Wild West experimentation phase
I think so that's a key difference
a year ago it was mostly exploration
developers starting from zero
trying to invent the wheel right
lots of trial and error figure out the basics
but now we're seeing actual repeatable patterns emerge
and smart teams aren't starting from scratch anymore
they're using these patterns
okay so what are these patterns
can we categorize them yeah
people are starting to identify a few common types
first up there's what they're calling next gen copilots
okay like the coding assistants we know
kind of but more advanced
they're context aware
they proactively help with complex stuff
think bolt dot nu
air ops colimit
how are they different from
say git hub copilot
well the idea seems to be
they go beyond just reacting to your commands
they actually understand the context of your work
ah so they anticipate what you need
yeah potentially
offering help or suggestions before you even ask
much more integrated interesting
okay what's next
then you get teammate agents
companies like Ravenna sale plane
base pilot are in this space
teammate agents so more autonomous
exactly they can
actually drive multi step work falls on their own
yeah it's like giving a complex task to an AI colleague
who just runs with it okay
so how does that differ from the next gen co pilot
is it just more independence
that seems to be the main thing
co pilots assist you within your workflow
teammate agents can own a whole process
start defending with less direct input
got it and there are more patterns
yep we're also seeing agent organizations
this is where you have multiple
specialized agents working together
like a team of AI's kind of ARRU's
probabilistic simulations were mentioned as an example
it's about coordination between different agents
to tackle something really complex
what's the benefit there is it just divide and conquer
pretty much division of labor
each agent is good at one thing
and they collaborate
almost like different departments in a company
to solve problems too big for any single agent
makes sense and the last pattern
agents as a service basically
companies offering specific AI
agent skills as a service for other developers
uh so like an API for a specific agent capability
exactly
instead of building say a web browsing agent yourself
you just use one
someone else provides plugging in expertise
okay that lets developers
focus on their main application logic right
not reinventing every AI wheel
precisely so all these patterns
co pilots teammates
organizations services
they're obviously
putting new demands on the underlying tech
which brings us back to the infrastructure stack
it seems to be shaking out into three main layers tools
data and orchestration why those three
why are they becoming the key pillars
well think about what you need to build these things
you need the tools to give the agents capabilities
letting them browse authenticate
use other software
you need data for their memory and context
and you need orchestration to manage them
especially when you have multiple agents
working together right
capabilities memory management
and the innovation within each layer is intense
right now
people are racing to solve the specific problems AI
agents create
which often aren't the same as traditional software
problem okay
let's break those down tools first
you said this has seen huge growth
yeah it feels like it's exploded
yeah
giving agents the ability to do things in the world
a big piece is browser infrastructure and UI automation
why is letting agents use websites so critical
well
just think about how much happens in a browser window
right for work
for personal stuff
if agents can't interact with those visual interfaces
they're locked out of a huge part of the digital world
they can't just rely on APIs for everything
good point APIs aren't always available or sufficient
and so we see companies like Browser Base Light Panda
Browserless building that core tech
and then things like stage hand on top
offering easier ways to code common interactions
so you have the low level plumbing
and the higher level components exactly
another critical tool area
hmm authentication and security
yeah this feels tricky definitely
if an agent is acting for a user
how do permissions work
the old username password model doesn't quite fit
what are the big security headaches here
well mainly
ensuring the agent only does what the user
actually allowed it to do
you need fine grained control
agent specific permission models right
you don't want your agent
go and row with your credentials
exactly so
companies like clerk and on statics
dot AI are jumping in
building this whole off for agents category makes sense
okay what else in the tools layer
tool discovery
how does an agent even find out what tools it can use
yeah that's huge
right now it seems a bit ad hoc
hmm but there's a lot of talk about Anthropics Model
Context Protocol MCP
people are calling it potentially the TCP I
P for AI agents whoa
t C P I P for agents that's a big claim
what would a standard like m C
P actually do the hope is it provides a common language
a standard way for agents to discover tools
understand how to use them
and pass context along reliably
and the fact that big names
Stripe Neo 4 j
Cloudflare are already setting up MCP servers
that gives us some weight that adoption is key
it could really kick start an ecosystem
where agents and tools just work together better
exactly but protocols can be low level
so we're also seeing abstraction layers built on top
like middleware sort of companies like Composio
they offer MCP
compatible ways to access things like Gmail
or linear
but through standard SDK's like typescript or Python
what's the advantage for developers there
oh that makes it way easier right
you don't need to understand the nitty gritty of MCP
itself you just use a familiar library or API call
faster development right
and Arcade Dev is doing something similar
but maybe broader
a unified API for off and tool management
even compatible with open AI stuff
so you get choices depending on how deep you want to go
yeah this layered approach seems smart
it caters to different needs
it's definitely better than the current situation
which feels like finding tools is mostly just asking
around word of mouth
uh huh yeah
these protocols and layers
could make it much more systematic
okay let's shift gears to the data layer
memory at scale
this really shows
how existing infrastructure has to change
Neon's experience again
agents creating databases 4 x faster
that creates totally new demands
doesn't it
what kind of pressures does that put on their systems
well you suddenly need instant database provisioning
you need it to scale up massively automatically
and you need strong isolation
so one agent's database doesn't mess with another's
think about that create dot X y
z example yeah
the developer agent spinning up 20,000 data bases
in 36 hours just based on user prompts
that demands incredible agility and scale
from the data platform mind boggling scale
and within this data layer
we're seeing specialization too right
like specific memory system
exactly things like Memzero or Zep
they're built specifically for agents to
remember stuff over time to recall context
how are they different from just using say
Redis or a regular database
well
traditional databases are usually about structured data
right transactions
these agent memory systems are often geared towards
maybe more unstructured info
and optimized for that kind of contextual recall
agents need for longer conversations or tasks
got it and then there's storage itself
we see traditional players like neon adapting
and vector databases like pinecone
becoming super important right
why are vector databases such a good fit here
because agents deal with so much unstructured stuff
text images etc
vector databases are great at finding similar things
in that kind of data semantic search
finding relevant context that's their superpower
it's crucial for making agents smart right
and we can't forget E P L extract transform load
new services are popping up just to handle
pulling in and cleaning up unstructured data
for these agent systems makes sense
garbage in garbage out still applies even for AI
especially for AI okay
final layer orchestration
managing the complexity yeah
once you start using multiple agents
maybe specialized ones talking to each other
it gets complicated fast
you need something to manage the whole flow
that's where tools like Lane Graph
Crew AI let it fit in managed orchestration exactly
they give you frameworks to design
and run these multi agent workflows
making sure they talk correctly
handle steps in order recover from errors
what are the main headaches
when you try to coordinate multiple agents
no lots
making sure Agent B waits for Agent a to finish
handling it if one agent fails
sharing information between them reliably
keeping track of the overall goal
orchestration tools try to hide some of that messiness
right and for long running processes
you need state persistence
which is where things like ingest hatchet
hemorrhoid come in
they make sure the system remembers where it was
even if a process takes hours or days
reliability for complex long running agent jobs
so looking back that analogy from a year ago
developers speeding across a half finished bridge
yeah I like that one
it felt accurate then and today
the bridge is definitely more built out
there's more structure but the traffic on the bridge
agent adoption is growing exponentially
good point so the construction isn't over
it has to keep pace with the ever increasing load
exactly which I think is the main point here
this rapid infrastructure evolution
it's not just incremental improvement
it feels like a fundamental
shift in how software gets built
a whole new paradigm emerging and this maturing stack
the tools data orchestration
the protocols like some damn CP
it's all scrambling to keep up to enable the shift
but even with all this progress
feels like we're still pretty early days right
oh absolutely
very early innings
the potential for new ideas and innovation
across all these layers tools
data orchestration
it's still massive huge opportunities remain
okay so let's quickly recap the main points
from our deep dive
we saw the crazy growth of companies using agents
the shift from just experimenting to using clear
repeatable patterns
and how the infrastructure is solidifying
around those three key layers tools
data and orchestration
and each one is buzzing with activity
and all this evolving
tech is paving the way for completely
new kinds of applications
and services we probably haven't even thought of yet
definitely
so maybe a final thought to leave everyone with
how will these big infrastructure shifts
actually change the apps and services we use every day
think beyond just code
what new things become possible in
say healthcare or education or entertainment
now that this foundation is getting stronger
it's a really exciting time to watch this space
absolutely fascinating
thanks for diving deep with us today