welcome to money is freedom
a podcast
exploring how finance and freedom connect in our lives
created from thoughtful research
and narrated by Notebook lmaai
this series brings you clear
meaningful insights into finance and beyond
welcome back to episode 51
today we're diving into one of the most comprehensive
looks at AI
implementation in the software industry that I've seen
this year
we're breaking down a confidential
2,025 report from Iconic Capital
that essentially serves as a roadmap for any company
serious about building AI
powered products
the pace of AI development
it feels absolutely dizzying sometimes
doesn't it oh definitely
every week something new exactly
new breakthroughs models capabilities
but you know for companies
the real challenge isn't just thinking up these cool AI
ideas it's making them real
making them work right
turning them into actual tangible
you know revenue driving products
so how are businesses actually doing that
building and operationalizing AI in this crazy
fast market that's the million dollar question
well this deep dive is hopefully
your shortcut to understanding that
how to
we're pulling back the curtain on the real strategies
the tactics companies are using right now
and our source for this is pretty cutting edge
it's the icon IQ 2025 State of AI report
and specifically
we're looking at the builder's playbook
yeah and this isn't just theory
it's based on a proprietary survey
they did back in April 2025
mm hmm who they talk to about 300 executives
mm hmm we're
talking architects engineers
product leaders
the people actually building AI
products at software companies
so our mission today is really to unpack
the core dimensions of this builder's playbook
for you okay
we'll get into you know
how they're thinking about product roadmaps
go to market strategies
but also the really crucial stuff like talent
managing costs and even how they're using AI
internally to boost their own productivity
so like a blueprint from the front lines
exactly straight from the people doing the work okay
building on that let's start with
who exactly are these builders they surveyed
and what kind of AI are we talking about here
right good question
so the survey included CEOs
heads of engineering AI leads
product leads mostly from North America
about 88%
and the report makes this important distinction
it splits companies into two main types
first you've got AI enabled companies
okay what does that mean
basically
they're adding AI features to existing products
or maybe creating some new AI
products that aren't like their core business
got it and the second type
those are the AI native companies hmm
for them AI is the core product
their whole business model is built
around generative intelligence
right from the start uh okay
AI native versus AI enabled
that makes sense
so here's where it gets really interesting
I think we always hear about the breakneck speed of AI
does this report actually confirm that these AI
native companies are moving faster like
through their product cycles
oh absolutely
the data shows a really stark difference
yeah so get this
About 47% of the AI native companies
their main AI product has already hit critical scale
and proven market fit wow
almost half yeah
now compare that to the AI enabled companies
Only 13% are at that stage 13
that's a huge gap it is
and what's maybe even more telling
only 1% of AI native companies are still pre launch
just 1% yeah
whereas for AI enabled it's 11%
so the AI
natives are much further along the development path
exactly it really suggests that these AI
native orgs might just be structurally
better set up for this maybe it's their teams
their processes their tech stack
they seem better equipped to validate ideas
and scale them effectively
they're kind of built for AI speed from day one
that definitely sounds like a significant
operational advantage so okay
across both these types of companies
what are the most common kinds of AI
products are actually building right now
well
the report points strongly towards agentic workflows
agentic workflows remind us what that means exactly
sure think of them as um
automated AI systems that can get
to make decisions and take actions on their own
to achieve a specific goal
they're more autonomous okay
goal oriented AI right
that and the application layer in general
are the most common things being built
but what's really noteworthy is that around 80%
8 0%
of the AI native companies
are currently building these agentic workflows
80% that's huge
it really shows
a strong trend towards these more complex
autonomous AI systems okay
that leads perfectly to the next question
how are they actually building these things
are they mostly using existing models
building their own from scratch
or is it more of a mix it's definitely a blend
but there's a clear lean towards
using what's already out there
okay a big majority
80% of companies building AI apps
are relying on third party AI APIs
you know calling out to open AI and Tropic
etcetera makes sense
it's faster right
but the picture changes a bit
when you look specifically
at the high growth companies
how so well
a larger chunk of them about 77%
are also fine tuning existing foundation models
and over half 54%
are even developing their own proprietary models
from scratch ah okay
so
the high growth players are doing more customization
and building their own tech
yeah which isn't too surprising right
they often have more resources
and maybe more importantly
they have that critical need for really deep
enterprise specific customization
that you might not get from an off the shelf API
that distinction
for high growth companies is really key
so
when companies are choosing these foundational models
especially for customer facing stuff
what's driving those decisions
what do they care about most accuracy
model accuracy is uh
hands down the top priority
74% ranked it in their top three
okay accuracy first
no surprise there no
but what is fascinating
in a noticeable shift this year is cost
cost ranked much higher 57% put it in their top three
really cost is becoming that important
yeah and it really echoes this
idea that the model layer itself is becoming well
somewhat commoditized if the
basic capabilities of the big models
are getting closer then cost
naturally becomes a much bigger factor in the decision
that makes total sense if they all perform well
price matters more what else is important
privacy is still a big one at 41%
and the ability to fine tune or customize the model
that's key for 34% okay
accuracy cost privacy customizability
got it and speaking of choices
who are the big players
which model providers are most popular right now
and is there like
a trend in how companies are mixing and matching models
well Openai's GPT models are still the dominant force
right used by 95 percent of the respondents
huge penetration 95% okay
but and this is crucial for strategy
many companies
are increasingly adopting a multi model approach
they're not just sticking to one
ah interesting
yeah they're leveraging different providers
different models probably
optimizing for different tasks within their products
on average respondents are using about 2.8 models
almost three different models on average
wow exactly
it suggests that strategic
flexibility is becoming the norm
you know maybe use one model for summarization
another for coding another for chat makes sense
besides open AI who else is commonly used
Anthropic's Claude models are up there
Google's Gemini and Meta's Lamae models too
those are the main ones mentioned frequently after GPT
okay so they choose their models
often more than one right
but how do they get them ready for prime time
what are the most common training techniques
they're using to hit that accuracy and performance bar
two main techniques really stand out
first is retrieval augmented generation or R I G
R I G right
that's where the model pulls in external data exactly
it retrieves relevant info from a knowledge base
before generating a response
super useful for grounding the AI
that's used by 66% overall and even slightly higher
69% for high growth companies
okay our rag is big
what's the other main one
fine tuning taking a pre trained model
and adapting it to your specific task or dataset
that's also really common
68% overall 67% for high growth
so R a G and fine tuning are the workhorses pretty much
although it's interesting
the high growth companies
also tend to use a wider variety of prompt
based techniques things like few shot learning
giving the model a few examples right
that's at 49% for them and even zero shot learning
where the model does the task with no examples
that's at 25%
it suggests they're doing more sophisticated
prompt engineering over there
okay and what about the underlying tech
the AI infrastructure
how are companies actually deploying these models
where are they running them
the vast majority are leaning
really heavily into fully managed AI solutions
meaning the cloud providers
yeah about 68% operate entirely in the cloud
and 64% are relying on those external AI
API providers we talked about
so they're outsourcing the infrastructure complexity
exactly it minimizes the upfront investment
you know buying servers and all that
it reduces the operational headache
and crucially it maximizes their speed to market
less time managing infrastructure
more time building the actual product features
makes sense what about hybrid or on prem
still niche really
hybrid models around 23% yeah
at running things fully on premise
that's fewer than one in 10 companies
the momentum is clearly towards cloud
and managed services okay
so they're pushing for rapid deployment
using cloud and APIs
but this raises an important question
what are the biggest hurdles they're hitting
when they actually try to put these AI
models into production what are the roadblocks
yeah the report highlights some key challenges
top of the list hallucinations
ah the AI making stuff up exactly
when the AI generates confident
but totally incorrect or nonsensical information
that's the biggest pain point for 39% of respondents
okay number one challenge
what's next
right behind it is explainability and trust at 38%
basically
understanding why the AI made a certain decision
and getting users to actually trust the output
that seems crucial especially for business applications
totally and the third biggest challenge is proving ROI
return on investment at 34%
showing that the AI
is actually delivering tangible business value
and it's worth noting
that explainability and trust issue
it ranked even higher for companies building AI
for specific verticals
especially in regulated industries
like finance or healthcare
transparency and compliance are non negotiable there
those are definitely persistent challenges
hallucinations trust ROI
it really drives home that
building AI isn't just about the tech
it's about managing its impact and proving its worth
absolutely so as these AI products get more mature
as they start scaling
how are companies thinking about performance monitoring
does that become more critical over time
oh definitely
the data shows a clear shift
when a product is pre launch
About 75% have no formal monitoring in place
wow three quarters are flying blind initially kinda
yeah but once those products start scaling
that number drops sharply
Only 31% of scaling products lack formal monitoring
okay
so monitoring becomes standard practice as you scale
right and you even
see a move towards more sophisticated approaches
About 12% of those scaling products
are already using fully automated model
monitoring and retraining pipelines
continuous optimization okay
that makes sense
you mentioned agentic workflows earlier
those autonomous AI systems as a big trend
especially for AI natives
where do those stand in terms of actual deployment
right now
are companies putting AI agents into production
or is it still mostly experimental
well a lot
of companies are definitely evaluating them
but there's a clear gap in actual deployment
ah the high growth AI companies are much further ahead
here About 47% of them are actively deploying
AI agents in production okay
almost half of the high growth ones
yeah compare that to all the other companies
only 23% are actively deploying agents
so high growth is definitely
leading the charge on putting these more advanced AI
systems to work got it
okay let's pivot a bit towards strategy now
thinking about the product roadmap
how much of a typical company's
roadmap is now focused on AI driven features
and how are they pricing this stuff right
so for the AI enabled companies
those adding AI onto existing products
they're dedicating around 20
35% of their product roadmap to AI features
20 to 35% okay
but again
the high growth companies are pushing that boundary
they're dedicating closer to 30
45% of their roadmap to AI
so AI is taking up a bigger chunk of development effort
especially for the fast movers
what about pricing how do you charge for AI
that's still evolving it seems
yeah many companies are using a hybrid model
they'll combine their standard
subscription or plan based pricing
with something extra
maybe usage based pricing for AI calls
or maybe outcome based pricing
interesting specifically for the AI features themselves
about 40% are bundling them into a premium tier product
so you'd pay more to get the AI goodness and
UPS exactly
and about a third
33% are including AI features at no extra cost
which
kind of reinforces this idea that AI is often seen as
a tiebreaker or upsell hook right now
rather than a standalone profit center just yet
so
use AI to win the deal or get them onto a higher plan
pretty much but it's definitely in flux
About 37% of respondents
said they're actively exploring totally
new pricing models
maybe based purely on consumption or ROI delivered
or different usage tiers so
the industry is
still figuring out the best way to capture
the value of AI directly
yeah it sounds like the monetization
playbook is still being written
okay with all this focus on AI products
what about the crucial areas like explainability
transparency compliance governance
how are companies handling these as their AI scales up
it definitely becomes more critical as you scale
providing detailed transparency
reports about the models or even just basic insights
into how the AI is influencing outcomes
that becomes really important
for building and maintaining
user trust makes sense
are they putting formal policies in place
most companies say they have some guardrails around AI
ethics and governance about two thirds
66% say they practice
basic compliance with data privacy laws
like GDPR or CCA okay
basic compliance
what about ensuring fairness and safety
a significant number
38% are using human in the loop oversight
basically
having people review or monitor the AI's decisions
to catch potential biases or errors
that seems to be a key practical step right now
so putting all this together
what does it mean for building the right team
the right organization around AI
are we seeing dedicated AI leadership emerge
what kind of talent are they hunting for
yeah
dedicated leadership is definitely becoming more common
especially as companies grow
between 50
and 61% of companies with over $100 million in revenue
have dedicated AI or ML leadership roles
okay so scale drives the need for specific leadership
seems like it
probably'cause the operational complexity
just ramps up significantly
in terms of specific roles they're hiring for
the most common are AML engineers
that's 88% of companies the core builders right
then data scientists at 72%
and AI product managers at 54%
those PMS who really understand AI are becoming crucial
and which roles are hardest to fill
interestingly the report says
ML engineers take the longest time to hire
on average about 70 days 70 days
wow
that really highlights how specialized and in demand
that talent is it really does
so given that demand
how are companies feeling about their hiring speed
for AI talent overall
are they keeping up
it's pretty much split down the middle
About 54% feel they are not hiring fast enough okay
just over half feel they're behind
and why is that the main reason cited
by 60% of those who feel they're behind
is simply a lack of qualified candidates
the talent crunch is real seems like it
the demand is outstripping the supply of experienced AI
professionals and looking at the bigger picture
within engineering teams
what slice of the pie are companies dedicating
specifically to AI work
on average companies are planning to have about 20
30% of their entire engineering team focused on AI
20 to 30% that's a significant chunk
it is and again
high growth companies are aiming even higher
they're targeting around 37% of their engineering team
focused on AI by 2026
37% that really shows a strategic commitment
doesn't it AI isn't just a side project
it's becoming a core part of engineering
exactly
it's moving from an add on to a fundamental discipline
okay
let's unpack the economics of this all this building
scaling hiring
it obviously comes with a significant cost
what are companies actually spending on AI development
and which costs are the trickiest to get a handle on
right the budget question
companies are allocating a pretty substantial slice
roughly 10 20% of their total R&D budget
specifically to AI development
10 to 20% of R&D wow
yeah
and most of them plan to increase that spend in 2025
so the investment is growing
what's interesting is how the cost mix changes
as AI products scale initially
talent might be a bigger proportion of the cost yeah
but as you scale
the cost of talent tends to decrease proportionally
while infrastructure and compute costs
really start to climb ah
so it shifts from
upfront people investment to ongoing operational costs
exactly and when you ask what's hardest to control
API usage fees come out on top
70% of respondents cited those as the most challenging
infrastructure costs 70%
why are ATI fees so hard to manage
probably the unpredictability
it's a variable cost
tied directly to how much your product is used
how many calls it makes to external models
it can fluctuate a lot
and be hard to forecast accurately
I can see that being a headache
what other costs are challenging
inference cost the actual cost of running the models
to generate predictions that's a big one too at 49%
and the cost of model retraining
and updates is right behind it
at 48%
keeping the models fresh and performing isn't cheap
so if those API usage
fees and inference costs are the big pain points
what are companies doing to try and get those costs
under control
what optimization strategies are they using
they're primarily exploring two main routes right now
first
about 41% are looking at moving towards open source
models to reduce dependency on paid APIs
exactly if you host your own open source model
you might have higher infrastructure costs
but you avoid those per call API fees
the second main strategy
cited by 37% is optimizing inference efficiency
making the models run faster and cheaper right
using techniques like quantization
pruning or deploying specialized hardware
to reduce the compute needed for each prediction
makes sense tackle
the API dependency
and make the models themselves cheaper to run
what about the specific costs for things like
say model training versus inference
as these products mature
do we have numbers on how those scale
we do and the scaling is pretty dramatic
for model training or fine tuning
most companies are doing this at least monthly
the estimated monthly cost for this
it starts around $163,000 for pre launch products
okay 163 k a month
even before launch yeah
but then it skyrockets for products that are scaling
that monthly training cost
jumps to an estimated 1.5 million dollars
wow from 163 k to 1.5 million dollars a month
that's almost a tenfold increase
it's huge hmm
and inference costs they surge right after launch
and here again
high growth companies are spending significantly more
how much more at the general availability GA stage
high growth companies are spending around $1.6 million
monthly on inference
compared to $1.0 million for their peers
and at the scaling stage
it's $2.3 million for high growth
versus $1.1 million for others
so double the inference cost for high growth companies
at scale roughly
yeah and it's not just compute
data storage and processing costs also climb steeply
after GA again
high growth companies are spending more there too
managing the data
becomes a bigger financial piece of the puzzle
as you scale okay
shifting gears slightly mm hmm
beyond building customer facing products
how are companies using AI internally
you know to boost their own productivity what
kind of investment are we seeing there
this is a really interesting area
the report shows that annual budgets for internal AI
productivity tools are set to nearly double in 2025
across all company sizes double
okay so internal AI is getting serious investment
how much are they spending relative to revenue
it varies but typically
somewhere between 1% and 8% of total company revenue
is being allocated to internal AI
productivity efforts
and where's that budget coming from still RND
RND is still the most common source at 59%
but interestingly
headcount budgets are also being tapped at 23%
headcount budgets so
they're potentially funding AI
tools instead of hiring more people
it certainly suggests
they're viewing these tools as a way to
enhance the productivity of their existing workforce
maybe offsetting the need for additional hires
in some areas it signals that internal AI
is moving beyond just R&D experiments
and becoming part of core operational strategy
that's a significant shift in mindset
but okay they're investing the money
what are the challenges
in actually getting employees to use these internal AI
tools effectively
what does adoption look like
yeah there's a bit of a gap there
yeah while a good majority
around 70% of employees
apparently have access to various AI tools internally
70% have access right
but only about half of them
50% are actually using these tools on an ongoing basis
only half are regular users
why the gap the report suggests adoption is proving
tougher in more mature larger enterprises
those over $1 billion in revenue
maybe it's ingrained habits
legacy systems change management challenges
could be and does the decision process for internal AI
tools differ from customer facing ones
yes significantly
for internal use cases
cost is the most important factor
cited by 74% cost over accuracy for internal tools
yep accuracy is still crucial
close behind at 72% and privacy is important too at 50%
but cost really leads the way internally
it's a different optimization equation
compared to external products
where accuracy is usually Paramount
interesting
and what are the biggest roadblocks to deploying these
internal tools is it tech hurdles
not primarily no
the biggest challenges seem to be more strategic
finding the right use cases is number one at 46%
figuring out where AI can genuinely make a difference
yeah and number two is proving the ROI at 42%
so it's less about can we build it
and more about should we build it
and how do we prove it's working exactly
it's about integrating AI
effectively into the daily workflow
and demonstrating its value
more than just the technical implementation
okay so if finding the right use cases is key
how many are companies exploring
and which internal AI applications
are actually delivering the biggest impact
right now there seems to be a link between exploration
and adoption companies that have high employee adoption
remember over 50% usage
they typically explore more use cases around 7 or more
internally compared to
compared to only about 4.6 use cases
companies with low adoption
so broader experimentation
seems to correlate with
more people actually using the tools
makes sense which departments are using AI most
R&D leads the way 77% are using AI there
sales and marketing is also high at 65%
general and administrative functions like finance or HR
lag a bit behind okay
R&D and sales marketing are the hot spots
and the killer app
which use case has the biggest impact
Wi Fi it's coding assistance
65% cite that is having a tangible impact
coding assistance like Github Code Pilot
exactly
respondents report an average productivity gain of 15
30% across these various Genie I use cases
uh huh but
coding assistance
seems to be the most consistently impactful one
15 to 30% gain that's substantial
it really is and again
high growth companies are leaning into this
more heavily they see
an average of 33% of their total code being written
with AI assistance
compared to 27% for other companies
that coding assistance impact is really impressive
what's the general attitude towards internal AI
especially among those leading high growth companies
are they fully embracing it
yes very much so
the report highlights that
high growth companies
tend to be much more active in experimenting with
and adopting new AI tools
92% of them are doing this 92% yeah
it strongly suggests they view internal AI
not just as a nice to have
but as a core strategic lever
they're moving
faster to integrate it across their own workflows
likely seeing a clear
competitive advantage in making their own teams
more efficient with AI right
it's like they're eating their own dog food
really leveraging AI internally as well
how are companies actually measuring the ROI
for this internal AI use what metrics are they tracking
they're primarily focused on productivity improvements
that's 75% and direct cost savings at 51% okay
efficiency and cost reduction
right and about 30% are tracking a mix of both
quantitative measures things like time saved
task completed faster and qualitative measures
like employee satisfaction surveys
or feedback on the tools
getting both the hard numbers and the user sentiment
okay this is super insightful
now here's where it gets really interesting
for the builders listening
if you're actually building this stuff
what are the specific
tools and technologies powering all this innovation
let's do a quick
deep dive into the AI text stack itself
absolutely let's break it down
for LLM and AI application development
kind of the core building blocks
tools like Langchain and Hugging
Face's toolset are really prominent for orchestration
orchestration meaning
managing the flow between models and data exactly
they act like the conductor of the AI orchestra
guardrails tools are also important for safety checks
making sure outputs are OK
and Versailes AI sk pops up too
used by 23% likely cause it helps deploy things quickly
then for model training and fine tuning the
heavy lifters are Pytorch and Tensorflow
they account for over half of all usage
these are the classic deep learning frameworks right
but you also see popular managed services like AWS
Sage Maker
and Openai's fine tuning API being used a lot
showing that mix of DIY control and managed convenience
for monitoring and observability
keeping an eye on how things are performing
nearly half of teams are actually adapting
their existing software
monitoring tools like Data Dog or honeycomb
using what they already have
yeah but specialized ML native platforms like Langsmith
and weights and biases are definitely gaining traction
hitting around 17% adoption
hmm suggest a growing need for AI specific monitoring
inference optimization
making models run efficiently
it's heavily dominated by Nvidia
their tensor RT inference server command
over 60% adoption
Nvidia is still the king of deployment
hardware and software
pretty much for non Nvidia solutions on
Nvidia runtime is a notable player at 18%
model hosting
most teams just hit the model provider APIs directly
Openai Anthropic etcetera
but the big cloud hyperscalers like AWS
Bedrock and Google
Vertex AI have carved out a big chunk of the market
especially for larger later stage companies
needing more integrated hosting solutions
model evaluation checking how good the models are
nearly a quarter
use built in features from their platforms
but purpose built tools like Langsmith and Langfuse
are leading the dedicated evaluation tool space
data processing and feature engineering
getting the data ready it's the classic
big data tools a patchy spark at 44%
Kafka for streaming at 42%
and pandas is still indispensable for Python
data manipulation at 41%
the workhorses of data engineering
exactly vector databases
crucial for Apache
Elastic and Pinecone are leading the pack
showing a mix of established players
adapting and purpose built solutions
open source options are gaining ground too
synthetic data and data augmentation
so it's interesting over half
52% are building their own custom tooling for this
building their own synthetic data tools
yeah shows how specific data needs can be scale
AI is the clear third party leader here though at 21%
coding assistance we talked about its impact
Github Copilot is the giant
used by almost three quarters of dev teams 75%
cursor is a strong second at 50%
copilot is everywhere
almost Devops and Mlops managing the whole life cycle
ML flow is the clear front runner at 36%
followed by weights and biases at 20%
but the market here still feels pretty fragmented
lots of tools product and design
it's basically Figma 87% use it for UI UX
Miro's popular too at 37%
for more high level whiteboarding and collaboration
and finally
those internal productivity tools we discussed
you see AI features being baked into the platform
teams already use daily Salesforce for sales
Canva for marketing Zendesk for support
notion for knowledge management
service now for it LinkedIn for HR
AI is becoming embedded everywhere
wow okay
this deep dive has really given us an incredible
real world
look into how companies are actually doing it building
scaling managing AI products
and just as importantly
how they're weaving AI into their own
internal operations
it really feels like a view from the trenches
I think so too and it's more than just
you know interesting facts
it feels like a practical blueprint
based on what hundreds of executives
are actually experiencing
and doing right now yeah
this data offers some really actionable insights
I think
for anyone trying to navigate AI development because
you know knowledge is great
but it's most valuable
when you can actually understand it
and apply it
this report gives a pretty solid roadmap for that
definitely so as we wrap up here
thinking back over everything we've unpacked from this
AI builder's playbook hmm
what's the one big takeaway
that really stands out to you
hmm it's a tough one
I think for me it's the growing importance
almost the necessity now of that multi model strategy
combined with rigorous cost management
especially around inference
it shows a real maturation
moving from just can we make it work
hmm to how do we make it work efficiently
reliably and affordably at scale
that feels like the current frontier
that's a great point
the operational reality is setting in
and maybe a final thought for everyone
listening to Molover we saw that high growth AI
companies are already dedicating
way more engineering resources to AI
they're spending significantly more on core costs
like inference and data
and they're actively experimenting with all these new
internal tools so the question is
what fundamental shifts do other companies need to make
in their talent strategy
their budgets
maybe even their company culture around AI
to genuinely close that gap and compete effectively
in this landscape
that's just evolving so incredibly fast