1 00:00:00,100 --> 00:00:01,966 welcome to money is freedom 2 00:00:01,966 --> 00:00:02,866 a podcast 3 00:00:02,866 --> 00:00:06,066 exploring how finance and freedom connect in our lives 4 00:00:07,033 --> 00:00:08,833 created from thoughtful research 5 00:00:08,833 --> 00:00:11,300 and narrated by Notebook lmaai 6 00:00:11,400 --> 00:00:13,266 this series brings you clear 7 00:00:13,266 --> 00:00:15,966 meaningful insights into finance and beyond 8 00:00:16,066 --> 00:00:18,366 welcome back to episode 51 9 00:00:19,266 --> 00:00:22,166 today we're diving into one of the most comprehensive 10 00:00:22,166 --> 00:00:23,033 looks at AI 11 00:00:23,033 --> 00:00:25,800 implementation in the software industry that I've seen 12 00:00:25,800 --> 00:00:26,666 this year 13 00:00:27,366 --> 00:00:29,366 we're breaking down a confidential 14 00:00:29,366 --> 00:00:32,633 2,025 report from Iconic Capital 15 00:00:32,633 --> 00:00:35,600 that essentially serves as a roadmap for any company 16 00:00:35,600 --> 00:00:37,100 serious about building AI 17 00:00:37,100 --> 00:00:38,166 powered products 18 00:00:38,266 --> 00:00:40,100 the pace of AI development 19 00:00:40,100 --> 00:00:42,066 it feels absolutely dizzying sometimes 20 00:00:42,066 --> 00:00:43,200 doesn't it oh definitely 21 00:00:43,200 --> 00:00:45,000 every week something new exactly 22 00:00:45,000 --> 00:00:47,800 new breakthroughs models capabilities 23 00:00:47,900 --> 00:00:49,500 but you know for companies 24 00:00:49,500 --> 00:00:52,866 the real challenge isn't just thinking up these cool AI 25 00:00:52,866 --> 00:00:54,366 ideas it's making them real 26 00:00:54,366 --> 00:00:55,666 making them work right 27 00:00:55,666 --> 00:00:57,700 turning them into actual tangible 28 00:00:57,700 --> 00:00:59,766 you know revenue driving products 29 00:00:59,966 --> 00:01:01,966 so how are businesses actually doing that 30 00:01:01,966 --> 00:01:05,266 building and operationalizing AI in this crazy 31 00:01:05,266 --> 00:01:07,800 fast market that's the million dollar question 32 00:01:07,800 --> 00:01:09,866 well this deep dive is hopefully 33 00:01:09,866 --> 00:01:11,766 your shortcut to understanding that 34 00:01:11,766 --> 00:01:12,666 how to 35 00:01:12,833 --> 00:01:15,466 we're pulling back the curtain on the real strategies 36 00:01:15,466 --> 00:01:17,666 the tactics companies are using right now 37 00:01:17,666 --> 00:01:19,433 and our source for this is pretty cutting edge 38 00:01:19,433 --> 00:01:23,066 it's the icon IQ 2025 State of AI report 39 00:01:23,100 --> 00:01:24,033 and specifically 40 00:01:24,033 --> 00:01:26,066 we're looking at the builder's playbook 41 00:01:26,066 --> 00:01:27,900 yeah and this isn't just theory 42 00:01:27,900 --> 00:01:29,666 it's based on a proprietary survey 43 00:01:29,666 --> 00:01:32,000 they did back in April 2025 44 00:01:32,000 --> 00:01:34,700 mm hmm who they talk to about 300 executives 45 00:01:34,700 --> 00:01:34,900 mm hmm we're 46 00:01:34,900 --> 00:01:36,800 talking architects engineers 47 00:01:37,100 --> 00:01:38,133 product leaders 48 00:01:38,800 --> 00:01:40,400 the people actually building AI 49 00:01:40,400 --> 00:01:42,000 products at software companies 50 00:01:43,466 --> 00:01:45,366 so our mission today is really to unpack 51 00:01:45,366 --> 00:01:47,233 the core dimensions of this builder's playbook 52 00:01:47,233 --> 00:01:48,000 for you okay 53 00:01:48,000 --> 00:01:49,400 we'll get into you know 54 00:01:49,400 --> 00:01:51,166 how they're thinking about product roadmaps 55 00:01:51,166 --> 00:01:52,466 go to market strategies 56 00:01:52,600 --> 00:01:55,100 but also the really crucial stuff like talent 57 00:01:55,300 --> 00:01:58,600 managing costs and even how they're using AI 58 00:01:58,600 --> 00:02:01,200 internally to boost their own productivity 59 00:02:01,200 --> 00:02:03,000 so like a blueprint from the front lines 60 00:02:03,000 --> 00:02:05,500 exactly straight from the people doing the work okay 61 00:02:05,500 --> 00:02:07,266 building on that let's start with 62 00:02:07,266 --> 00:02:10,266 who exactly are these builders they surveyed 63 00:02:10,266 --> 00:02:12,466 and what kind of AI are we talking about here 64 00:02:12,466 --> 00:02:13,533 right good question 65 00:02:13,566 --> 00:02:16,000 so the survey included CEOs 66 00:02:16,200 --> 00:02:18,166 heads of engineering AI leads 67 00:02:18,166 --> 00:02:20,366 product leads mostly from North America 68 00:02:20,366 --> 00:02:21,666 about 88% 69 00:02:21,833 --> 00:02:24,233 and the report makes this important distinction 70 00:02:24,233 --> 00:02:26,566 it splits companies into two main types 71 00:02:26,866 --> 00:02:29,200 first you've got AI enabled companies 72 00:02:29,200 --> 00:02:30,400 okay what does that mean 73 00:02:30,400 --> 00:02:30,833 basically 74 00:02:30,833 --> 00:02:33,033 they're adding AI features to existing products 75 00:02:33,033 --> 00:02:34,400 or maybe creating some new AI 76 00:02:34,400 --> 00:02:37,233 products that aren't like their core business 77 00:02:37,233 --> 00:02:38,566 got it and the second type 78 00:02:38,566 --> 00:02:40,566 those are the AI native companies hmm 79 00:02:40,566 --> 00:02:42,800 for them AI is the core product 80 00:02:43,033 --> 00:02:44,400 their whole business model is built 81 00:02:44,400 --> 00:02:45,866 around generative intelligence 82 00:02:45,866 --> 00:02:47,966 right from the start uh okay 83 00:02:47,966 --> 00:02:50,066 AI native versus AI enabled 84 00:02:50,066 --> 00:02:51,133 that makes sense 85 00:02:51,666 --> 00:02:53,033 so here's where it gets really interesting 86 00:02:53,033 --> 00:02:56,433 I think we always hear about the breakneck speed of AI 87 00:02:56,433 --> 00:02:58,833 does this report actually confirm that these AI 88 00:02:58,833 --> 00:03:01,166 native companies are moving faster like 89 00:03:01,166 --> 00:03:02,266 through their product cycles 90 00:03:02,266 --> 00:03:02,800 oh absolutely 91 00:03:02,800 --> 00:03:04,666 the data shows a really stark difference 92 00:03:04,666 --> 00:03:06,033 yeah so get this 93 00:03:06,033 --> 00:03:09,333 About 47% of the AI native companies 94 00:03:09,766 --> 00:03:12,966 their main AI product has already hit critical scale 95 00:03:12,966 --> 00:03:14,800 and proven market fit wow 96 00:03:14,800 --> 00:03:16,066 almost half yeah 97 00:03:16,066 --> 00:03:18,266 now compare that to the AI enabled companies 98 00:03:18,266 --> 00:03:21,266 Only 13% are at that stage 13 99 00:03:21,266 --> 00:03:23,000 that's a huge gap it is 100 00:03:23,000 --> 00:03:24,566 and what's maybe even more telling 101 00:03:24,566 --> 00:03:27,800 only 1% of AI native companies are still pre launch 102 00:03:27,800 --> 00:03:29,000 just 1% yeah 103 00:03:29,000 --> 00:03:31,266 whereas for AI enabled it's 11% 104 00:03:31,266 --> 00:03:31,833 so the AI 105 00:03:31,833 --> 00:03:34,366 natives are much further along the development path 106 00:03:34,433 --> 00:03:36,700 exactly it really suggests that these AI 107 00:03:36,700 --> 00:03:38,300 native orgs might just be structurally 108 00:03:38,300 --> 00:03:40,966 better set up for this maybe it's their teams 109 00:03:40,966 --> 00:03:42,966 their processes their tech stack 110 00:03:43,233 --> 00:03:45,600 they seem better equipped to validate ideas 111 00:03:45,600 --> 00:03:46,966 and scale them effectively 112 00:03:47,100 --> 00:03:49,500 they're kind of built for AI speed from day one 113 00:03:49,500 --> 00:03:51,266 that definitely sounds like a significant 114 00:03:51,266 --> 00:03:53,800 operational advantage so okay 115 00:03:53,800 --> 00:03:55,433 across both these types of companies 116 00:03:55,433 --> 00:03:57,666 what are the most common kinds of AI 117 00:03:57,666 --> 00:03:59,433 products are actually building right now 118 00:03:59,433 --> 00:04:00,166 well 119 00:04:00,166 --> 00:04:03,066 the report points strongly towards agentic workflows 120 00:04:03,066 --> 00:04:05,500 agentic workflows remind us what that means exactly 121 00:04:05,500 --> 00:04:07,866 sure think of them as um 122 00:04:08,300 --> 00:04:10,200 automated AI systems that can get 123 00:04:10,200 --> 00:04:12,700 to make decisions and take actions on their own 124 00:04:12,700 --> 00:04:14,000 to achieve a specific goal 125 00:04:14,000 --> 00:04:15,400 they're more autonomous okay 126 00:04:15,400 --> 00:04:17,500 goal oriented AI right 127 00:04:17,500 --> 00:04:19,600 that and the application layer in general 128 00:04:19,733 --> 00:04:21,366 are the most common things being built 129 00:04:21,466 --> 00:04:24,400 but what's really noteworthy is that around 80% 130 00:04:24,400 --> 00:04:25,900 8 0% 131 00:04:25,900 --> 00:04:27,266 of the AI native companies 132 00:04:27,266 --> 00:04:29,266 are currently building these agentic workflows 133 00:04:29,266 --> 00:04:31,100 80% that's huge 134 00:04:31,100 --> 00:04:31,700 it really shows 135 00:04:31,700 --> 00:04:34,166 a strong trend towards these more complex 136 00:04:34,166 --> 00:04:36,233 autonomous AI systems okay 137 00:04:36,233 --> 00:04:37,900 that leads perfectly to the next question 138 00:04:38,400 --> 00:04:40,633 how are they actually building these things 139 00:04:40,633 --> 00:04:42,566 are they mostly using existing models 140 00:04:42,566 --> 00:04:43,966 building their own from scratch 141 00:04:43,966 --> 00:04:46,933 or is it more of a mix it's definitely a blend 142 00:04:46,966 --> 00:04:49,033 but there's a clear lean towards 143 00:04:49,033 --> 00:04:50,300 using what's already out there 144 00:04:50,300 --> 00:04:52,033 okay a big majority 145 00:04:52,033 --> 00:04:54,300 80% of companies building AI apps 146 00:04:54,300 --> 00:04:56,966 are relying on third party AI APIs 147 00:04:57,266 --> 00:04:59,633 you know calling out to open AI and Tropic 148 00:04:59,633 --> 00:05:00,466 etcetera makes sense 149 00:05:00,466 --> 00:05:01,800 it's faster right 150 00:05:01,800 --> 00:05:03,900 but the picture changes a bit 151 00:05:03,900 --> 00:05:04,833 when you look specifically 152 00:05:04,833 --> 00:05:05,866 at the high growth companies 153 00:05:05,866 --> 00:05:07,166 how so well 154 00:05:07,166 --> 00:05:09,666 a larger chunk of them about 77% 155 00:05:09,666 --> 00:05:12,466 are also fine tuning existing foundation models 156 00:05:12,833 --> 00:05:14,566 and over half 54% 157 00:05:14,566 --> 00:05:17,033 are even developing their own proprietary models 158 00:05:17,033 --> 00:05:18,866 from scratch ah okay 159 00:05:18,866 --> 00:05:19,100 so 160 00:05:19,100 --> 00:05:21,766 the high growth players are doing more customization 161 00:05:21,766 --> 00:05:22,966 and building their own tech 162 00:05:22,966 --> 00:05:25,833 yeah which isn't too surprising right 163 00:05:25,833 --> 00:05:27,200 they often have more resources 164 00:05:27,200 --> 00:05:28,266 and maybe more importantly 165 00:05:28,266 --> 00:05:31,000 they have that critical need for really deep 166 00:05:31,000 --> 00:05:33,200 enterprise specific customization 167 00:05:33,300 --> 00:05:35,966 that you might not get from an off the shelf API 168 00:05:36,066 --> 00:05:37,000 that distinction 169 00:05:37,000 --> 00:05:39,000 for high growth companies is really key 170 00:05:39,233 --> 00:05:39,800 so 171 00:05:39,800 --> 00:05:42,400 when companies are choosing these foundational models 172 00:05:42,400 --> 00:05:44,866 especially for customer facing stuff 173 00:05:45,000 --> 00:05:46,633 what's driving those decisions 174 00:05:46,633 --> 00:05:49,100 what do they care about most accuracy 175 00:05:49,100 --> 00:05:51,266 model accuracy is uh 176 00:05:51,266 --> 00:05:52,733 hands down the top priority 177 00:05:52,800 --> 00:05:54,900 74% ranked it in their top three 178 00:05:54,900 --> 00:05:56,266 okay accuracy first 179 00:05:56,266 --> 00:05:57,600 no surprise there no 180 00:05:57,600 --> 00:05:59,100 but what is fascinating 181 00:05:59,100 --> 00:06:01,866 in a noticeable shift this year is cost 182 00:06:02,166 --> 00:06:06,000 cost ranked much higher 57% put it in their top three 183 00:06:06,000 --> 00:06:08,000 really cost is becoming that important 184 00:06:08,000 --> 00:06:09,466 yeah and it really echoes this 185 00:06:09,466 --> 00:06:12,133 idea that the model layer itself is becoming well 186 00:06:12,233 --> 00:06:14,300 somewhat commoditized if the 187 00:06:14,300 --> 00:06:16,066 basic capabilities of the big models 188 00:06:16,066 --> 00:06:18,166 are getting closer then cost 189 00:06:18,166 --> 00:06:20,466 naturally becomes a much bigger factor in the decision 190 00:06:20,466 --> 00:06:22,466 that makes total sense if they all perform well 191 00:06:22,466 --> 00:06:24,266 price matters more what else is important 192 00:06:24,266 --> 00:06:26,466 privacy is still a big one at 41% 193 00:06:26,466 --> 00:06:28,766 and the ability to fine tune or customize the model 194 00:06:28,766 --> 00:06:30,833 that's key for 34% okay 195 00:06:30,833 --> 00:06:33,100 accuracy cost privacy customizability 196 00:06:33,266 --> 00:06:35,300 got it and speaking of choices 197 00:06:35,300 --> 00:06:36,733 who are the big players 198 00:06:36,866 --> 00:06:39,000 which model providers are most popular right now 199 00:06:39,000 --> 00:06:39,700 and is there like 200 00:06:39,700 --> 00:06:42,866 a trend in how companies are mixing and matching models 201 00:06:42,866 --> 00:06:46,500 well Openai's GPT models are still the dominant force 202 00:06:46,500 --> 00:06:48,800 right used by 95 percent of the respondents 203 00:06:49,166 --> 00:06:51,400 huge penetration 95% okay 204 00:06:51,400 --> 00:06:53,733 but and this is crucial for strategy 205 00:06:53,966 --> 00:06:54,766 many companies 206 00:06:54,766 --> 00:06:57,400 are increasingly adopting a multi model approach 207 00:06:57,400 --> 00:06:58,700 they're not just sticking to one 208 00:06:58,700 --> 00:06:59,700 ah interesting 209 00:06:59,700 --> 00:07:01,233 yeah they're leveraging different providers 210 00:07:01,233 --> 00:07:02,633 different models probably 211 00:07:02,633 --> 00:07:05,300 optimizing for different tasks within their products 212 00:07:05,466 --> 00:07:08,300 on average respondents are using about 2.8 models 213 00:07:08,300 --> 00:07:10,266 almost three different models on average 214 00:07:10,266 --> 00:07:11,533 wow exactly 215 00:07:11,600 --> 00:07:12,800 it suggests that strategic 216 00:07:12,800 --> 00:07:14,466 flexibility is becoming the norm 217 00:07:14,466 --> 00:07:17,166 you know maybe use one model for summarization 218 00:07:17,366 --> 00:07:20,900 another for coding another for chat makes sense 219 00:07:20,900 --> 00:07:23,033 besides open AI who else is commonly used 220 00:07:23,033 --> 00:07:24,833 Anthropic's Claude models are up there 221 00:07:24,833 --> 00:07:27,700 Google's Gemini and Meta's Lamae models too 222 00:07:27,700 --> 00:07:30,100 those are the main ones mentioned frequently after GPT 223 00:07:30,100 --> 00:07:31,500 okay so they choose their models 224 00:07:31,500 --> 00:07:32,866 often more than one right 225 00:07:32,866 --> 00:07:34,866 but how do they get them ready for prime time 226 00:07:35,066 --> 00:07:36,966 what are the most common training techniques 227 00:07:36,966 --> 00:07:40,533 they're using to hit that accuracy and performance bar 228 00:07:40,600 --> 00:07:42,433 two main techniques really stand out 229 00:07:42,433 --> 00:07:46,300 first is retrieval augmented generation or R I G 230 00:07:46,433 --> 00:07:47,100 R I G right 231 00:07:47,100 --> 00:07:50,000 that's where the model pulls in external data exactly 232 00:07:50,000 --> 00:07:52,266 it retrieves relevant info from a knowledge base 233 00:07:52,266 --> 00:07:54,000 before generating a response 234 00:07:54,366 --> 00:07:56,466 super useful for grounding the AI 235 00:07:56,466 --> 00:08:00,000 that's used by 66% overall and even slightly higher 236 00:08:00,100 --> 00:08:02,233 69% for high growth companies 237 00:08:02,233 --> 00:08:03,566 okay our rag is big 238 00:08:03,566 --> 00:08:04,466 what's the other main one 239 00:08:04,466 --> 00:08:06,800 fine tuning taking a pre trained model 240 00:08:06,800 --> 00:08:09,566 and adapting it to your specific task or dataset 241 00:08:09,600 --> 00:08:10,933 that's also really common 242 00:08:11,033 --> 00:08:14,466 68% overall 67% for high growth 243 00:08:14,466 --> 00:08:19,066 so R a G and fine tuning are the workhorses pretty much 244 00:08:19,066 --> 00:08:19,866 although it's interesting 245 00:08:19,866 --> 00:08:20,833 the high growth companies 246 00:08:20,833 --> 00:08:22,966 also tend to use a wider variety of prompt 247 00:08:22,966 --> 00:08:25,933 based techniques things like few shot learning 248 00:08:25,933 --> 00:08:27,466 giving the model a few examples right 249 00:08:27,466 --> 00:08:30,433 that's at 49% for them and even zero shot learning 250 00:08:30,433 --> 00:08:32,600 where the model does the task with no examples 251 00:08:32,600 --> 00:08:33,833 that's at 25% 252 00:08:33,833 --> 00:08:35,500 it suggests they're doing more sophisticated 253 00:08:35,500 --> 00:08:36,700 prompt engineering over there 254 00:08:36,700 --> 00:08:38,900 okay and what about the underlying tech 255 00:08:38,900 --> 00:08:40,400 the AI infrastructure 256 00:08:40,400 --> 00:08:42,766 how are companies actually deploying these models 257 00:08:42,766 --> 00:08:43,966 where are they running them 258 00:08:43,966 --> 00:08:45,300 the vast majority are leaning 259 00:08:45,300 --> 00:08:48,400 really heavily into fully managed AI solutions 260 00:08:48,766 --> 00:08:50,300 meaning the cloud providers 261 00:08:50,300 --> 00:08:53,666 yeah about 68% operate entirely in the cloud 262 00:08:54,100 --> 00:08:57,366 and 64% are relying on those external AI 263 00:08:57,366 --> 00:08:58,800 API providers we talked about 264 00:08:58,800 --> 00:09:01,200 so they're outsourcing the infrastructure complexity 265 00:09:01,200 --> 00:09:04,000 exactly it minimizes the upfront investment 266 00:09:04,000 --> 00:09:05,666 you know buying servers and all that 267 00:09:05,666 --> 00:09:07,400 it reduces the operational headache 268 00:09:07,400 --> 00:09:10,666 and crucially it maximizes their speed to market 269 00:09:11,166 --> 00:09:12,866 less time managing infrastructure 270 00:09:12,866 --> 00:09:15,300 more time building the actual product features 271 00:09:15,300 --> 00:09:18,333 makes sense what about hybrid or on prem 272 00:09:18,366 --> 00:09:19,400 still niche really 273 00:09:19,400 --> 00:09:21,766 hybrid models around 23% yeah 274 00:09:21,766 --> 00:09:23,766 at running things fully on premise 275 00:09:23,900 --> 00:09:25,733 that's fewer than one in 10 companies 276 00:09:25,766 --> 00:09:27,666 the momentum is clearly towards cloud 277 00:09:27,666 --> 00:09:29,066 and managed services okay 278 00:09:29,066 --> 00:09:30,866 so they're pushing for rapid deployment 279 00:09:30,866 --> 00:09:32,300 using cloud and APIs 280 00:09:32,300 --> 00:09:34,100 but this raises an important question 281 00:09:34,100 --> 00:09:35,800 what are the biggest hurdles they're hitting 282 00:09:35,800 --> 00:09:37,366 when they actually try to put these AI 283 00:09:37,366 --> 00:09:39,833 models into production what are the roadblocks 284 00:09:39,833 --> 00:09:42,000 yeah the report highlights some key challenges 285 00:09:42,033 --> 00:09:44,000 top of the list hallucinations 286 00:09:44,066 --> 00:09:46,633 ah the AI making stuff up exactly 287 00:09:46,633 --> 00:09:48,266 when the AI generates confident 288 00:09:48,266 --> 00:09:50,866 but totally incorrect or nonsensical information 289 00:09:50,866 --> 00:09:53,500 that's the biggest pain point for 39% of respondents 290 00:09:53,500 --> 00:09:54,966 okay number one challenge 291 00:09:54,966 --> 00:09:55,866 what's next 292 00:09:55,866 --> 00:09:59,766 right behind it is explainability and trust at 38% 293 00:10:00,166 --> 00:10:00,766 basically 294 00:10:00,766 --> 00:10:03,400 understanding why the AI made a certain decision 295 00:10:03,466 --> 00:10:05,966 and getting users to actually trust the output 296 00:10:05,966 --> 00:10:08,400 that seems crucial especially for business applications 297 00:10:08,600 --> 00:10:11,666 totally and the third biggest challenge is proving ROI 298 00:10:11,666 --> 00:10:14,000 return on investment at 34% 299 00:10:14,100 --> 00:10:15,166 showing that the AI 300 00:10:15,166 --> 00:10:17,666 is actually delivering tangible business value 301 00:10:17,700 --> 00:10:18,800 and it's worth noting 302 00:10:18,800 --> 00:10:20,766 that explainability and trust issue 303 00:10:20,966 --> 00:10:23,366 it ranked even higher for companies building AI 304 00:10:23,366 --> 00:10:24,733 for specific verticals 305 00:10:24,900 --> 00:10:26,633 especially in regulated industries 306 00:10:26,633 --> 00:10:28,166 like finance or healthcare 307 00:10:28,433 --> 00:10:30,866 transparency and compliance are non negotiable there 308 00:10:30,866 --> 00:10:32,666 those are definitely persistent challenges 309 00:10:32,666 --> 00:10:34,800 hallucinations trust ROI 310 00:10:34,800 --> 00:10:35,866 it really drives home that 311 00:10:35,866 --> 00:10:37,766 building AI isn't just about the tech 312 00:10:37,766 --> 00:10:40,400 it's about managing its impact and proving its worth 313 00:10:40,400 --> 00:10:43,066 absolutely so as these AI products get more mature 314 00:10:43,066 --> 00:10:44,200 as they start scaling 315 00:10:44,200 --> 00:10:46,800 how are companies thinking about performance monitoring 316 00:10:46,800 --> 00:10:48,700 does that become more critical over time 317 00:10:48,700 --> 00:10:49,500 oh definitely 318 00:10:49,566 --> 00:10:51,266 the data shows a clear shift 319 00:10:51,300 --> 00:10:52,900 when a product is pre launch 320 00:10:52,900 --> 00:10:56,366 About 75% have no formal monitoring in place 321 00:10:56,366 --> 00:10:59,366 wow three quarters are flying blind initially kinda 322 00:10:59,366 --> 00:11:02,266 yeah but once those products start scaling 323 00:11:02,666 --> 00:11:04,366 that number drops sharply 324 00:11:04,433 --> 00:11:08,166 Only 31% of scaling products lack formal monitoring 325 00:11:08,166 --> 00:11:08,400 okay 326 00:11:08,400 --> 00:11:10,966 so monitoring becomes standard practice as you scale 327 00:11:10,966 --> 00:11:12,700 right and you even 328 00:11:12,766 --> 00:11:15,200 see a move towards more sophisticated approaches 329 00:11:15,433 --> 00:11:17,866 About 12% of those scaling products 330 00:11:17,966 --> 00:11:20,633 are already using fully automated model 331 00:11:20,633 --> 00:11:22,600 monitoring and retraining pipelines 332 00:11:23,033 --> 00:11:24,566 continuous optimization okay 333 00:11:24,566 --> 00:11:25,300 that makes sense 334 00:11:25,300 --> 00:11:27,833 you mentioned agentic workflows earlier 335 00:11:27,833 --> 00:11:30,233 those autonomous AI systems as a big trend 336 00:11:30,233 --> 00:11:31,800 especially for AI natives 337 00:11:32,100 --> 00:11:34,666 where do those stand in terms of actual deployment 338 00:11:34,666 --> 00:11:35,066 right now 339 00:11:35,066 --> 00:11:37,400 are companies putting AI agents into production 340 00:11:37,400 --> 00:11:39,666 or is it still mostly experimental 341 00:11:39,666 --> 00:11:40,266 well a lot 342 00:11:40,266 --> 00:11:42,300 of companies are definitely evaluating them 343 00:11:42,300 --> 00:11:44,666 but there's a clear gap in actual deployment 344 00:11:44,666 --> 00:11:47,400 ah the high growth AI companies are much further ahead 345 00:11:47,400 --> 00:11:49,966 here About 47% of them are actively deploying 346 00:11:49,966 --> 00:11:51,500 AI agents in production okay 347 00:11:51,500 --> 00:11:52,766 almost half of the high growth ones 348 00:11:52,766 --> 00:11:54,500 yeah compare that to all the other companies 349 00:11:54,500 --> 00:11:57,066 only 23% are actively deploying agents 350 00:11:57,400 --> 00:11:58,666 so high growth is definitely 351 00:11:58,666 --> 00:12:01,033 leading the charge on putting these more advanced AI 352 00:12:01,033 --> 00:12:02,966 systems to work got it 353 00:12:03,233 --> 00:12:06,000 okay let's pivot a bit towards strategy now 354 00:12:06,266 --> 00:12:08,500 thinking about the product roadmap 355 00:12:09,166 --> 00:12:10,700 how much of a typical company's 356 00:12:10,700 --> 00:12:14,266 roadmap is now focused on AI driven features 357 00:12:14,266 --> 00:12:16,300 and how are they pricing this stuff right 358 00:12:16,300 --> 00:12:17,966 so for the AI enabled companies 359 00:12:17,966 --> 00:12:20,133 those adding AI onto existing products 360 00:12:20,166 --> 00:12:21,433 they're dedicating around 20 361 00:12:21,433 --> 00:12:24,966 35% of their product roadmap to AI features 362 00:12:24,966 --> 00:12:26,566 20 to 35% okay 363 00:12:26,566 --> 00:12:27,033 but again 364 00:12:27,033 --> 00:12:29,266 the high growth companies are pushing that boundary 365 00:12:29,266 --> 00:12:30,766 they're dedicating closer to 30 366 00:12:30,766 --> 00:12:33,100 45% of their roadmap to AI 367 00:12:33,100 --> 00:12:36,400 so AI is taking up a bigger chunk of development effort 368 00:12:36,400 --> 00:12:38,100 especially for the fast movers 369 00:12:38,200 --> 00:12:41,066 what about pricing how do you charge for AI 370 00:12:41,066 --> 00:12:42,466 that's still evolving it seems 371 00:12:42,466 --> 00:12:44,800 yeah many companies are using a hybrid model 372 00:12:44,900 --> 00:12:46,033 they'll combine their standard 373 00:12:46,033 --> 00:12:47,966 subscription or plan based pricing 374 00:12:48,033 --> 00:12:49,166 with something extra 375 00:12:49,166 --> 00:12:51,466 maybe usage based pricing for AI calls 376 00:12:51,466 --> 00:12:53,200 or maybe outcome based pricing 377 00:12:53,200 --> 00:12:55,500 interesting specifically for the AI features themselves 378 00:12:55,500 --> 00:12:59,166 about 40% are bundling them into a premium tier product 379 00:12:59,433 --> 00:13:01,866 so you'd pay more to get the AI goodness and 380 00:13:01,866 --> 00:13:02,900 UPS exactly 381 00:13:02,900 --> 00:13:03,666 and about a third 382 00:13:03,666 --> 00:13:07,566 33% are including AI features at no extra cost 383 00:13:07,633 --> 00:13:08,200 which 384 00:13:08,200 --> 00:13:11,566 kind of reinforces this idea that AI is often seen as 385 00:13:11,600 --> 00:13:13,866 a tiebreaker or upsell hook right now 386 00:13:13,900 --> 00:13:16,500 rather than a standalone profit center just yet 387 00:13:16,500 --> 00:13:17,000 so 388 00:13:17,000 --> 00:13:20,133 use AI to win the deal or get them onto a higher plan 389 00:13:20,233 --> 00:13:22,500 pretty much but it's definitely in flux 390 00:13:22,500 --> 00:13:24,266 About 37% of respondents 391 00:13:24,266 --> 00:13:26,166 said they're actively exploring totally 392 00:13:26,166 --> 00:13:27,300 new pricing models 393 00:13:27,300 --> 00:13:30,366 maybe based purely on consumption or ROI delivered 394 00:13:30,366 --> 00:13:32,200 or different usage tiers so 395 00:13:32,200 --> 00:13:32,800 the industry is 396 00:13:32,800 --> 00:13:34,700 still figuring out the best way to capture 397 00:13:34,700 --> 00:13:36,233 the value of AI directly 398 00:13:36,233 --> 00:13:37,666 yeah it sounds like the monetization 399 00:13:37,666 --> 00:13:38,966 playbook is still being written 400 00:13:39,266 --> 00:13:42,100 okay with all this focus on AI products 401 00:13:42,300 --> 00:13:44,866 what about the crucial areas like explainability 402 00:13:44,866 --> 00:13:47,133 transparency compliance governance 403 00:13:47,233 --> 00:13:50,066 how are companies handling these as their AI scales up 404 00:13:50,066 --> 00:13:52,266 it definitely becomes more critical as you scale 405 00:13:52,500 --> 00:13:54,033 providing detailed transparency 406 00:13:54,033 --> 00:13:56,500 reports about the models or even just basic insights 407 00:13:56,500 --> 00:13:58,866 into how the AI is influencing outcomes 408 00:13:58,866 --> 00:13:59,900 that becomes really important 409 00:13:59,900 --> 00:14:01,000 for building and maintaining 410 00:14:01,000 --> 00:14:02,100 user trust makes sense 411 00:14:02,100 --> 00:14:03,866 are they putting formal policies in place 412 00:14:03,866 --> 00:14:06,400 most companies say they have some guardrails around AI 413 00:14:06,400 --> 00:14:09,000 ethics and governance about two thirds 414 00:14:09,000 --> 00:14:11,266 66% say they practice 415 00:14:11,266 --> 00:14:13,466 basic compliance with data privacy laws 416 00:14:13,466 --> 00:14:15,566 like GDPR or CCA okay 417 00:14:15,566 --> 00:14:16,633 basic compliance 418 00:14:16,633 --> 00:14:18,900 what about ensuring fairness and safety 419 00:14:18,966 --> 00:14:20,033 a significant number 420 00:14:20,033 --> 00:14:23,200 38% are using human in the loop oversight 421 00:14:23,466 --> 00:14:24,066 basically 422 00:14:24,066 --> 00:14:27,700 having people review or monitor the AI's decisions 423 00:14:27,700 --> 00:14:30,300 to catch potential biases or errors 424 00:14:30,433 --> 00:14:32,866 that seems to be a key practical step right now 425 00:14:32,866 --> 00:14:34,566 so putting all this together 426 00:14:34,700 --> 00:14:36,800 what does it mean for building the right team 427 00:14:36,800 --> 00:14:38,766 the right organization around AI 428 00:14:38,766 --> 00:14:41,700 are we seeing dedicated AI leadership emerge 429 00:14:41,700 --> 00:14:43,366 what kind of talent are they hunting for 430 00:14:43,366 --> 00:14:43,633 yeah 431 00:14:43,633 --> 00:14:45,666 dedicated leadership is definitely becoming more common 432 00:14:45,666 --> 00:14:47,166 especially as companies grow 433 00:14:47,433 --> 00:14:48,100 between 50 434 00:14:48,100 --> 00:14:51,233 and 61% of companies with over $100 million in revenue 435 00:14:51,233 --> 00:14:54,033 have dedicated AI or ML leadership roles 436 00:14:54,033 --> 00:14:56,833 okay so scale drives the need for specific leadership 437 00:14:56,833 --> 00:14:57,766 seems like it 438 00:14:57,800 --> 00:14:59,433 probably'cause the operational complexity 439 00:14:59,433 --> 00:15:01,033 just ramps up significantly 440 00:15:01,033 --> 00:15:03,400 in terms of specific roles they're hiring for 441 00:15:03,400 --> 00:15:05,200 the most common are AML engineers 442 00:15:05,200 --> 00:15:08,433 that's 88% of companies the core builders right 443 00:15:08,433 --> 00:15:10,766 then data scientists at 72% 444 00:15:11,033 --> 00:15:13,700 and AI product managers at 54% 445 00:15:13,700 --> 00:15:17,033 those PMS who really understand AI are becoming crucial 446 00:15:17,033 --> 00:15:19,333 and which roles are hardest to fill 447 00:15:20,000 --> 00:15:21,566 interestingly the report says 448 00:15:21,566 --> 00:15:23,600 ML engineers take the longest time to hire 449 00:15:23,600 --> 00:15:26,666 on average about 70 days 70 days 450 00:15:26,666 --> 00:15:26,900 wow 451 00:15:26,900 --> 00:15:29,500 that really highlights how specialized and in demand 452 00:15:29,500 --> 00:15:31,500 that talent is it really does 453 00:15:31,500 --> 00:15:32,566 so given that demand 454 00:15:32,566 --> 00:15:34,966 how are companies feeling about their hiring speed 455 00:15:35,000 --> 00:15:36,600 for AI talent overall 456 00:15:36,600 --> 00:15:37,500 are they keeping up 457 00:15:37,500 --> 00:15:39,366 it's pretty much split down the middle 458 00:15:39,500 --> 00:15:43,233 About 54% feel they are not hiring fast enough okay 459 00:15:43,233 --> 00:15:44,900 just over half feel they're behind 460 00:15:44,900 --> 00:15:47,300 and why is that the main reason cited 461 00:15:47,300 --> 00:15:49,300 by 60% of those who feel they're behind 462 00:15:49,300 --> 00:15:51,500 is simply a lack of qualified candidates 463 00:15:51,500 --> 00:15:54,200 the talent crunch is real seems like it 464 00:15:54,200 --> 00:15:57,100 the demand is outstripping the supply of experienced AI 465 00:15:57,100 --> 00:15:59,100 professionals and looking at the bigger picture 466 00:15:59,100 --> 00:16:00,666 within engineering teams 467 00:16:00,666 --> 00:16:03,000 what slice of the pie are companies dedicating 468 00:16:03,000 --> 00:16:05,000 specifically to AI work 469 00:16:05,000 --> 00:16:07,700 on average companies are planning to have about 20 470 00:16:07,700 --> 00:16:10,966 30% of their entire engineering team focused on AI 471 00:16:10,966 --> 00:16:13,500 20 to 30% that's a significant chunk 472 00:16:13,666 --> 00:16:14,900 it is and again 473 00:16:14,900 --> 00:16:17,100 high growth companies are aiming even higher 474 00:16:17,100 --> 00:16:20,600 they're targeting around 37% of their engineering team 475 00:16:20,600 --> 00:16:23,000 focused on AI by 2026 476 00:16:23,000 --> 00:16:26,366 37% that really shows a strategic commitment 477 00:16:26,366 --> 00:16:28,766 doesn't it AI isn't just a side project 478 00:16:28,766 --> 00:16:30,766 it's becoming a core part of engineering 479 00:16:30,766 --> 00:16:31,200 exactly 480 00:16:31,200 --> 00:16:33,866 it's moving from an add on to a fundamental discipline 481 00:16:33,866 --> 00:16:34,100 okay 482 00:16:34,100 --> 00:16:36,800 let's unpack the economics of this all this building 483 00:16:36,800 --> 00:16:38,066 scaling hiring 484 00:16:38,066 --> 00:16:40,300 it obviously comes with a significant cost 485 00:16:40,500 --> 00:16:42,900 what are companies actually spending on AI development 486 00:16:42,900 --> 00:16:45,766 and which costs are the trickiest to get a handle on 487 00:16:45,766 --> 00:16:47,266 right the budget question 488 00:16:47,500 --> 00:16:49,966 companies are allocating a pretty substantial slice 489 00:16:50,266 --> 00:16:52,700 roughly 10 20% of their total R&D budget 490 00:16:52,700 --> 00:16:54,500 specifically to AI development 491 00:16:54,500 --> 00:16:56,633 10 to 20% of R&D wow 492 00:16:56,633 --> 00:16:57,000 yeah 493 00:16:57,000 --> 00:16:59,666 and most of them plan to increase that spend in 2025 494 00:16:59,666 --> 00:17:01,066 so the investment is growing 495 00:17:01,233 --> 00:17:03,466 what's interesting is how the cost mix changes 496 00:17:03,466 --> 00:17:06,166 as AI products scale initially 497 00:17:06,166 --> 00:17:08,466 talent might be a bigger proportion of the cost yeah 498 00:17:08,466 --> 00:17:09,233 but as you scale 499 00:17:09,233 --> 00:17:12,000 the cost of talent tends to decrease proportionally 500 00:17:12,000 --> 00:17:13,700 while infrastructure and compute costs 501 00:17:13,700 --> 00:17:15,233 really start to climb ah 502 00:17:15,233 --> 00:17:16,000 so it shifts from 503 00:17:16,000 --> 00:17:19,933 upfront people investment to ongoing operational costs 504 00:17:19,966 --> 00:17:22,400 exactly and when you ask what's hardest to control 505 00:17:22,700 --> 00:17:24,900 API usage fees come out on top 506 00:17:25,166 --> 00:17:28,100 70% of respondents cited those as the most challenging 507 00:17:28,100 --> 00:17:30,033 infrastructure costs 70% 508 00:17:30,033 --> 00:17:32,400 why are ATI fees so hard to manage 509 00:17:32,600 --> 00:17:34,300 probably the unpredictability 510 00:17:34,833 --> 00:17:36,166 it's a variable cost 511 00:17:36,166 --> 00:17:38,366 tied directly to how much your product is used 512 00:17:38,366 --> 00:17:40,566 how many calls it makes to external models 513 00:17:40,900 --> 00:17:42,000 it can fluctuate a lot 514 00:17:42,000 --> 00:17:43,633 and be hard to forecast accurately 515 00:17:43,633 --> 00:17:45,066 I can see that being a headache 516 00:17:45,066 --> 00:17:46,833 what other costs are challenging 517 00:17:46,833 --> 00:17:48,966 inference cost the actual cost of running the models 518 00:17:48,966 --> 00:17:52,466 to generate predictions that's a big one too at 49% 519 00:17:52,666 --> 00:17:54,366 and the cost of model retraining 520 00:17:54,366 --> 00:17:55,633 and updates is right behind it 521 00:17:55,633 --> 00:17:56,766 at 48% 522 00:17:56,800 --> 00:17:58,866 keeping the models fresh and performing isn't cheap 523 00:17:58,866 --> 00:18:00,300 so if those API usage 524 00:18:00,300 --> 00:18:02,766 fees and inference costs are the big pain points 525 00:18:02,966 --> 00:18:05,300 what are companies doing to try and get those costs 526 00:18:05,300 --> 00:18:06,033 under control 527 00:18:06,033 --> 00:18:08,200 what optimization strategies are they using 528 00:18:08,200 --> 00:18:11,200 they're primarily exploring two main routes right now 529 00:18:11,333 --> 00:18:12,066 first 530 00:18:12,066 --> 00:18:15,200 about 41% are looking at moving towards open source 531 00:18:15,200 --> 00:18:17,866 models to reduce dependency on paid APIs 532 00:18:18,000 --> 00:18:20,633 exactly if you host your own open source model 533 00:18:20,633 --> 00:18:22,600 you might have higher infrastructure costs 534 00:18:22,700 --> 00:18:25,666 but you avoid those per call API fees 535 00:18:25,800 --> 00:18:27,300 the second main strategy 536 00:18:27,300 --> 00:18:30,833 cited by 37% is optimizing inference efficiency 537 00:18:30,833 --> 00:18:33,366 making the models run faster and cheaper right 538 00:18:33,366 --> 00:18:35,166 using techniques like quantization 539 00:18:35,166 --> 00:18:37,166 pruning or deploying specialized hardware 540 00:18:37,166 --> 00:18:39,666 to reduce the compute needed for each prediction 541 00:18:39,666 --> 00:18:40,600 makes sense tackle 542 00:18:40,600 --> 00:18:41,633 the API dependency 543 00:18:41,633 --> 00:18:44,000 and make the models themselves cheaper to run 544 00:18:44,266 --> 00:18:46,366 what about the specific costs for things like 545 00:18:46,366 --> 00:18:48,866 say model training versus inference 546 00:18:48,866 --> 00:18:50,300 as these products mature 547 00:18:50,300 --> 00:18:52,066 do we have numbers on how those scale 548 00:18:52,066 --> 00:18:54,000 we do and the scaling is pretty dramatic 549 00:18:54,000 --> 00:18:55,666 for model training or fine tuning 550 00:18:55,666 --> 00:18:57,866 most companies are doing this at least monthly 551 00:18:57,900 --> 00:18:59,400 the estimated monthly cost for this 552 00:18:59,400 --> 00:19:03,466 it starts around $163,000 for pre launch products 553 00:19:03,466 --> 00:19:05,300 okay 163 k a month 554 00:19:05,300 --> 00:19:07,000 even before launch yeah 555 00:19:07,033 --> 00:19:09,766 but then it skyrockets for products that are scaling 556 00:19:09,766 --> 00:19:11,166 that monthly training cost 557 00:19:11,166 --> 00:19:14,066 jumps to an estimated 1.5 million dollars 558 00:19:14,066 --> 00:19:17,700 wow from 163 k to 1.5 million dollars a month 559 00:19:17,700 --> 00:19:19,500 that's almost a tenfold increase 560 00:19:19,666 --> 00:19:20,600 it's huge hmm 561 00:19:20,600 --> 00:19:23,933 and inference costs they surge right after launch 562 00:19:24,066 --> 00:19:25,166 and here again 563 00:19:25,233 --> 00:19:27,766 high growth companies are spending significantly more 564 00:19:27,766 --> 00:19:31,066 how much more at the general availability GA stage 565 00:19:31,066 --> 00:19:34,366 high growth companies are spending around $1.6 million 566 00:19:34,366 --> 00:19:35,466 monthly on inference 567 00:19:35,466 --> 00:19:38,166 compared to $1.0 million for their peers 568 00:19:38,700 --> 00:19:40,166 and at the scaling stage 569 00:19:40,366 --> 00:19:42,600 it's $2.3 million for high growth 570 00:19:42,766 --> 00:19:44,800 versus $1.1 million for others 571 00:19:44,800 --> 00:19:47,400 so double the inference cost for high growth companies 572 00:19:47,400 --> 00:19:48,666 at scale roughly 573 00:19:48,666 --> 00:19:50,466 yeah and it's not just compute 574 00:19:50,466 --> 00:19:53,566 data storage and processing costs also climb steeply 575 00:19:53,566 --> 00:19:55,066 after GA again 576 00:19:55,066 --> 00:19:57,500 high growth companies are spending more there too 577 00:19:57,500 --> 00:19:58,266 managing the data 578 00:19:58,266 --> 00:20:00,300 becomes a bigger financial piece of the puzzle 579 00:20:00,300 --> 00:20:01,400 as you scale okay 580 00:20:01,400 --> 00:20:02,966 shifting gears slightly mm hmm 581 00:20:02,966 --> 00:20:04,900 beyond building customer facing products 582 00:20:04,900 --> 00:20:07,366 how are companies using AI internally 583 00:20:07,366 --> 00:20:09,033 you know to boost their own productivity what 584 00:20:09,033 --> 00:20:10,733 kind of investment are we seeing there 585 00:20:10,833 --> 00:20:12,100 this is a really interesting area 586 00:20:12,100 --> 00:20:14,766 the report shows that annual budgets for internal AI 587 00:20:14,766 --> 00:20:18,666 productivity tools are set to nearly double in 2025 588 00:20:18,833 --> 00:20:21,200 across all company sizes double 589 00:20:21,200 --> 00:20:23,466 okay so internal AI is getting serious investment 590 00:20:23,466 --> 00:20:25,566 how much are they spending relative to revenue 591 00:20:25,633 --> 00:20:27,466 it varies but typically 592 00:20:27,566 --> 00:20:31,900 somewhere between 1% and 8% of total company revenue 593 00:20:31,900 --> 00:20:34,433 is being allocated to internal AI 594 00:20:34,433 --> 00:20:35,600 productivity efforts 595 00:20:35,600 --> 00:20:37,800 and where's that budget coming from still RND 596 00:20:37,800 --> 00:20:41,333 RND is still the most common source at 59% 597 00:20:41,466 --> 00:20:42,800 but interestingly 598 00:20:42,800 --> 00:20:45,800 headcount budgets are also being tapped at 23% 599 00:20:45,800 --> 00:20:47,100 headcount budgets so 600 00:20:47,100 --> 00:20:48,600 they're potentially funding AI 601 00:20:48,600 --> 00:20:50,866 tools instead of hiring more people 602 00:20:50,866 --> 00:20:51,566 it certainly suggests 603 00:20:51,566 --> 00:20:53,766 they're viewing these tools as a way to 604 00:20:53,766 --> 00:20:56,300 enhance the productivity of their existing workforce 605 00:20:56,466 --> 00:20:58,700 maybe offsetting the need for additional hires 606 00:20:58,700 --> 00:21:01,766 in some areas it signals that internal AI 607 00:21:01,766 --> 00:21:04,433 is moving beyond just R&D experiments 608 00:21:04,433 --> 00:21:06,966 and becoming part of core operational strategy 609 00:21:06,966 --> 00:21:08,833 that's a significant shift in mindset 610 00:21:08,833 --> 00:21:10,466 but okay they're investing the money 611 00:21:10,633 --> 00:21:11,433 what are the challenges 612 00:21:11,433 --> 00:21:14,033 in actually getting employees to use these internal AI 613 00:21:14,033 --> 00:21:15,000 tools effectively 614 00:21:15,000 --> 00:21:16,266 what does adoption look like 615 00:21:16,266 --> 00:21:17,966 yeah there's a bit of a gap there 616 00:21:17,966 --> 00:21:19,200 yeah while a good majority 617 00:21:19,200 --> 00:21:20,766 around 70% of employees 618 00:21:20,866 --> 00:21:23,800 apparently have access to various AI tools internally 619 00:21:23,800 --> 00:21:25,500 70% have access right 620 00:21:25,500 --> 00:21:27,100 but only about half of them 621 00:21:27,100 --> 00:21:30,633 50% are actually using these tools on an ongoing basis 622 00:21:30,633 --> 00:21:32,166 only half are regular users 623 00:21:32,166 --> 00:21:35,066 why the gap the report suggests adoption is proving 624 00:21:35,066 --> 00:21:37,600 tougher in more mature larger enterprises 625 00:21:37,900 --> 00:21:40,366 those over $1 billion in revenue 626 00:21:40,800 --> 00:21:42,466 maybe it's ingrained habits 627 00:21:42,466 --> 00:21:45,433 legacy systems change management challenges 628 00:21:45,433 --> 00:21:49,033 could be and does the decision process for internal AI 629 00:21:49,033 --> 00:21:51,166 tools differ from customer facing ones 630 00:21:51,166 --> 00:21:52,633 yes significantly 631 00:21:52,633 --> 00:21:54,033 for internal use cases 632 00:21:54,033 --> 00:21:56,100 cost is the most important factor 633 00:21:56,100 --> 00:22:00,633 cited by 74% cost over accuracy for internal tools 634 00:22:00,633 --> 00:22:02,700 yep accuracy is still crucial 635 00:22:02,800 --> 00:22:06,966 close behind at 72% and privacy is important too at 50% 636 00:22:07,033 --> 00:22:09,166 but cost really leads the way internally 637 00:22:09,166 --> 00:22:10,833 it's a different optimization equation 638 00:22:10,833 --> 00:22:12,166 compared to external products 639 00:22:12,166 --> 00:22:14,300 where accuracy is usually Paramount 640 00:22:14,366 --> 00:22:15,200 interesting 641 00:22:15,266 --> 00:22:18,066 and what are the biggest roadblocks to deploying these 642 00:22:18,066 --> 00:22:19,833 internal tools is it tech hurdles 643 00:22:19,833 --> 00:22:21,166 not primarily no 644 00:22:21,166 --> 00:22:23,533 the biggest challenges seem to be more strategic 645 00:22:23,700 --> 00:22:26,833 finding the right use cases is number one at 46% 646 00:22:26,833 --> 00:22:29,300 figuring out where AI can genuinely make a difference 647 00:22:29,300 --> 00:22:32,066 yeah and number two is proving the ROI at 42% 648 00:22:32,066 --> 00:22:33,566 so it's less about can we build it 649 00:22:33,566 --> 00:22:34,900 and more about should we build it 650 00:22:34,900 --> 00:22:37,166 and how do we prove it's working exactly 651 00:22:37,266 --> 00:22:38,800 it's about integrating AI 652 00:22:38,800 --> 00:22:40,933 effectively into the daily workflow 653 00:22:41,100 --> 00:22:42,666 and demonstrating its value 654 00:22:42,666 --> 00:22:45,233 more than just the technical implementation 655 00:22:45,233 --> 00:22:47,900 okay so if finding the right use cases is key 656 00:22:48,600 --> 00:22:50,566 how many are companies exploring 657 00:22:50,900 --> 00:22:52,866 and which internal AI applications 658 00:22:52,866 --> 00:22:54,833 are actually delivering the biggest impact 659 00:22:54,833 --> 00:22:58,066 right now there seems to be a link between exploration 660 00:22:58,066 --> 00:23:01,000 and adoption companies that have high employee adoption 661 00:23:01,000 --> 00:23:02,666 remember over 50% usage 662 00:23:02,666 --> 00:23:06,233 they typically explore more use cases around 7 or more 663 00:23:06,233 --> 00:23:07,233 internally compared to 664 00:23:07,233 --> 00:23:10,333 compared to only about 4.6 use cases 665 00:23:10,600 --> 00:23:12,000 companies with low adoption 666 00:23:12,033 --> 00:23:13,400 so broader experimentation 667 00:23:13,400 --> 00:23:14,366 seems to correlate with 668 00:23:14,366 --> 00:23:16,200 more people actually using the tools 669 00:23:16,200 --> 00:23:18,866 makes sense which departments are using AI most 670 00:23:18,866 --> 00:23:22,200 R&D leads the way 77% are using AI there 671 00:23:22,200 --> 00:23:25,066 sales and marketing is also high at 65% 672 00:23:25,200 --> 00:23:28,133 general and administrative functions like finance or HR 673 00:23:28,233 --> 00:23:29,633 lag a bit behind okay 674 00:23:29,633 --> 00:23:31,666 R&D and sales marketing are the hot spots 675 00:23:31,666 --> 00:23:32,666 and the killer app 676 00:23:32,666 --> 00:23:34,200 which use case has the biggest impact 677 00:23:34,200 --> 00:23:35,666 Wi Fi it's coding assistance 678 00:23:35,666 --> 00:23:37,866 65% cite that is having a tangible impact 679 00:23:37,866 --> 00:23:40,000 coding assistance like Github Code Pilot 680 00:23:40,000 --> 00:23:40,800 exactly 681 00:23:41,000 --> 00:23:43,633 respondents report an average productivity gain of 15 682 00:23:43,633 --> 00:23:46,366 30% across these various Genie I use cases 683 00:23:46,366 --> 00:23:47,400 uh huh but 684 00:23:47,633 --> 00:23:48,900 coding assistance 685 00:23:49,000 --> 00:23:51,033 seems to be the most consistently impactful one 686 00:23:51,033 --> 00:23:53,766 15 to 30% gain that's substantial 687 00:23:53,766 --> 00:23:54,800 it really is and again 688 00:23:54,800 --> 00:23:56,433 high growth companies are leaning into this 689 00:23:56,433 --> 00:23:57,500 more heavily they see 690 00:23:57,500 --> 00:24:00,433 an average of 33% of their total code being written 691 00:24:00,433 --> 00:24:01,566 with AI assistance 692 00:24:01,566 --> 00:24:03,633 compared to 27% for other companies 693 00:24:03,633 --> 00:24:06,000 that coding assistance impact is really impressive 694 00:24:06,066 --> 00:24:08,800 what's the general attitude towards internal AI 695 00:24:08,800 --> 00:24:11,400 especially among those leading high growth companies 696 00:24:11,433 --> 00:24:12,833 are they fully embracing it 697 00:24:12,833 --> 00:24:14,533 yes very much so 698 00:24:14,633 --> 00:24:15,666 the report highlights that 699 00:24:15,666 --> 00:24:16,500 high growth companies 700 00:24:16,500 --> 00:24:18,800 tend to be much more active in experimenting with 701 00:24:18,800 --> 00:24:20,866 and adopting new AI tools 702 00:24:21,000 --> 00:24:24,166 92% of them are doing this 92% yeah 703 00:24:24,300 --> 00:24:26,700 it strongly suggests they view internal AI 704 00:24:26,700 --> 00:24:28,700 not just as a nice to have 705 00:24:28,700 --> 00:24:30,766 but as a core strategic lever 706 00:24:31,033 --> 00:24:31,500 they're moving 707 00:24:31,500 --> 00:24:34,066 faster to integrate it across their own workflows 708 00:24:34,066 --> 00:24:35,066 likely seeing a clear 709 00:24:35,066 --> 00:24:37,100 competitive advantage in making their own teams 710 00:24:37,100 --> 00:24:38,633 more efficient with AI right 711 00:24:38,633 --> 00:24:40,566 it's like they're eating their own dog food 712 00:24:40,566 --> 00:24:42,933 really leveraging AI internally as well 713 00:24:43,166 --> 00:24:45,600 how are companies actually measuring the ROI 714 00:24:45,600 --> 00:24:48,633 for this internal AI use what metrics are they tracking 715 00:24:48,633 --> 00:24:50,900 they're primarily focused on productivity improvements 716 00:24:50,900 --> 00:24:55,266 that's 75% and direct cost savings at 51% okay 717 00:24:55,266 --> 00:24:56,800 efficiency and cost reduction 718 00:24:56,800 --> 00:24:59,666 right and about 30% are tracking a mix of both 719 00:24:59,666 --> 00:25:02,600 quantitative measures things like time saved 720 00:25:02,700 --> 00:25:05,100 task completed faster and qualitative measures 721 00:25:05,233 --> 00:25:07,000 like employee satisfaction surveys 722 00:25:07,000 --> 00:25:08,700 or feedback on the tools 723 00:25:09,066 --> 00:25:11,900 getting both the hard numbers and the user sentiment 724 00:25:11,900 --> 00:25:13,500 okay this is super insightful 725 00:25:13,500 --> 00:25:14,900 now here's where it gets really interesting 726 00:25:14,900 --> 00:25:15,900 for the builders listening 727 00:25:15,900 --> 00:25:17,766 if you're actually building this stuff 728 00:25:17,833 --> 00:25:19,166 what are the specific 729 00:25:19,166 --> 00:25:22,200 tools and technologies powering all this innovation 730 00:25:22,366 --> 00:25:23,100 let's do a quick 731 00:25:23,100 --> 00:25:25,300 deep dive into the AI text stack itself 732 00:25:25,300 --> 00:25:26,766 absolutely let's break it down 733 00:25:26,866 --> 00:25:29,500 for LLM and AI application development 734 00:25:29,500 --> 00:25:31,066 kind of the core building blocks 735 00:25:31,066 --> 00:25:32,633 tools like Langchain and Hugging 736 00:25:32,633 --> 00:25:35,200 Face's toolset are really prominent for orchestration 737 00:25:35,200 --> 00:25:36,600 orchestration meaning 738 00:25:36,666 --> 00:25:39,600 managing the flow between models and data exactly 739 00:25:39,600 --> 00:25:42,266 they act like the conductor of the AI orchestra 740 00:25:42,900 --> 00:25:45,566 guardrails tools are also important for safety checks 741 00:25:45,566 --> 00:25:47,133 making sure outputs are OK 742 00:25:47,300 --> 00:25:49,766 and Versailes AI sk pops up too 743 00:25:49,766 --> 00:25:53,133 used by 23% likely cause it helps deploy things quickly 744 00:25:53,966 --> 00:25:56,300 then for model training and fine tuning the 745 00:25:56,300 --> 00:25:58,866 heavy lifters are Pytorch and Tensorflow 746 00:25:59,066 --> 00:26:01,400 they account for over half of all usage 747 00:26:01,566 --> 00:26:04,233 these are the classic deep learning frameworks right 748 00:26:04,233 --> 00:26:07,033 but you also see popular managed services like AWS 749 00:26:07,033 --> 00:26:07,700 Sage Maker 750 00:26:07,700 --> 00:26:10,666 and Openai's fine tuning API being used a lot 751 00:26:10,700 --> 00:26:14,100 showing that mix of DIY control and managed convenience 752 00:26:14,800 --> 00:26:16,200 for monitoring and observability 753 00:26:16,200 --> 00:26:18,133 keeping an eye on how things are performing 754 00:26:18,233 --> 00:26:20,066 nearly half of teams are actually adapting 755 00:26:20,066 --> 00:26:21,100 their existing software 756 00:26:21,100 --> 00:26:23,833 monitoring tools like Data Dog or honeycomb 757 00:26:23,833 --> 00:26:24,800 using what they already have 758 00:26:24,800 --> 00:26:27,800 yeah but specialized ML native platforms like Langsmith 759 00:26:27,800 --> 00:26:30,166 and weights and biases are definitely gaining traction 760 00:26:30,200 --> 00:26:32,033 hitting around 17% adoption 761 00:26:32,033 --> 00:26:34,800 hmm suggest a growing need for AI specific monitoring 762 00:26:34,800 --> 00:26:35,866 inference optimization 763 00:26:35,866 --> 00:26:37,333 making models run efficiently 764 00:26:37,400 --> 00:26:39,066 it's heavily dominated by Nvidia 765 00:26:39,066 --> 00:26:41,166 their tensor RT inference server command 766 00:26:41,166 --> 00:26:42,500 over 60% adoption 767 00:26:42,500 --> 00:26:44,033 Nvidia is still the king of deployment 768 00:26:44,033 --> 00:26:45,133 hardware and software 769 00:26:45,166 --> 00:26:47,300 pretty much for non Nvidia solutions on 770 00:26:47,300 --> 00:26:49,966 Nvidia runtime is a notable player at 18% 771 00:26:50,000 --> 00:26:50,966 model hosting 772 00:26:51,233 --> 00:26:53,900 most teams just hit the model provider APIs directly 773 00:26:53,900 --> 00:26:55,933 Openai Anthropic etcetera 774 00:26:56,100 --> 00:26:58,866 but the big cloud hyperscalers like AWS 775 00:26:58,866 --> 00:26:59,900 Bedrock and Google 776 00:26:59,900 --> 00:27:03,033 Vertex AI have carved out a big chunk of the market 777 00:27:03,033 --> 00:27:05,100 especially for larger later stage companies 778 00:27:05,100 --> 00:27:07,133 needing more integrated hosting solutions 779 00:27:07,466 --> 00:27:09,900 model evaluation checking how good the models are 780 00:27:09,900 --> 00:27:10,633 nearly a quarter 781 00:27:10,633 --> 00:27:12,700 use built in features from their platforms 782 00:27:12,800 --> 00:27:15,433 but purpose built tools like Langsmith and Langfuse 783 00:27:15,433 --> 00:27:17,800 are leading the dedicated evaluation tool space 784 00:27:18,133 --> 00:27:19,900 data processing and feature engineering 785 00:27:19,900 --> 00:27:21,566 getting the data ready it's the classic 786 00:27:21,566 --> 00:27:24,733 big data tools a patchy spark at 44% 787 00:27:24,766 --> 00:27:27,300 Kafka for streaming at 42% 788 00:27:27,433 --> 00:27:29,833 and pandas is still indispensable for Python 789 00:27:29,833 --> 00:27:31,700 data manipulation at 41% 790 00:27:31,700 --> 00:27:33,300 the workhorses of data engineering 791 00:27:33,300 --> 00:27:34,966 exactly vector databases 792 00:27:34,966 --> 00:27:35,633 crucial for Apache 793 00:27:35,633 --> 00:27:37,733 Elastic and Pinecone are leading the pack 794 00:27:37,766 --> 00:27:39,233 showing a mix of established players 795 00:27:39,233 --> 00:27:41,066 adapting and purpose built solutions 796 00:27:41,233 --> 00:27:43,266 open source options are gaining ground too 797 00:27:43,366 --> 00:27:45,233 synthetic data and data augmentation 798 00:27:45,233 --> 00:27:46,433 so it's interesting over half 799 00:27:46,433 --> 00:27:49,266 52% are building their own custom tooling for this 800 00:27:49,266 --> 00:27:51,266 building their own synthetic data tools 801 00:27:51,366 --> 00:27:54,166 yeah shows how specific data needs can be scale 802 00:27:54,166 --> 00:27:57,666 AI is the clear third party leader here though at 21% 803 00:27:57,766 --> 00:27:59,866 coding assistance we talked about its impact 804 00:28:00,066 --> 00:28:01,600 Github Copilot is the giant 805 00:28:01,600 --> 00:28:04,300 used by almost three quarters of dev teams 75% 806 00:28:04,300 --> 00:28:06,300 cursor is a strong second at 50% 807 00:28:06,300 --> 00:28:08,000 copilot is everywhere 808 00:28:08,000 --> 00:28:11,966 almost Devops and Mlops managing the whole life cycle 809 00:28:12,033 --> 00:28:14,866 ML flow is the clear front runner at 36% 810 00:28:14,866 --> 00:28:17,000 followed by weights and biases at 20% 811 00:28:17,100 --> 00:28:19,366 but the market here still feels pretty fragmented 812 00:28:19,366 --> 00:28:21,266 lots of tools product and design 813 00:28:21,266 --> 00:28:24,800 it's basically Figma 87% use it for UI UX 814 00:28:24,866 --> 00:28:27,166 Miro's popular too at 37% 815 00:28:27,200 --> 00:28:29,800 for more high level whiteboarding and collaboration 816 00:28:30,166 --> 00:28:31,066 and finally 817 00:28:31,066 --> 00:28:33,266 those internal productivity tools we discussed 818 00:28:33,266 --> 00:28:35,533 you see AI features being baked into the platform 819 00:28:35,666 --> 00:28:38,300 teams already use daily Salesforce for sales 820 00:28:38,300 --> 00:28:40,633 Canva for marketing Zendesk for support 821 00:28:40,733 --> 00:28:41,733 notion for knowledge management 822 00:28:42,033 --> 00:28:44,266 service now for it LinkedIn for HR 823 00:28:44,566 --> 00:28:46,900 AI is becoming embedded everywhere 824 00:28:46,900 --> 00:28:47,566 wow okay 825 00:28:47,566 --> 00:28:49,766 this deep dive has really given us an incredible 826 00:28:49,766 --> 00:28:50,400 real world 827 00:28:50,400 --> 00:28:52,966 look into how companies are actually doing it building 828 00:28:52,966 --> 00:28:54,733 scaling managing AI products 829 00:28:54,900 --> 00:28:55,933 and just as importantly 830 00:28:55,933 --> 00:28:57,700 how they're weaving AI into their own 831 00:28:57,700 --> 00:28:58,633 internal operations 832 00:28:58,633 --> 00:29:00,700 it really feels like a view from the trenches 833 00:29:00,700 --> 00:29:02,933 I think so too and it's more than just 834 00:29:02,933 --> 00:29:04,466 you know interesting facts 835 00:29:04,566 --> 00:29:06,533 it feels like a practical blueprint 836 00:29:06,533 --> 00:29:08,166 based on what hundreds of executives 837 00:29:08,166 --> 00:29:09,166 are actually experiencing 838 00:29:09,166 --> 00:29:10,500 and doing right now yeah 839 00:29:10,500 --> 00:29:13,433 this data offers some really actionable insights 840 00:29:13,433 --> 00:29:14,200 I think 841 00:29:14,200 --> 00:29:16,900 for anyone trying to navigate AI development because 842 00:29:16,900 --> 00:29:17,800 you know knowledge is great 843 00:29:17,800 --> 00:29:18,700 but it's most valuable 844 00:29:18,700 --> 00:29:19,733 when you can actually understand it 845 00:29:19,733 --> 00:29:20,666 and apply it 846 00:29:20,966 --> 00:29:23,133 this report gives a pretty solid roadmap for that 847 00:29:23,133 --> 00:29:25,833 definitely so as we wrap up here 848 00:29:25,833 --> 00:29:28,166 thinking back over everything we've unpacked from this 849 00:29:28,166 --> 00:29:29,933 AI builder's playbook hmm 850 00:29:30,000 --> 00:29:31,766 what's the one big takeaway 851 00:29:31,766 --> 00:29:33,433 that really stands out to you 852 00:29:33,600 --> 00:29:35,300 hmm it's a tough one 853 00:29:35,400 --> 00:29:38,333 I think for me it's the growing importance 854 00:29:38,366 --> 00:29:41,933 almost the necessity now of that multi model strategy 855 00:29:41,933 --> 00:29:44,033 combined with rigorous cost management 856 00:29:44,033 --> 00:29:45,366 especially around inference 857 00:29:45,366 --> 00:29:46,833 it shows a real maturation 858 00:29:46,833 --> 00:29:48,566 moving from just can we make it work 859 00:29:48,566 --> 00:29:50,300 hmm to how do we make it work efficiently 860 00:29:50,300 --> 00:29:52,233 reliably and affordably at scale 861 00:29:52,333 --> 00:29:54,100 that feels like the current frontier 862 00:29:54,100 --> 00:29:54,766 that's a great point 863 00:29:54,766 --> 00:29:56,366 the operational reality is setting in 864 00:29:56,366 --> 00:29:58,433 and maybe a final thought for everyone 865 00:29:58,433 --> 00:30:00,900 listening to Molover we saw that high growth AI 866 00:30:00,900 --> 00:30:02,533 companies are already dedicating 867 00:30:02,533 --> 00:30:04,733 way more engineering resources to AI 868 00:30:04,733 --> 00:30:06,900 they're spending significantly more on core costs 869 00:30:06,900 --> 00:30:08,700 like inference and data 870 00:30:08,700 --> 00:30:11,166 and they're actively experimenting with all these new 871 00:30:11,166 --> 00:30:13,433 internal tools so the question is 872 00:30:13,500 --> 00:30:16,000 what fundamental shifts do other companies need to make 873 00:30:16,000 --> 00:30:17,333 in their talent strategy 874 00:30:17,333 --> 00:30:18,000 their budgets 875 00:30:18,000 --> 00:30:20,000 maybe even their company culture around AI 876 00:30:20,000 --> 00:30:23,100 to genuinely close that gap and compete effectively 877 00:30:23,366 --> 00:30:24,333 in this landscape 878 00:30:24,333 --> 00:30:26,666 that's just evolving so incredibly fast