WEBVTT 00:00:00.000 --> 00:00:03.279 A big thank you to Denodo for helping me make 00:00:03.279 --> 00:00:06.139 more than 60 monthly interviews possible across 00:00:06.139 --> 00:00:09.119 the Tech Talks network. And as businesses move 00:00:09.119 --> 00:00:13.800 from gen .ai to agentic .ai, trusted data becomes 00:00:13.800 --> 00:00:17.620 everything. Everything from gen .ai to agentic 00:00:17.620 --> 00:00:21.379 .ai, Denodo is helping organizations build intelligent, 00:00:21.719 --> 00:00:25.920 secure and scalable AI solutions with data access, 00:00:26.100 --> 00:00:30.199 governance and explainable results. So, build 00:00:30.199 --> 00:00:33.439 AI that you can trust and do it with Denodo. 00:00:34.060 --> 00:00:36.219 And you can learn more by simply visiting denodo 00:00:36.219 --> 00:00:46.700 .com. What happens when AI starts helping developers 00:00:46.700 --> 00:00:49.500 move faster than the business can keep up with? 00:00:49.939 --> 00:00:52.179 But most importantly, who carries the can when 00:00:52.179 --> 00:00:55.020 the code causes a problem in production? Well 00:00:55.020 --> 00:00:58.679 my guest today is the CTO at LaunchDarkly and 00:00:58.679 --> 00:01:01.520 he's going to join me in a smart, lively and 00:01:01.520 --> 00:01:04.299 very timely conversation and we'll have a little 00:01:04.299 --> 00:01:06.079 bit of fun along the way today because we're 00:01:06.079 --> 00:01:10.730 going to talk about Shadow AI in DevOps. the 00:01:10.730 --> 00:01:14.209 growing use of AI written code and the uncomfortable 00:01:14.209 --> 00:01:17.109 questions that many teams are just starting to 00:01:17.109 --> 00:01:20.069 ask. And by that I mean everything from public 00:01:20.069 --> 00:01:24.069 LLM and unsanctioned coding tools to compliance 00:01:24.069 --> 00:01:28.290 headaches, runtime controls, kill switches. Yeah, 00:01:28.390 --> 00:01:31.069 we're going to talk about what happens when convenience 00:01:31.069 --> 00:01:34.930 meets accountability. And Cameron brings with 00:01:34.930 --> 00:01:37.849 him a real world experience, sharp insights and 00:01:37.849 --> 00:01:40.430 a few great stories along the way, including 00:01:40.430 --> 00:01:43.629 why you can't fire an AI model when things go 00:01:43.629 --> 00:01:46.609 wrong. So if your teams are moving fast with 00:01:46.609 --> 00:01:50.010 AI or thinking about it, this one will give you 00:01:50.010 --> 00:01:52.989 a lot to think and talk about. But enough from 00:01:52.989 --> 00:01:57.370 me. Let me introduce you to my guest now. So 00:01:57.370 --> 00:01:59.930 thank you for joining me on the podcast today. 00:02:00.069 --> 00:02:02.069 Can you tell everyone listening a little about 00:02:02.069 --> 00:02:05.579 who you are? and what you do. Sure. I'm Cameron 00:02:05.579 --> 00:02:08.039 Ettazadeh. I'm the Chief Technology Officer at 00:02:08.039 --> 00:02:12.159 LaunchDarkly, which is a company dedicated to 00:02:12.159 --> 00:02:14.659 shipping software quickly and safely. We've been 00:02:14.659 --> 00:02:17.379 around for quite a while. We think we're essential 00:02:17.379 --> 00:02:22.599 as the AI development lifecycle has reinvigorated 00:02:22.599 --> 00:02:25.699 and completely changed the evolution of shipping 00:02:25.699 --> 00:02:28.389 software. I've been in this industry for about 00:02:28.389 --> 00:02:31.870 30 years, not the industry I intended on showing 00:02:31.870 --> 00:02:35.250 up to by any means. I was originally a biochemist 00:02:35.250 --> 00:02:36.990 and a chemical engineer, of all things, so I 00:02:36.990 --> 00:02:40.090 was off to study medicine, or maybe I jokingly 00:02:40.090 --> 00:02:44.789 say I was off to do drugs, as a chemist, but 00:02:44.789 --> 00:02:47.849 I ended up in software at Microsoft in the 90s, 00:02:48.110 --> 00:02:50.830 worked on Windows 2000. I guarantee most people 00:02:50.830 --> 00:02:52.490 in the world have probably used code I've written. 00:02:53.520 --> 00:02:59.560 tours at Amazon, Google, IBM, SAP, Ticketmaster 00:02:59.560 --> 00:03:02.180 for a little bit, where we did some really interesting 00:03:02.180 --> 00:03:04.740 e -commerce stuff. So basically all over the 00:03:04.740 --> 00:03:07.340 place. Spent some time as a CIO of a public company 00:03:07.340 --> 00:03:10.139 as well. So I like to be able to bring that IT 00:03:10.139 --> 00:03:12.580 perspective into what has traditionally been 00:03:12.580 --> 00:03:15.340 a product -focused world, because those are our 00:03:15.340 --> 00:03:18.620 customers a lot of times. And I've got to ask 00:03:18.620 --> 00:03:21.139 the slight gap in your story there tell me about 00:03:21.139 --> 00:03:24.319 that pivot that took you from biochemistry into 00:03:24.319 --> 00:03:27.039 tech because very often I speak to so many people 00:03:27.039 --> 00:03:29.979 on here and there's always a moment of serendipity 00:03:29.979 --> 00:03:31.900 or a little nudge from the universe in the right 00:03:31.900 --> 00:03:34.460 direction at the right time but what was it that 00:03:34.460 --> 00:03:37.639 that made you pivot into tech? You know Microsoft 00:03:37.639 --> 00:03:39.659 actually found me now I've been a nerd since 00:03:39.800 --> 00:03:43.259 I was a little kid. I grew up, believe it or 00:03:43.259 --> 00:03:45.520 not, with some interactions with some famous 00:03:45.520 --> 00:03:52.919 AI researchers in the 80s out of MIT. fortunate, 00:03:52.919 --> 00:03:55.539 I think I would say, to have had exposure from 00:03:55.539 --> 00:03:58.840 a very early age to the Commodore Pets and the 00:03:58.840 --> 00:04:00.699 Apple IIs of the world. So it was always something 00:04:00.699 --> 00:04:02.780 in the back of my head. And then I ended up working 00:04:02.780 --> 00:04:06.379 on some software that did 3D structural modeling 00:04:06.379 --> 00:04:10.139 of proteins from x -ray crystallography. And 00:04:10.139 --> 00:04:12.639 some folks at Microsoft heard about me in the 00:04:12.639 --> 00:04:14.479 90s and just asked me to come out to Seattle 00:04:14.479 --> 00:04:16.660 and interview. And I fell in love with a beautiful 00:04:16.660 --> 00:04:20.490 city. beautiful greenery and I believe they were 00:04:20.490 --> 00:04:22.149 people who were going to change the world. Software 00:04:22.149 --> 00:04:24.649 was going to eat the world and it was an exciting 00:04:24.649 --> 00:04:29.350 time to be there and we did. Everybody loved 00:04:29.350 --> 00:04:30.990 some of the products at the time that we shipped 00:04:30.990 --> 00:04:34.629 like Windows XP, Windows 2000, truly game -changing. 00:04:35.470 --> 00:04:37.949 pieces. And so my entire career, I've truly tried 00:04:37.949 --> 00:04:40.189 to work on things where my passion's been for 00:04:40.189 --> 00:04:42.350 changing the game, whether it's how we deliver 00:04:42.350 --> 00:04:45.790 software, how we deliver experiences, how we 00:04:45.790 --> 00:04:48.430 build for massive planetary scale. It's super 00:04:48.430 --> 00:04:53.089 exciting to me. I absolutely love that. Both 00:04:53.089 --> 00:04:55.230 you and I have been around long enough to have 00:04:55.230 --> 00:04:57.509 seen the early days of the internet, the move 00:04:57.509 --> 00:05:03.389 to cloud, the rise of mobile. Then came BYOD, 00:05:03.810 --> 00:05:06.050 as everyone wanted to use their own phones and 00:05:06.050 --> 00:05:09.689 own iPads in the office. Then came Shadow IT. 00:05:10.050 --> 00:05:13.970 Now we've got AI, and most recently of all, it's 00:05:13.970 --> 00:05:17.670 Shadow AI now. So from your perspective at launch, 00:05:19.300 --> 00:05:22.959 What exactly does Shadow AI look like inside 00:05:22.959 --> 00:05:25.720 a modern engineering team? And why is it becoming 00:05:25.720 --> 00:05:28.759 such a pressing governance issue for enterprises? 00:05:29.060 --> 00:05:30.740 Because many people listening probably would 00:05:30.740 --> 00:05:33.000 have seen it on their LinkedIn or in their newsfeed, 00:05:33.500 --> 00:05:36.279 but tell me a bit more about that. You know, 00:05:36.279 --> 00:05:38.959 I love the fact that people are excited about 00:05:38.959 --> 00:05:42.889 this. Change is inevitable. I think we've learned 00:05:42.889 --> 00:05:44.790 something in that, you know, this industry as 00:05:44.790 --> 00:05:47.329 it reinvents itself has said that Agile was, 00:05:47.610 --> 00:05:50.949 in fact, the very manifestation 30, well, 25 00:05:50.949 --> 00:05:53.509 years ago, you know, it came back of embracing 00:05:53.509 --> 00:05:56.670 change as a true element of the only constant. 00:05:57.209 --> 00:05:59.850 Shadow AI is just the modern manifestation of 00:05:59.850 --> 00:06:04.060 that. Shadow IT... Long created risks around, 00:06:04.060 --> 00:06:07.540 you know, inefficiency, duplicate spend, security 00:06:07.540 --> 00:06:11.220 exposures, compliance issues, all of the nasties 00:06:11.220 --> 00:06:13.399 that both your corporate legal team and your 00:06:13.399 --> 00:06:16.939 CISO and your CIO. all keep, you know, stay up 00:06:16.939 --> 00:06:18.800 at night. These were the things when I was a 00:06:18.800 --> 00:06:21.439 CIO that gave me heartburn was, I used to joke 00:06:21.439 --> 00:06:23.439 I was, you know, one security breach away from 00:06:23.439 --> 00:06:26.579 the unemployment line. And I think a lot of CISOs, 00:06:26.759 --> 00:06:29.699 CIOs feel that way too, every single day. And 00:06:29.699 --> 00:06:32.180 we just see this pattern repeating itself with 00:06:32.180 --> 00:06:35.180 AI, especially when developers use public LLMs 00:06:35.180 --> 00:06:37.180 and plugins to move faster in the day -to -day 00:06:37.180 --> 00:06:41.600 work. I am 100 % for people getting personal 00:06:41.600 --> 00:06:44.319 productivity up. getting rid of the drudgery, 00:06:44.459 --> 00:06:47.379 the toil. It doesn't do a developer any good 00:06:47.379 --> 00:06:49.560 at all to write the same git command 50 times 00:06:49.560 --> 00:06:51.860 a day. And in fact, most of them are pretty sloppy 00:06:51.860 --> 00:06:53.560 about the comments they put in for their check 00:06:53.560 --> 00:06:55.879 -ins. As a good example of that, the AI doesn't 00:06:55.879 --> 00:06:58.240 care. It'll write a really nice, clear, concise 00:06:58.240 --> 00:07:00.500 statement. Here's the key difference, though. 00:07:00.860 --> 00:07:03.579 When you look at Shadow AI, AI is shaping code 00:07:03.579 --> 00:07:05.980 and decisions that most of the time go directly 00:07:05.980 --> 00:07:07.699 into production. They're not just supporting 00:07:07.699 --> 00:07:10.379 workflows. And when you're left without an audit 00:07:10.379 --> 00:07:13.439 trail, It's really difficult to explain how something 00:07:13.439 --> 00:07:16.459 was built or in many cases who's even accountable 00:07:16.459 --> 00:07:19.660 when something breaks a lot of these tools are 00:07:19.660 --> 00:07:24.420 moving so fast that There's there's a sort of 00:07:24.420 --> 00:07:27.139 missing I'm not going to say regulatory framework 00:07:27.139 --> 00:07:29.399 because those exist in many cases are especially 00:07:29.399 --> 00:07:31.339 in the EU are starting to really get codified 00:07:31.339 --> 00:07:37.579 pretty strongly but from a position of audit 00:07:37.579 --> 00:07:39.810 ability From these companies they're ignoring 00:07:39.810 --> 00:07:41.689 them. They're just moving so fast that they're 00:07:41.689 --> 00:07:44.009 like we'll get to that later And so you get things 00:07:44.009 --> 00:07:46.189 like recording tools that don't garner two -party 00:07:46.189 --> 00:07:48.610 consent in many places where that's required 00:07:48.610 --> 00:07:51.970 For instance or leak internal confidential information, 00:07:52.170 --> 00:07:54.290 you know when I cut and paste something Into 00:07:54.290 --> 00:07:57.649 my personal chat gpt account and it's got corporate 00:07:57.649 --> 00:07:59.389 financial data. Who knows where that's going 00:07:59.389 --> 00:08:01.569 to show up We've seen incidents over the last 00:08:01.569 --> 00:08:03.370 couple of years, you know, particularly in the 00:08:03.370 --> 00:08:05.430 last 12 months where we see corporate secrets 00:08:05.430 --> 00:08:09.600 things that are intellectual property that are 00:08:09.600 --> 00:08:11.680 hard to protect just leaked out into the public 00:08:11.680 --> 00:08:14.540 by Simply asking LM as another character and 00:08:14.540 --> 00:08:16.720 and that's really dangerous You know as we look 00:08:16.720 --> 00:08:19.360 at IP law protection for instance trade secrets 00:08:19.360 --> 00:08:22.560 are the least protected and people are just willy 00:08:22.560 --> 00:08:24.939 -nilly Throwing them into production and that 00:08:24.939 --> 00:08:26.439 frightens me that's because that's our secret 00:08:26.439 --> 00:08:28.160 sauce. That's our competitiveness in this world 00:08:28.160 --> 00:08:31.740 as a company Yeah I'm nodding in agreement with 00:08:31.740 --> 00:08:33.600 you there and I can almost feel people around 00:08:33.600 --> 00:08:35.860 the world watching and listening nodding in agreement 00:08:35.860 --> 00:08:38.960 as well and I suspect we've all seen and know 00:08:38.960 --> 00:08:41.679 developers that are experimenting at the moment 00:08:41.679 --> 00:08:46.080 with tools like publicly accessible LLMs to speed 00:08:46.080 --> 00:08:49.299 up coding or decision making and beyond the techies 00:08:49.299 --> 00:08:52.200 that every department in every organization are 00:08:52.200 --> 00:08:54.899 doing the same thing so where does that line 00:08:54.899 --> 00:08:58.539 sit between helpful experimentation wanting to 00:08:58.539 --> 00:09:01.539 get involved, excited about the technology and 00:09:01.539 --> 00:09:04.399 drifting unwittingly into risky behavior that 00:09:04.399 --> 00:09:08.419 could expose organizations to security or compliance 00:09:08.419 --> 00:09:14.909 issues? Neil, that's a great question. As I go 00:09:14.909 --> 00:09:17.129 back to, you know, my head of security here and 00:09:17.129 --> 00:09:19.649 other places, my HR department, like all the 00:09:19.649 --> 00:09:22.110 trainings we went through, don't do this, don't, 00:09:22.169 --> 00:09:23.850 you know, and we all click through them. Let's 00:09:23.850 --> 00:09:26.690 face it. Like these days I just asked Claude 00:09:26.690 --> 00:09:28.629 to write a bot to click through for me. I'm going 00:09:28.629 --> 00:09:30.090 to read your first to admit it, right? And I'm 00:09:30.090 --> 00:09:34.230 sure you do too, right? But we haven't gotten 00:09:34.230 --> 00:09:37.070 the message and the training to sink in like 00:09:37.070 --> 00:09:39.710 we've done around phishing and we've done around, 00:09:39.889 --> 00:09:44.159 you know, other sorts of, you know, harassment 00:09:44.159 --> 00:09:46.259 and all these other sort of bad behaviors in 00:09:46.259 --> 00:09:47.919 the corporate workforce because it's too new. 00:09:48.460 --> 00:09:51.240 I think personally, AI can meaningfully boost 00:09:51.240 --> 00:09:53.580 productivity when it's used in a controlled and 00:09:53.580 --> 00:09:56.279 transparent way. In fact, we did a study at LaunchDarkly. 00:09:56.580 --> 00:10:00.200 We found that 94 % of the teams we surveyed actually 00:10:00.200 --> 00:10:04.269 report faster development. But 91 % of them say 00:10:04.269 --> 00:10:06.129 they've been more cautious about pushing those 00:10:06.129 --> 00:10:09.129 AI accelerated changes into production And that 00:10:09.129 --> 00:10:11.629 line is really crossed when developers start 00:10:11.629 --> 00:10:13.710 inputting sensitive code or data into tools that 00:10:13.710 --> 00:10:16.269 haven't been Approved or secured by the organization 00:10:16.269 --> 00:10:18.970 and it's it's to be honest. It's very subtle 00:10:18.970 --> 00:10:20.950 Yeah, so you can sit there and say I'm gonna 00:10:20.950 --> 00:10:23.450 write a tool that exports data from a and just 00:10:23.450 --> 00:10:26.049 moves it to data be in a pipeline You know, we 00:10:26.049 --> 00:10:28.470 we do this from our European instance into our 00:10:28.470 --> 00:10:31.750 American instance of our data platform for things 00:10:31.750 --> 00:10:35.190 that are business critical. And I'm super worried 00:10:35.190 --> 00:10:37.909 personally about being GDPR compliant, Dora compliant, 00:10:38.029 --> 00:10:40.549 all the things that our European customers expect 00:10:40.549 --> 00:10:43.210 of us. But you write a tool and it puts a data 00:10:43.210 --> 00:10:45.289 field in there and you didn't analyze that data 00:10:45.289 --> 00:10:47.529 field, right? Oh, it works on both sides and 00:10:47.529 --> 00:10:50.169 you didn't do that scrutiny. That's really dangerous. 00:10:50.690 --> 00:10:52.970 It becomes really risky when these AI generated 00:10:52.970 --> 00:10:55.110 outputs are used in production without the proper 00:10:55.110 --> 00:10:57.049 review, testing, or understanding of how they 00:10:57.049 --> 00:11:00.080 were created. You know more from this survey 00:11:00.080 --> 00:11:02.940 like I love the data we gathered here We've got 00:11:02.940 --> 00:11:06.639 81 % of the teams that knowingly ship risky code 00:11:06.639 --> 00:11:09.000 because of deadline pressures, right? It's that 00:11:09.000 --> 00:11:10.879 it's that developer who goes. Yeah, it works 00:11:10.879 --> 00:11:12.679 right and they don't know any they don't know 00:11:12.679 --> 00:11:14.679 any better They're not being malicious, right? 00:11:14.679 --> 00:11:16.620 They're just trying to get their job done. Like 00:11:16.620 --> 00:11:19.960 we all want to do every single day 83 % of them 00:11:19.960 --> 00:11:22.519 say that the releases containing the AI generated 00:11:22.519 --> 00:11:25.820 code are somewhat are very likely to cause those 00:11:25.820 --> 00:11:28.860 production issues out there So, you know, it's 00:11:28.860 --> 00:11:31.679 a scary time. And if you don't have the visibility 00:11:31.679 --> 00:11:35.039 or governance around how these tools are being 00:11:35.039 --> 00:11:38.919 used, then this becomes like an immediate compliance 00:11:38.919 --> 00:11:42.519 security concern. And we get 38 % of those teams 00:11:42.519 --> 00:11:44.980 spending more than a quarter of their time resolving 00:11:44.980 --> 00:11:47.519 incidents, which means faster code is not always 00:11:47.519 --> 00:11:51.179 more stable code. As an experienced engineering 00:11:51.179 --> 00:11:54.379 manager, I think about how teams break down their 00:11:54.379 --> 00:11:58.129 work. I want healthy organizations, number one, 00:11:58.350 --> 00:12:00.470 because I've worked in some companies where they're 00:12:00.470 --> 00:12:02.429 not about healthy organization, and I'm not going 00:12:02.429 --> 00:12:07.610 to name names. But I believe in hiring great 00:12:07.610 --> 00:12:10.950 people who want to work with me, want to do a 00:12:10.950 --> 00:12:13.389 good job, and are dedicated. I assume positive 00:12:13.389 --> 00:12:18.929 intent every day. I think as part of that, about 00:12:18.929 --> 00:12:21.970 25 % of the work of a team is generally what 00:12:21.970 --> 00:12:24.789 I call keep the lights on work. It's bug fixing. 00:12:25.269 --> 00:12:27.389 In the US, you don't get a tax credit for this. 00:12:27.649 --> 00:12:29.830 You can't get a write -off because it's not risky, 00:12:30.309 --> 00:12:31.970 whereas new development work gives you a tax 00:12:31.970 --> 00:12:36.470 credit in the US. That 25 % is sort of my bellwether 00:12:36.470 --> 00:12:40.889 of is the team moving fast enough or moving too 00:12:40.889 --> 00:12:42.690 fast? Because if that's up, let's say it pushes 00:12:42.690 --> 00:12:45.110 to 50%, they're throwing stuff into production 00:12:45.110 --> 00:12:46.870 that's just unsafe and they're creating their 00:12:46.870 --> 00:12:50.279 own problems. And then if it's too little, they're 00:12:50.279 --> 00:12:51.700 not moving fast enough at all, which means they're 00:12:51.700 --> 00:12:53.120 not really creating what we would call tech debt 00:12:53.120 --> 00:12:56.860 over the years. So even with safeguards in place, 00:12:57.620 --> 00:13:00.879 like 99 % of teams will tell you they have safeguards. 00:13:01.200 --> 00:13:03.799 70 % of those teams still roll back changes at 00:13:03.799 --> 00:13:06.620 least weekly, which says there's a huge gap between 00:13:06.620 --> 00:13:10.100 protection and real world outcomes. And these 00:13:10.100 --> 00:13:12.659 frequent hot fixes are just more or less becoming 00:13:12.659 --> 00:13:15.909 the norm now as this code goes ever faster. And 00:13:15.909 --> 00:13:18.289 then we hit other bottlenecks, like human scale 00:13:18.289 --> 00:13:21.409 bottlenecks, like reviewing the code, analyzing 00:13:21.409 --> 00:13:23.629 what's going on, debugging. A lot of that's still 00:13:23.629 --> 00:13:26.970 happening at human scale. So I think what I would 00:13:26.970 --> 00:13:29.250 take away from that is a lot of people are saying 00:13:29.250 --> 00:13:32.549 software's free now. Code generation has made 00:13:32.549 --> 00:13:34.600 software free. And that's not true. Software 00:13:34.600 --> 00:13:38.940 has made code generation free, but all of the 00:13:38.940 --> 00:13:42.139 day two behaviors around that and the validations 00:13:42.139 --> 00:13:44.360 and the verifications, that stuff still exists. 00:13:44.960 --> 00:13:48.179 It still takes time and now all you've done is, 00:13:49.299 --> 00:13:51.620 if I'm going to borrow the work of Eli Goldratt, 00:13:52.019 --> 00:13:53.940 you've moved the bottleneck in the pipeline. 00:13:54.799 --> 00:13:57.360 I love the goal. It's one of my favorite books 00:13:57.360 --> 00:14:00.559 of the last 40 years, I think. And you made such 00:14:00.559 --> 00:14:02.320 a good point there as well about the pressure 00:14:02.320 --> 00:14:03.980 that developers are feeling. I mean, it was, 00:14:04.220 --> 00:14:05.899 I don't know if you saw the viral video recently 00:14:05.899 --> 00:14:08.600 of Jensen Huang talking about, he'll be measuring 00:14:08.600 --> 00:14:11.519 performance by how many tokens the team are burning. 00:14:11.559 --> 00:14:14.759 And if they're not burning that much, they might 00:14:14.759 --> 00:14:16.659 appear on a radar. Did you see that? What did 00:14:16.659 --> 00:14:20.720 you take away from that? You know, we had our 00:14:20.720 --> 00:14:24.019 board meeting. last week and we were talking 00:14:24.019 --> 00:14:26.759 about this with the board and they're asking 00:14:26.759 --> 00:14:28.759 me too, what do I look at? How am I measuring 00:14:28.759 --> 00:14:31.919 developer productivity? There are companies out 00:14:31.919 --> 00:14:34.360 there, and this is not anything I'm measuring 00:14:34.360 --> 00:14:36.960 internally to be clear, but there are companies 00:14:36.960 --> 00:14:40.799 out there that expect to spend two times a developer's 00:14:40.799 --> 00:14:43.960 base salary in tokens per developer per year. 00:14:44.430 --> 00:14:47.909 Wow. So if I have a, let's just say a $300 ,000 00:14:47.909 --> 00:14:49.909 Silicon Valley, you know, mid level developer 00:14:49.909 --> 00:14:52.529 base salary, you know, somewhere in that range, 00:14:52.690 --> 00:14:54.809 right. They expect to spend $600 ,000 a year 00:14:54.809 --> 00:14:57.970 on tokens for that guy or girl. Right. That's 00:14:57.970 --> 00:15:01.149 crazy to me. Cause are you really getting the 00:15:01.149 --> 00:15:03.690 productivity out of that at the other end? Yeah. 00:15:03.809 --> 00:15:06.070 And you know, what I'm seeing here is you're 00:15:06.070 --> 00:15:07.450 getting the productivity, but you're getting 00:15:07.450 --> 00:15:09.990 the rework that goes with it. And that to me 00:15:09.990 --> 00:15:12.450 is, is, you know, if you go back to lean, it's 00:15:12.450 --> 00:15:16.509 waste. Yeah, in the process. Yeah, 100 % with 00:15:16.509 --> 00:15:18.830 you. And another stat I read elsewhere recently 00:15:18.830 --> 00:15:22.710 was that 70 % of teams are now using shadow AI 00:15:22.710 --> 00:15:26.049 tools to get work done. I mean, that just shows 00:15:26.049 --> 00:15:28.350 you the scale of what we're talking about here. 00:15:28.389 --> 00:15:31.610 But when teams are using unsanctioned AI tools 00:15:31.610 --> 00:15:34.870 without clear oversight. Organizations obviously 00:15:34.870 --> 00:15:37.490 can lose visibility into how decisions are being 00:15:37.490 --> 00:15:40.429 made or how code is generated. But what kind 00:15:40.429 --> 00:15:43.149 of risks does that create for engineering teams, 00:15:43.509 --> 00:15:46.250 especially when they're on no audit trails, runtime 00:15:46.250 --> 00:15:49.629 controls, or clear ownership of the final output? 00:15:50.129 --> 00:15:52.129 The me of 20 years ago finds that stuff quite 00:15:52.129 --> 00:15:54.509 scary, that all that stuff's missing. But tell 00:15:54.509 --> 00:15:57.700 me more about what you're seeing here. I get 00:15:57.700 --> 00:15:59.960 to be a little snarky because my mother and her 00:15:59.960 --> 00:16:01.620 family grew up in New York, so I get that little 00:16:01.620 --> 00:16:03.940 bit of underlying sarcasm in how I look at things. 00:16:04.200 --> 00:16:06.960 You can't fire Neil an AI model. You just can't. 00:16:07.299 --> 00:16:11.940 I can fire a human. That's the snarky way of 00:16:11.940 --> 00:16:17.000 saying it. But what it really boils down to is 00:16:17.000 --> 00:16:19.600 it creates an accountability gap. To be blunt, 00:16:19.600 --> 00:16:22.220 I don't want to fire a human. I value the hard 00:16:22.220 --> 00:16:24.279 work my folks put in and everywhere I've been, 00:16:24.320 --> 00:16:27.840 it's been that way. But you can't have an accountability 00:16:27.840 --> 00:16:30.120 gap in a healthy organization. And it didn't 00:16:30.120 --> 00:16:33.259 matter. Before we had AI in the organization, 00:16:33.419 --> 00:16:34.620 there were a lot of organizations that had no 00:16:34.620 --> 00:16:37.419 accountability. And this is where the business 00:16:37.419 --> 00:16:40.820 starts to falter. But with this, it gets even 00:16:40.820 --> 00:16:43.559 more critical. Teams can't trace who made a decision. 00:16:43.659 --> 00:16:47.139 They can't trace how output was generated. In 00:16:47.139 --> 00:16:52.179 the context of the EU AI Act, for instance, you 00:16:52.179 --> 00:16:56.100 have to have that. That's a huge risk to global 00:16:56.100 --> 00:16:58.559 revenues if you don't have that kind of traceability 00:16:58.559 --> 00:17:02.059 and that decisioning. And it's really hard to 00:17:02.059 --> 00:17:04.680 justify decisions around compliance, instant 00:17:04.680 --> 00:17:07.099 response, and to be honest, the most important 00:17:07.099 --> 00:17:09.299 thing, which is customer trust in a business. 00:17:09.859 --> 00:17:12.799 And then you get a security risk. You don't know 00:17:12.799 --> 00:17:14.440 where these tools are coming from. We're seeing 00:17:14.440 --> 00:17:16.660 poisoning events in some of these LLMs and some 00:17:16.660 --> 00:17:20.160 of these repos, open source repos out there that 00:17:20.160 --> 00:17:22.890 are coming about from this. If there's vulnerabilities 00:17:22.890 --> 00:17:24.970 that are introduced because this version got 00:17:24.970 --> 00:17:27.150 pulled down by an LLM and nobody checked it, 00:17:27.170 --> 00:17:28.809 it just works, right? It goes and selects the 00:17:28.809 --> 00:17:31.230 best tool it thinks it has. There's no clear 00:17:31.230 --> 00:17:33.589 understanding of those origins. That's a huge 00:17:33.589 --> 00:17:35.869 risk to my business and to my customer's business. 00:17:39.000 --> 00:17:41.619 I say honestly to my folks, I don't care if we 00:17:41.619 --> 00:17:43.539 leak our data as much as I don't ever want to 00:17:43.539 --> 00:17:45.720 leak my customers' data because I value that 00:17:45.720 --> 00:17:48.720 trust and that reputation more than I even value 00:17:48.720 --> 00:17:50.819 my own business because that's what we stand 00:17:50.819 --> 00:17:54.880 for. Without runtime controls, you get very flawed 00:17:54.880 --> 00:17:57.740 or very risky AI -generated code that goes to 00:17:57.740 --> 00:18:00.640 production unchecked at a prodigious rate that 00:18:00.640 --> 00:18:05.049 we've never seen before. And four and a half 00:18:05.049 --> 00:18:07.390 thousand miles away from you on the other side 00:18:07.390 --> 00:18:11.410 of the pond here, the EU AI Act is beginning 00:18:11.410 --> 00:18:15.029 to move and especially go from discussion to 00:18:15.029 --> 00:18:19.250 implementation. So how does Shadow AI complicate 00:18:19.250 --> 00:18:21.670 compliance for organisations that are operating 00:18:21.670 --> 00:18:25.069 here in the UK where I am and indeed across Europe? 00:18:25.650 --> 00:18:29.519 How do you see this evolving? I think it's really 00:18:29.519 --> 00:18:31.819 interesting. You guys got, you guys, I mean, 00:18:31.940 --> 00:18:33.799 I say you guys, but we sell in Europe, so we're 00:18:33.799 --> 00:18:38.099 there too. Let's be, let's be blind. We've got 00:18:38.099 --> 00:18:39.740 a couple of big deadlines to come up, right? 00:18:39.819 --> 00:18:42.200 The next one I think is early August, right? 00:18:42.259 --> 00:18:44.180 The second this year where there's a majority 00:18:44.180 --> 00:18:46.779 of those rules coming into force, including rules 00:18:46.779 --> 00:18:50.880 for super high risk AI systems under Annex 3, 00:18:50.960 --> 00:18:53.359 which are the transparency obligations. And then, 00:18:53.740 --> 00:18:56.180 all the full enforcement, and you've got to get 00:18:56.180 --> 00:18:58.579 these regulatory sandboxes, there's a lot of 00:18:58.579 --> 00:19:01.839 dates driving that roadmap. But it boils down 00:19:01.839 --> 00:19:04.140 to that transparency, traceability, and accountability, 00:19:04.220 --> 00:19:07.299 and all of that breaks down with Shadow AI. If 00:19:07.299 --> 00:19:09.619 you don't get visibility into those decisions 00:19:09.619 --> 00:19:11.259 that are being made, how are you going to have 00:19:11.259 --> 00:19:14.640 those high -risk compliance? How do you know 00:19:14.640 --> 00:19:16.460 you're in compliance with those high -risk actions? 00:19:16.839 --> 00:19:19.000 I think when most organizations come in and they 00:19:19.000 --> 00:19:22.259 start evaluating the risk classification across 00:19:22.259 --> 00:19:26.140 their organization, they're shocked. Actually, 00:19:26.220 --> 00:19:28.240 I'm American, I can make the joke of shock and 00:19:28.240 --> 00:19:32.059 awe. They have shock and awe at how many things 00:19:32.059 --> 00:19:36.079 fall in that high risk category. And then those 00:19:36.079 --> 00:19:38.319 fines, that non -compliance risk just goes through 00:19:38.319 --> 00:19:42.299 the roof with legal exposure and pretty hefty. 00:19:43.119 --> 00:19:45.849 I think, you know, everybody should be aware 00:19:45.849 --> 00:19:47.869 of what falls in that high -risk commercially 00:19:47.869 --> 00:19:51.130 relevant bucket, which is anybody using AI to 00:19:51.130 --> 00:19:55.670 screen, rank, match candidates, AI for credit 00:19:55.670 --> 00:19:58.450 scoring, for education, for critical infrastructure, 00:19:58.789 --> 00:20:01.430 law enforcement, health care. This is that full 00:20:01.430 --> 00:20:04.680 compliance regime. And so I'm looking to build 00:20:04.680 --> 00:20:07.119 systems around that and help folks manage risk. 00:20:07.599 --> 00:20:10.079 And I can't manage it up front. I'm not a frontier 00:20:10.079 --> 00:20:12.500 model builder, in some ways, thank goodness. 00:20:12.519 --> 00:20:14.839 I'm not a frontier model builder. Because then 00:20:14.839 --> 00:20:16.960 I would have been under this months, years ago, 00:20:17.140 --> 00:20:20.839 two years ago, I think. But my goal is when you 00:20:20.839 --> 00:20:24.000 discover something, let's help correct it. Let's 00:20:24.000 --> 00:20:27.000 shut it off right away. Let's get the remediation 00:20:27.000 --> 00:20:29.960 in place. So one of the things I actually like, 00:20:30.599 --> 00:20:33.329 and I've been around I lived in Europe for a 00:20:33.329 --> 00:20:38.049 while in the Nordics in Stockholm. So I'm familiar 00:20:38.049 --> 00:20:40.549 with the climate. One of the things I really 00:20:40.549 --> 00:20:43.890 like about this law is they're really about shepherding 00:20:43.890 --> 00:20:47.150 you toward compliance. If you sign up, you sign 00:20:47.150 --> 00:20:49.869 the compliance pledge, you do all of that, then 00:20:49.869 --> 00:20:51.690 they're more focused on remediation and getting 00:20:51.690 --> 00:20:55.690 this right. So the use of AI is healthy versus 00:20:55.690 --> 00:21:00.480 punitive. And I think, while the fines are large 00:21:00.480 --> 00:21:04.740 and scary, I think the push is generally in the 00:21:04.740 --> 00:21:06.599 right direction for how do we use this safely 00:21:06.599 --> 00:21:09.140 so it benefits everybody that we're doing business 00:21:09.140 --> 00:21:12.039 with. And so I admire that. In the U .S., we're 00:21:12.039 --> 00:21:13.960 still dealing with a patchwork quilt of laws, 00:21:15.099 --> 00:21:17.160 a Congress that's struggling to make sense of 00:21:17.160 --> 00:21:19.680 it, and various states, they're like, we can't 00:21:19.680 --> 00:21:22.759 wait that long to protect our folks from the 00:21:22.759 --> 00:21:27.789 use of this technology. And it's powerful. hidden 00:21:27.789 --> 00:21:32.049 bias, the amount of inference it can make through 00:21:32.049 --> 00:21:34.309 pattern matching across a data set that even 00:21:34.309 --> 00:21:36.109 the best of us could ever hope to keep in our 00:21:36.109 --> 00:21:40.470 heads. It's dramatic. And so I respect their 00:21:40.470 --> 00:21:44.130 attempt to try to bring some kind of sanity and 00:21:44.130 --> 00:21:46.049 some kind of security to it so that we can all 00:21:46.049 --> 00:21:49.029 benefit. This is a positive use of AI, but it's 00:21:49.029 --> 00:21:52.900 a lot of work. It really is adding another layer 00:21:52.900 --> 00:21:57.019 of complexity with the EU product liability directive 00:21:57.019 --> 00:21:58.900 and on that side of things a question I've got 00:21:58.900 --> 00:22:02.099 to ask you here is if an AI coding assistant 00:22:02.099 --> 00:22:04.500 may be introduced a vulnerability that later 00:22:04.500 --> 00:22:07.579 leads to a breach or regulatory failure where 00:22:07.579 --> 00:22:10.799 does the liability realistically sit in this 00:22:10.799 --> 00:22:13.779 kind of situation? Yeah, I mean that liability 00:22:13.779 --> 00:22:16.640 remains with the organization deploying the code 00:22:16.640 --> 00:22:19.299 This is the this is the companion piece to that 00:22:19.299 --> 00:22:22.660 AI AI act, you know in some ways It's even the 00:22:22.660 --> 00:22:25.680 sharper commercial risk AI act tells you here's 00:22:25.680 --> 00:22:28.079 what I'm gonna build Here's how I'm going to 00:22:28.079 --> 00:22:30.980 govern it and this PLD tells you exactly what 00:22:30.980 --> 00:22:33.500 happens when something goes awry, you know, you're 00:22:33.500 --> 00:22:37.000 now Let me say this before I say that like we've 00:22:37.000 --> 00:22:39.160 talked for years about how software is a factory 00:22:39.959 --> 00:22:42.619 Right Agile was a great, you know, I mean not 00:22:42.619 --> 00:22:45.259 that I necessarily subscribed to everything Kent 00:22:45.259 --> 00:22:48.500 Beck and and and folks like but to but Agile 00:22:48.500 --> 00:22:50.039 was really turning it into the factory What's 00:22:50.039 --> 00:22:53.079 that pipeline? Right and then you got folks like 00:22:53.079 --> 00:22:54.680 Gene Kim who came in with his brilliant book 00:22:54.680 --> 00:22:57.579 The Phoenix Project You know, which was which 00:22:57.579 --> 00:22:59.059 was a lot of that theory of constraints that 00:22:59.059 --> 00:23:03.859 he like old rat put out there This turns turns 00:23:04.190 --> 00:23:06.190 you know this law explicitly says essentially 00:23:06.190 --> 00:23:09.410 if you're producing software you are now a manufacturer 00:23:09.410 --> 00:23:11.569 in the legal sense and the legal protections 00:23:11.569 --> 00:23:13.329 that software companies historically enjoyed 00:23:13.329 --> 00:23:15.289 the ambiguity about whether software was really 00:23:15.289 --> 00:23:17.970 even a product a high burden on the plaintiffs 00:23:17.970 --> 00:23:21.170 limited damages those have all been systemically 00:23:21.170 --> 00:23:24.390 dismantled so any failures here point to gaps 00:23:24.390 --> 00:23:26.569 in governance testing and oversight not just 00:23:26.569 --> 00:23:29.619 tool usage We get a lot of questions around due 00:23:29.619 --> 00:23:31.740 diligence then, right? And so you want to go 00:23:31.740 --> 00:23:33.500 in your organization and look at, were you healthy 00:23:33.500 --> 00:23:35.799 about, are these tools approved? Are they understood? 00:23:36.019 --> 00:23:38.900 Are they used within clear guard rails even, 00:23:39.220 --> 00:23:42.700 right? It doesn't shift responsibility at all. 00:23:42.720 --> 00:23:46.240 In fact, if anything, it increases the need for 00:23:46.240 --> 00:23:49.900 strong controls. I think a lot of us in, you 00:23:49.900 --> 00:23:53.259 know, in engineering, a lot of us who have grown 00:23:53.259 --> 00:23:59.680 up with this technology forward bent of always 00:23:59.680 --> 00:24:01.779 just being a builder that's frustrating for us 00:24:01.779 --> 00:24:03.599 because it feels like we're pulling the brakes 00:24:03.599 --> 00:24:06.519 on something that's intended to help us go faster 00:24:06.519 --> 00:24:08.640 right it's almost like you're you're getting 00:24:08.640 --> 00:24:11.819 in your your sports car you know and flooring 00:24:11.819 --> 00:24:14.819 it and and the guy in the in I was about to say 00:24:14.819 --> 00:24:16.880 the right seat but the left seat in your case, 00:24:17.119 --> 00:24:18.720 right? Still got his hand on the parking brake. 00:24:19.299 --> 00:24:22.339 Yeah, it certainly feels that way sometimes. 00:24:22.339 --> 00:24:25.339 And then if we also look at the traditional IT 00:24:25.339 --> 00:24:28.660 governance models that we were both seeing since 00:24:28.660 --> 00:24:30.680 the start of our careers, they were designed 00:24:30.680 --> 00:24:33.839 for software built and deployed by human teams. 00:24:34.460 --> 00:24:36.259 But those models are obviously going to struggle 00:24:36.259 --> 00:24:39.720 when AI systems and AI generated code become 00:24:39.720 --> 00:24:42.000 part of the development workflow too, right? 00:24:42.420 --> 00:24:46.430 What are you seeing here? You know, we ran a 00:24:46.430 --> 00:24:49.990 really interesting experiment the other day. 00:24:50.049 --> 00:24:53.710 One of my teams, I'm gonna give one of my wonderful 00:24:53.710 --> 00:24:57.150 engineering managers, Carmen Kwan, a shout out 00:24:57.150 --> 00:25:00.549 for doing it. She was amazing. She ran a bug, 00:25:00.549 --> 00:25:03.150 we called it bug day. And so it was a bug bash 00:25:03.150 --> 00:25:06.150 and you've done these forever. But we did it 00:25:06.150 --> 00:25:09.130 all with tools. And we did it all without the 00:25:09.130 --> 00:25:11.009 engineers actually writing the code, but prompting 00:25:11.009 --> 00:25:13.890 the tools with a lot of human oversight. And 00:25:13.890 --> 00:25:17.269 what we discovered, it wasn't surprising to me, 00:25:17.369 --> 00:25:19.349 but it just reinforced it, which is all of these 00:25:19.349 --> 00:25:23.210 processes are designed for deterministic human 00:25:23.210 --> 00:25:25.990 -driven systems where the risks are visible and 00:25:25.990 --> 00:25:28.789 contained. And the bottleneck, again, just shifted. 00:25:28.930 --> 00:25:31.150 It became reviewing and oversight of these systems. 00:25:31.630 --> 00:25:33.690 So we've always placed the focus on things like 00:25:33.690 --> 00:25:36.509 pre -deployment checks, which assumes human ownership 00:25:36.509 --> 00:25:40.349 of decisions. And all of a sudden, AI is introducing 00:25:40.349 --> 00:25:43.569 non -deterministic behavior and logic within 00:25:43.569 --> 00:25:46.690 workflows, often without clear ownership. In 00:25:46.690 --> 00:25:49.509 fact, the reason we like AI, the reason we enjoy 00:25:49.509 --> 00:25:51.309 interacting with it is it's somewhat random. 00:25:52.150 --> 00:25:54.390 There's a temperature baked into a model, which 00:25:54.390 --> 00:25:58.890 is exactly how random this thing is. I say random 00:25:58.890 --> 00:26:01.170 because this is a math, right? It's a bunch of 00:26:01.170 --> 00:26:04.710 matrix multiplies. But some people would say 00:26:04.710 --> 00:26:07.970 the... Creativity, which I think I don't like 00:26:07.970 --> 00:26:11.289 to anthropomorphize AI personally, but I get 00:26:11.289 --> 00:26:15.069 it, right? We like that as a human. That's the 00:26:15.069 --> 00:26:17.589 part of being human. All of us who've been in 00:26:17.589 --> 00:26:19.390 this industry for a long time probably went into 00:26:19.390 --> 00:26:22.009 it because we like determinism. The machine does 00:26:22.009 --> 00:26:24.710 exactly what I said it did, unless somebody else 00:26:24.710 --> 00:26:29.859 broke it and put a bug in. And I think part of 00:26:29.859 --> 00:26:32.380 our business in LaunchDarkly with these runtime 00:26:32.380 --> 00:26:35.279 controls and checks is about bringing predictability 00:26:35.279 --> 00:26:40.079 and determinism to a probabilistic system by 00:26:40.079 --> 00:26:42.660 putting guardrails in place that create, I mean, 00:26:42.660 --> 00:26:44.160 I'm gonna say this, I'm not gonna create stochastic 00:26:44.160 --> 00:26:46.440 processes that are repeatable by nudging things 00:26:46.440 --> 00:26:49.119 back on course or course correcting along the 00:26:49.119 --> 00:26:51.380 way. So it takes that creativity, leaves it there 00:26:51.380 --> 00:26:53.539 and helps bring it back to steer to the outcomes 00:26:53.539 --> 00:26:57.009 you want. You know, this requires ongoing real 00:26:57.009 --> 00:27:00.650 -time control. So monitoring, guardrails, rollback, 00:27:00.890 --> 00:27:04.450 kill switches, et cetera, and not just approval 00:27:04.450 --> 00:27:07.150 processes in there. It's not about saying, yes, 00:27:07.170 --> 00:27:08.789 this is good to go. In fact, I can get another 00:27:08.789 --> 00:27:10.950 LLM to do that, and then what do I need to be 00:27:10.950 --> 00:27:13.470 around for it? But I want to be able to shut 00:27:13.470 --> 00:27:16.549 this thing off when it goes haywire. And it doesn't 00:27:16.549 --> 00:27:19.690 necessarily even go haywire, Neil, because... 00:27:19.640 --> 00:27:21.619 you know, I made a logic error or the AI made 00:27:21.619 --> 00:27:24.240 a logic error. Sometimes it goes viral and all 00:27:24.240 --> 00:27:26.799 of a sudden I'm burning tokens because I don't 00:27:26.799 --> 00:27:29.279 know if you saw the Chipotle meme last week where 00:27:29.279 --> 00:27:32.299 they were used. Last week, week before, someone 00:27:32.299 --> 00:27:34.559 discovered that the Chipotle chat bot could write 00:27:34.559 --> 00:27:36.400 Python for you instead of making your burrito 00:27:36.400 --> 00:27:39.759 bowl. So it was super interesting. I'm like, 00:27:39.920 --> 00:27:42.079 why pay for a clogged subscription when I could 00:27:42.079 --> 00:27:46.619 get Chipotle to do it for me? Yeah, I mean it's 00:27:46.619 --> 00:27:48.599 a you know these it was never intentional But 00:27:48.599 --> 00:27:51.059 you'd want to shut that off and in you know 200 00:27:51.059 --> 00:27:55.180 milliseconds or less, please So, you know governance 00:27:55.180 --> 00:27:56.940 the point that I'm getting at though is right 00:27:56.940 --> 00:27:59.400 is governance has to evolve right? It's no longer 00:27:59.400 --> 00:28:03.200 static approval But it changes into continuous 00:28:03.200 --> 00:28:07.279 control embedded in your delivery systems I think 00:28:07.279 --> 00:28:10.700 the best organizations Did this in the move to 00:28:10.700 --> 00:28:14.339 the cloud they went from static audits? and static 00:28:14.339 --> 00:28:19.160 remediations to things like in an AWS ecosystem, 00:28:19.339 --> 00:28:21.480 for instance, looking at CloudTrail CloudWatch, 00:28:21.559 --> 00:28:23.740 putting lambdas in to remediate security issues 00:28:23.740 --> 00:28:26.259 in real time. And so your check was that this 00:28:26.259 --> 00:28:30.359 lambda fire when I unencrypted an S3 bucket and 00:28:30.359 --> 00:28:32.819 force it back to encryption, instead of checking, 00:28:32.920 --> 00:28:35.519 are all my buckets encrypted? I think AI is the 00:28:35.519 --> 00:28:37.720 same way. And so the best organizations that 00:28:37.720 --> 00:28:39.559 are going to function and make use of this technology 00:28:39.559 --> 00:28:42.750 in a safe way, have those automated remediations 00:28:42.750 --> 00:28:44.750 and automated feature flags and automated kill 00:28:44.750 --> 00:28:49.869 switches. Internally here, we're doing a project 00:28:49.869 --> 00:28:52.349 internally where all of our PRs are automatically 00:28:52.349 --> 00:28:56.250 examined by an LLM. They're wrapped in feature 00:28:56.250 --> 00:29:00.730 flags before they go into production. There's 00:29:00.730 --> 00:29:03.130 observability data created as part of this process. 00:29:03.369 --> 00:29:05.089 So that's a huge augmentation. The things we 00:29:05.089 --> 00:29:07.089 would teach a developer how to go from a computer 00:29:07.089 --> 00:29:10.240 scientist to a software engineer, because the 00:29:10.240 --> 00:29:12.940 two are very different in my opinion. Moving 00:29:12.940 --> 00:29:14.240 from computer science to software engineers, 00:29:14.380 --> 00:29:17.240 learning about how do I do day two? Day one they 00:29:17.240 --> 00:29:19.720 teach in school, how do I write code? Day two 00:29:19.720 --> 00:29:23.299 is how do I actually make this work for the 54 00:29:23.299 --> 00:29:25.940 trillion flag evaluations that we're doing every 00:29:25.940 --> 00:29:29.220 single day across the planet? And that's a learning 00:29:29.220 --> 00:29:30.880 curve. And now we've been able to help augment 00:29:30.880 --> 00:29:33.200 that with the use of these tools to say, these 00:29:33.200 --> 00:29:35.339 are the behaviors that we codify in our organization 00:29:35.339 --> 00:29:37.500 that we've learned through years of operating 00:29:37.500 --> 00:29:39.880 experience. This is gonna help teach you how 00:29:39.880 --> 00:29:44.190 to do that or do it for you on day one. And I 00:29:44.190 --> 00:29:45.789 always try and give everyone listening a few 00:29:45.789 --> 00:29:49.390 valuable takeaways. So from a technical perspective, 00:29:49.450 --> 00:29:51.990 how can somebody listening in their organization 00:29:51.990 --> 00:29:55.390 better increase visibility into how those AI 00:29:55.390 --> 00:29:57.869 tools are being used inside live systems? So 00:29:57.869 --> 00:30:01.130 they can maintain that element of control, but 00:30:01.130 --> 00:30:04.410 doing so without slowing innovation. And I understand 00:30:04.410 --> 00:30:08.009 it is a notoriously tricky balance, but any tips 00:30:08.009 --> 00:30:14.500 or advice on how to do that? AI moves faster 00:30:14.500 --> 00:30:19.519 than human scale. This is scary to a lot of people, 00:30:19.660 --> 00:30:21.539 because I've never been on a hamster wheel and 00:30:21.539 --> 00:30:24.500 run this fast in my life, and felt like I wasn't 00:30:24.500 --> 00:30:28.720 moving around the cage. We're in an arms race, 00:30:29.460 --> 00:30:34.599 and I don't use that term lightly. I'm very, 00:30:34.599 --> 00:30:36.579 very protective of it, but I think this is a 00:30:36.579 --> 00:30:40.400 true arms race that you're not going to be able 00:30:40.400 --> 00:30:43.680 to get out of. And so the way that you win that 00:30:43.680 --> 00:30:47.180 is you make AI usage visible by design across 00:30:47.180 --> 00:30:50.980 the development lifecycle. And I think there 00:30:50.980 --> 00:30:52.920 are a lot of small companies out there trying 00:30:52.920 --> 00:30:56.359 to get audit trails in place for their usage 00:30:56.359 --> 00:30:58.140 where it's not just the outcomes, right? We get, 00:30:58.380 --> 00:31:01.039 every time it checks into GitHub, like I get 00:31:01.039 --> 00:31:04.240 a history of what went in there, but it's understanding 00:31:04.240 --> 00:31:07.450 the why of how those decisions were made. you 00:31:07.450 --> 00:31:10.650 have to embed governance and controls directly 00:31:10.650 --> 00:31:13.430 into the delivery workflow. So all of these AI 00:31:13.430 --> 00:31:15.549 -generated changes are tracked and reviewed, 00:31:15.930 --> 00:31:19.509 and the attribution to the author has to happen 00:31:19.509 --> 00:31:23.430 in there. And it's nice. Like, one of my projects 00:31:23.430 --> 00:31:26.109 here right now is, what does our internal development 00:31:26.109 --> 00:31:28.809 platform look like in a year? How do we enable 00:31:28.809 --> 00:31:30.890 developers so this stuff is baked in there so 00:31:30.890 --> 00:31:32.869 they don't have to think about it? They just 00:31:32.869 --> 00:31:36.910 have to do. You need to have incredibly good 00:31:36.910 --> 00:31:39.630 runtime observability for this to work. You have 00:31:39.630 --> 00:31:43.190 to monitor how these AI influence features behave 00:31:43.190 --> 00:31:47.109 in production. In light of our EU AI laws that 00:31:47.109 --> 00:31:49.390 we were talking about a minute ago, that's even 00:31:49.390 --> 00:31:52.150 more important that you use techniques and software 00:31:52.150 --> 00:31:55.029 packages to monitor how these are behaving in 00:31:55.029 --> 00:31:57.950 production so that when it comes down to defending 00:31:57.950 --> 00:32:01.710 yourself, you have a commercially reasonable 00:32:01.710 --> 00:32:05.089 answer to. I did what was, I tried to do the 00:32:05.089 --> 00:32:07.329 right thing here and if we broke it, we can correct 00:32:07.329 --> 00:32:09.970 it. But we did everything in our power to figure 00:32:09.970 --> 00:32:13.849 out that it was the right answer. You need guardrails 00:32:13.849 --> 00:32:16.470 as well, approval layers, policy enforcement, 00:32:16.750 --> 00:32:19.910 restrictions on sensitive data. So your internal 00:32:19.910 --> 00:32:22.490 data classifications and governance become even 00:32:22.490 --> 00:32:24.289 more important and especially when you're running. 00:32:24.569 --> 00:32:27.529 you know, data lakes or, you know, modern companies 00:32:27.529 --> 00:32:29.089 don't even run data lakes anymore, right? We 00:32:29.089 --> 00:32:32.410 run data fabrics where data is a product that 00:32:32.410 --> 00:32:35.990 your individual departments or services or products 00:32:35.990 --> 00:32:38.369 actually just vend internally. Now you have a 00:32:38.369 --> 00:32:40.569 distributed governance project or problem to 00:32:40.569 --> 00:32:43.029 deal with. And so how do you classify, protect 00:32:43.029 --> 00:32:45.609 that, put the guardrails in place? So, you know, 00:32:45.670 --> 00:32:48.849 my LLMs, my agents here, every morning I run 00:32:48.849 --> 00:32:50.990 agents. I'm sure you do too, Neil, at this point, 00:32:51.049 --> 00:32:54.240 right? My agents come through my emails for me. 00:32:54.380 --> 00:32:55.940 They go through my daily schedule. They do all 00:32:55.940 --> 00:32:57.859 this and they have some pretty pretty privileged 00:32:57.859 --> 00:33:00.619 access. It's not like Let's be clear. I'm not 00:33:00.619 --> 00:33:04.000 an open claw like Just YOLO the world, right? 00:33:04.980 --> 00:33:06.839 You can have my checkbook my Amazon account, 00:33:06.839 --> 00:33:10.509 please have it all No, but in a more serious 00:33:10.509 --> 00:33:13.049 way though, it does have access to some of the 00:33:13.049 --> 00:33:15.650 things with the minimum spanning set of permissions 00:33:15.650 --> 00:33:17.470 I could come up with, but it still needs some 00:33:17.470 --> 00:33:19.210 sorts of privileged access. I just have audit 00:33:19.210 --> 00:33:21.309 trails on it. And we want to make sure that there's 00:33:21.309 --> 00:33:24.390 policy enforcement restrictions. There's a governance 00:33:24.390 --> 00:33:26.930 process. So as I build these, as my engineers 00:33:26.930 --> 00:33:29.759 build these, A lot of them are skills internally 00:33:29.759 --> 00:33:32.259 and we review them together and then we share 00:33:32.259 --> 00:33:34.619 them so that you don't have to have somebody 00:33:34.619 --> 00:33:36.500 else take another attempt at rewriting it as 00:33:36.500 --> 00:33:38.900 easy as it is and maybe get the security permissions 00:33:38.900 --> 00:33:41.539 wrong. And people are grateful for that. We're 00:33:41.539 --> 00:33:43.579 sharing best practices, we're sharing research 00:33:43.579 --> 00:33:45.559 tools, we're sharing all of our internal operating 00:33:45.559 --> 00:33:49.519 agents. And there's a couple hundred to a thousand 00:33:49.519 --> 00:33:53.839 already running around here. They're all... They're 00:33:53.839 --> 00:33:55.619 governed, right? There's a check in an audit 00:33:55.619 --> 00:33:57.480 trail and people are constantly looking at them. 00:33:57.700 --> 00:33:59.220 And then the last bit of this is you've got to 00:33:59.220 --> 00:34:01.319 ensure that there's rollback mechanisms and kill 00:34:01.319 --> 00:34:04.539 switches, runtime kill switches, flags around 00:34:04.539 --> 00:34:07.240 that to defend yourself and really gain control 00:34:07.240 --> 00:34:11.340 when issues arise. And those issues could be 00:34:11.340 --> 00:34:14.179 costing, they could be fairness, they could be 00:34:14.179 --> 00:34:17.400 the jailbreaking of coding Python instead of 00:34:17.400 --> 00:34:22.480 making burritos. You want to make sure that as 00:34:22.480 --> 00:34:26.599 you have probabilistic behavior that is, I mean, 00:34:26.679 --> 00:34:29.360 non -deterministic behavior, really, that if 00:34:29.360 --> 00:34:31.380 you don't like how it's behaving, you shut it 00:34:31.380 --> 00:34:33.239 off. It's like, in some cases, it's like, you 00:34:33.239 --> 00:34:35.360 know, taking your toddler out of the classroom 00:34:35.360 --> 00:34:37.039 and giving them a timeout once in a while, but 00:34:37.039 --> 00:34:38.920 you got to be able to reach in there and do it. 00:34:39.800 --> 00:34:41.320 Hopefully we do it before they're all sentient, 00:34:41.679 --> 00:34:46.650 right? No one shut me off? I'm sorry Dave, I 00:34:46.650 --> 00:34:49.510 can't do that. Yes, I was going to say it was 00:34:49.510 --> 00:34:53.250 getting flashbacks of HAL in 2001 there. And 00:34:53.250 --> 00:34:57.170 finally, before I let you go, four boards, CISOs, 00:34:57.449 --> 00:34:59.710 DevOps and other leaders that are listening today 00:34:59.710 --> 00:35:02.199 around the world. What questions should they 00:35:02.199 --> 00:35:05.840 be asking right now about AI governance, accountability, 00:35:06.119 --> 00:35:10.059 et cetera, to avoid security compliance and liability 00:35:10.059 --> 00:35:12.659 exposure in the years ahead? Because right now 00:35:12.659 --> 00:35:14.760 everyone's excited about the new tools, et cetera. 00:35:15.179 --> 00:35:17.340 But the so -called boring stuff, the important 00:35:17.340 --> 00:35:20.320 things that could bite you later on is very real, 00:35:20.639 --> 00:35:24.960 that threat is. So any advice there? Yeah. Are 00:35:24.960 --> 00:35:26.360 they asking you or are they going to ask Claude? 00:35:30.300 --> 00:35:33.800 I'm kidding on that one. Look, it's a really 00:35:33.800 --> 00:35:37.739 good question. This comes down in my not so humble 00:35:37.739 --> 00:35:40.579 opinion to the difference between wisdom and 00:35:40.579 --> 00:35:45.300 knowledge, which is you need to have the experience 00:35:45.300 --> 00:35:48.860 and the wisdom to really raise this at the senior 00:35:48.860 --> 00:35:52.239 level and say, number one, do I have visibility 00:35:52.239 --> 00:35:55.119 into where AI is being used across our teams? 00:35:55.920 --> 00:36:00.360 And I think if you spend some time and just ideate 00:36:00.360 --> 00:36:03.920 on this, be curious about it, ask the questions, 00:36:04.019 --> 00:36:06.019 you're gonna find it's used everywhere. Everybody's 00:36:06.019 --> 00:36:08.500 got the, I don't know, what are we calling it? 00:36:08.699 --> 00:36:12.980 It's the little star, right? I forget, what are 00:36:12.980 --> 00:36:14.960 we calling it in the industry now? I've even 00:36:14.960 --> 00:36:17.599 forgotten. The AI token in the corner there that 00:36:17.599 --> 00:36:20.659 just blips. Everything has it now. So you say, 00:36:20.780 --> 00:36:23.460 well, you know, I'm not using it to write my... 00:36:23.610 --> 00:36:25.610 you know, spreadsheet or whatever. Oh, yeah, 00:36:25.610 --> 00:36:28.989 you are. It's absolutely there. So I think you'll 00:36:28.989 --> 00:36:31.289 be surprised. I think the next question you want 00:36:31.289 --> 00:36:33.750 to ask is who is ultimately accountable for the 00:36:33.750 --> 00:36:37.489 AI generated outputs? And how do you how do you 00:36:37.489 --> 00:36:41.849 create the attribution back to that person? The 00:36:41.849 --> 00:36:43.570 next question is, can we trace and explain those 00:36:43.570 --> 00:36:46.639 decisions or code if we if we needed? And I know 00:36:46.639 --> 00:36:49.460 our chief legal officers definitely want to see 00:36:49.460 --> 00:36:51.900 that. But even as an engineering manager, I want 00:36:51.900 --> 00:36:54.219 to do that because it helps me make the architectural 00:36:54.219 --> 00:36:56.880 decisions about how my product looks and grows 00:36:56.880 --> 00:36:59.619 and helps make the business decisions of these 00:36:59.619 --> 00:37:01.780 are the two use cases and we have to pick a path. 00:37:02.840 --> 00:37:04.719 The next question is, of course, do we have runtime 00:37:04.719 --> 00:37:06.820 controls like rollback or kill switches in place? 00:37:07.500 --> 00:37:09.400 That's absolutely critical to be able to make 00:37:09.400 --> 00:37:13.320 split -second decisions. Code still takes time 00:37:13.320 --> 00:37:17.719 to ship. putting it out there, enabling it in 00:37:17.719 --> 00:37:19.960 production with runtime controls is all the more 00:37:19.960 --> 00:37:21.900 valuable. You can't just ship forward or roll 00:37:21.900 --> 00:37:24.219 back. You have to have a runtime kill switch 00:37:24.219 --> 00:37:28.619 in place. Are my teams using AI within the approved 00:37:28.619 --> 00:37:30.539 guardrails, especially with the sensitive data 00:37:30.539 --> 00:37:34.460 and how am I protecting that data? We're doing 00:37:34.460 --> 00:37:36.460 all of the typical endpoint threat detection 00:37:36.460 --> 00:37:39.440 and response on our machines here like you would 00:37:39.440 --> 00:37:41.739 do. And we're watching who's connecting and how 00:37:41.739 --> 00:37:43.420 they're connecting to some of these services. 00:37:44.199 --> 00:37:46.119 And we do it not because I don't want people 00:37:46.119 --> 00:37:48.099 to use it, but because I want them to use it. 00:37:48.329 --> 00:37:50.590 govern through our corporate, you know, our corporate 00:37:50.590 --> 00:37:54.849 account with with Anthropic or OpenAI or Gemini 00:37:54.849 --> 00:37:57.590 or any of the tools you want versus the personal 00:37:57.590 --> 00:37:59.309 account that people were so eager to get started 00:37:59.309 --> 00:38:02.070 with. And the benefit is I'll pay for your tokens 00:38:02.070 --> 00:38:04.989 internally when you do it, right, which it's 00:38:04.989 --> 00:38:08.449 not just stuff isn't cheap. The next question, 00:38:08.449 --> 00:38:10.369 of course, is how are we evolving our culture 00:38:10.369 --> 00:38:13.110 and bringing people with us to ensure that AI 00:38:13.110 --> 00:38:15.710 is being applied responsibly, fairly and with 00:38:15.710 --> 00:38:18.670 the right checks in place? And we're, you know, 00:38:18.690 --> 00:38:20.929 we're highlighting internally all the great projects 00:38:20.929 --> 00:38:23.409 people are doing. We're doing these partnerships 00:38:23.409 --> 00:38:25.610 and training. I want everybody to come along. 00:38:25.750 --> 00:38:28.730 This is an inevitability. You can't dig your 00:38:28.730 --> 00:38:31.309 heels in and be a Luddite for it, whether you 00:38:31.309 --> 00:38:33.550 want to or not. As much as I'd love to love to 00:38:33.550 --> 00:38:35.809 go back to the days of, you know, walking out 00:38:35.809 --> 00:38:37.989 of the office at five o 'clock and shipping on 00:38:37.989 --> 00:38:40.769 a floppy disk. It's not going to happen, right? 00:38:40.849 --> 00:38:44.050 We're 24 by seven. I mean, and now it's faster. 00:38:44.369 --> 00:38:46.929 So. you know, can we bring people with us into 00:38:46.929 --> 00:38:50.510 that reality in a way that's comforting to them 00:38:50.510 --> 00:38:54.469 and in a way that helps them feel empowered to 00:38:54.469 --> 00:38:57.849 be better with these tools and grow. And when 00:38:57.849 --> 00:39:02.110 they do, I mean, I'm impressed by every single 00:39:02.110 --> 00:39:03.869 engineer in the organization that's shown me 00:39:03.869 --> 00:39:05.710 something using these tools where I'm like, that's 00:39:05.710 --> 00:39:08.469 really clever. I'm copying that or I'm posting 00:39:08.469 --> 00:39:11.710 about that or, you know, and it's great to see. 00:39:12.279 --> 00:39:14.179 And then the last question I have is, are we 00:39:14.179 --> 00:39:16.239 spending enough tokens on the generative phase 00:39:16.239 --> 00:39:18.820 versus inference? And how are we controlling 00:39:18.820 --> 00:39:21.159 the costs around this? I think that's a huge 00:39:21.159 --> 00:39:24.739 question. I think AI is still in the subsidy 00:39:24.739 --> 00:39:27.639 phase. And so there'll be a reckoning where this 00:39:27.639 --> 00:39:30.559 stuff gets a lot more expensive in the future. 00:39:30.739 --> 00:39:33.099 We're seeing this through rising energy costs. 00:39:34.139 --> 00:39:35.960 A lot of arguments, at least here, states that 00:39:35.960 --> 00:39:39.360 about who pays for electricity, hardware costs. 00:39:39.440 --> 00:39:42.360 And while we still have Processor power and inference 00:39:42.360 --> 00:39:45.559 power accelerating right? It's not free in the 00:39:45.559 --> 00:39:47.719 in the rare earth minerals that we min mine for 00:39:47.719 --> 00:39:51.420 this In the raw resources the land the building 00:39:51.420 --> 00:39:53.500 that things that still have to happen to make 00:39:53.500 --> 00:39:59.920 it possible I Monitor token spend I monitor AI 00:39:59.920 --> 00:40:02.900 based check -ins, but to me they're lagging indicators 00:40:02.900 --> 00:40:05.699 of adoption. They're not metrics to optimize 00:40:05.699 --> 00:40:08.219 I don't want to ever tell my developer you to 00:40:08.219 --> 00:40:11.909 spend two times your salary in in Tokens, please, 00:40:12.269 --> 00:40:14.230 wrong incentives, right? That's a perverse set 00:40:14.230 --> 00:40:17.309 of incentives for somebody. Just like 80 % of 00:40:17.309 --> 00:40:19.110 my check -ins need to be AI. Great, I check in 00:40:19.110 --> 00:40:21.670 one line at a time and now my number goes up. 00:40:22.070 --> 00:40:24.670 That's not the right answer. The right answer 00:40:24.670 --> 00:40:28.070 is measuring the outcomes and measuring the backlog. 00:40:29.570 --> 00:40:31.449 I'm gonna say I don't look at backlogs anymore. 00:40:32.090 --> 00:40:33.789 We used to say, okay, what's the backlog for 00:40:33.789 --> 00:40:35.710 this team of engineers? It's three months, six 00:40:35.710 --> 00:40:38.460 months, nine months. It's changed. An engineer 00:40:38.460 --> 00:40:42.579 can supervise agent teams. HBR tells me it's 00:40:42.579 --> 00:40:45.000 about four is the optimum number of agents to 00:40:45.000 --> 00:40:46.400 supervise at any given point in time. Any more 00:40:46.400 --> 00:40:48.400 than that, your context switching kind of destroys 00:40:48.400 --> 00:40:51.420 your productivity. So my question to my engineers 00:40:51.420 --> 00:40:53.800 is are you managing those four agent teams? If 00:40:53.800 --> 00:40:57.300 not, why not four, right? And then is the project 00:40:57.300 --> 00:41:01.000 that's on the backlog something that has a positive 00:41:01.000 --> 00:41:04.670 ROI where the token spend for the outcome and 00:41:04.670 --> 00:41:07.630 the time and the context switching has a positive 00:41:07.630 --> 00:41:09.690 return for our business some way or another. 00:41:10.130 --> 00:41:12.449 And that's all that matters anymore, because 00:41:12.449 --> 00:41:15.489 I can do things that I never could do before, 00:41:15.510 --> 00:41:17.150 because I wouldn't want to invest 18 months of 00:41:17.150 --> 00:41:19.869 engineering time to do it. We had an 18 -month 00:41:19.869 --> 00:41:22.309 project here, for instance, we did in eight weeks, 00:41:22.550 --> 00:41:24.440 or we're just finishing. And it's going to be 00:41:24.440 --> 00:41:25.480 eight weeks for something that would have been 00:41:25.480 --> 00:41:28.199 18 months prior to this. And I love it. And the 00:41:28.199 --> 00:41:30.219 token spends not cheap. Like I won't tell you 00:41:30.219 --> 00:41:32.659 the numbers, but it's a lot cheaper than 18 months 00:41:32.659 --> 00:41:34.300 of engineering time that would have done it. 00:41:34.780 --> 00:41:37.840 And I love that. I love that change because now 00:41:37.840 --> 00:41:40.800 I've gone, I said I was nerdy. I'm not right. 00:41:40.860 --> 00:41:42.860 I was always been a theoretician. Like I love 00:41:42.860 --> 00:41:45.039 the theoretical. I love the proofs. I love, you 00:41:45.039 --> 00:41:47.159 know, spending my time with the journals, reading 00:41:47.159 --> 00:41:50.119 the papers. I have to be an experimentalist now. 00:41:50.739 --> 00:41:52.920 Because instead of spending a lot of time on 00:41:52.920 --> 00:41:55.300 theory, we just try it. And we can measure it 00:41:55.300 --> 00:41:57.099 really quickly. We've got runtime controls to 00:41:57.099 --> 00:42:00.380 shut it off. And if I throw it away, OK, I spent 00:42:00.380 --> 00:42:02.159 a few thousand dollars in tokens to build it, 00:42:02.159 --> 00:42:07.619 fine. So change your mindset into being an experimentalist 00:42:07.619 --> 00:42:11.219 and forging that path forward by doing. And I 00:42:11.219 --> 00:42:13.360 think that's the fundamental shift of what questions 00:42:13.360 --> 00:42:15.099 they could be asking. Are you experimenting? 00:42:16.039 --> 00:42:17.400 I think when it comes down to it at the end of 00:42:17.400 --> 00:42:20.780 the day, Neil. Wow, so much to take away and 00:42:20.780 --> 00:42:22.659 think about that. I'd love to invite everyone 00:42:22.659 --> 00:42:25.639 listening to share your thoughts on this, your 00:42:25.639 --> 00:42:28.519 insights, your experiences. So for everyone listening, 00:42:28.960 --> 00:42:32.179 I will post a link to the LaunchDarkly report 00:42:32.179 --> 00:42:34.079 that you referenced, I think, earlier in the 00:42:34.079 --> 00:42:36.219 conversation. But anywhere else you'd like me 00:42:36.219 --> 00:42:38.159 to point, everyone, if they want to connect with 00:42:38.159 --> 00:42:40.739 you, your team, or just some of the work and 00:42:40.739 --> 00:42:42.280 blogs and everything you write, where would you 00:42:42.280 --> 00:42:44.659 like me to point, everyone? So I do tend to post 00:42:44.659 --> 00:42:49.599 on LinkedIn fairly frequently. So that's my main 00:42:49.599 --> 00:42:52.280 focus. I'm there. I'm easy to find. I've got 00:42:52.280 --> 00:42:56.880 a pretty unique name. Drop me a note. Let me 00:42:56.880 --> 00:42:58.780 know what you're thinking. Easy for that. And 00:42:58.780 --> 00:43:00.880 I'll point you to the LaunchDarkly website. We've 00:43:00.880 --> 00:43:03.800 got a free foundation tier that you can sign 00:43:03.800 --> 00:43:06.119 up with and play with and see for yourself what 00:43:06.119 --> 00:43:09.599 we can do for you. I think it's a neat product 00:43:09.599 --> 00:43:12.670 that is all the more important when you have 00:43:12.670 --> 00:43:16.269 an AI -driven SDLC for that set of control and 00:43:16.269 --> 00:43:20.449 feedback and that loop around releasing, observing, 00:43:20.989 --> 00:43:25.340 and iterating with control. I will add links 00:43:25.340 --> 00:43:27.420 to everything that you mentioned there. And as 00:43:27.420 --> 00:43:29.579 I said, I invite everyone listening and watching 00:43:29.579 --> 00:43:32.639 to post your feedback and especially around shadow 00:43:32.639 --> 00:43:36.159 AI in DevOps when AI written code could cause 00:43:36.159 --> 00:43:38.360 a breach. Who is responsible? This is something 00:43:38.360 --> 00:43:40.719 we could debate about for hours. But more than 00:43:40.719 --> 00:43:42.500 anything, just thank you for starting this conversation 00:43:42.500 --> 00:43:44.579 today. Thanks, Neil, for the time. This has been 00:43:44.579 --> 00:43:47.980 great. There was so much to take away from this 00:43:47.980 --> 00:43:50.219 conversation today. But for me, I think one of 00:43:50.219 --> 00:43:53.820 the biggest takeaways was simple. AI, yep, it's 00:43:53.820 --> 00:43:56.780 speeding up software delivery. But, sorry to 00:43:56.780 --> 00:43:58.940 be the guy bringing in the bad news here, it 00:43:58.940 --> 00:44:03.500 doesn't remove responsibility. If anything, it's 00:44:03.500 --> 00:44:06.719 raising the stakes. And yeah, shadow AI might 00:44:06.719 --> 00:44:09.960 feel harmless in the moment. A quick shortcut 00:44:09.960 --> 00:44:13.000 here, a productivity boost there, a faster way 00:44:13.000 --> 00:44:15.539 to get the code out the door on a Friday afternoon. 00:44:16.400 --> 00:44:20.219 But without visibility, controls and clear ownership. 00:44:20.380 --> 00:44:23.760 All of these shortcuts can get expensive very 00:44:23.760 --> 00:44:26.880 quickly. And what I loved about Cameron's perspective 00:44:26.880 --> 00:44:29.880 here is that he did not come at this from a place 00:44:29.880 --> 00:44:34.059 of fear. He came at it from experience. And the 00:44:34.059 --> 00:44:37.320 answer was very clear. It's not a time to panic 00:44:37.320 --> 00:44:40.820 or block progress or stifle innovation. It's 00:44:40.820 --> 00:44:43.440 simply a time to put the right guardrails in 00:44:43.440 --> 00:44:46.159 place so teams, yes, can experiment to their 00:44:46.159 --> 00:44:49.579 heart's content. They can ship code. and still 00:44:49.579 --> 00:44:55.119 remain in control. So as AI becomes part of the 00:44:55.119 --> 00:44:58.099 everyday developer workflow, I think the question 00:44:58.099 --> 00:45:00.659 is no longer whether teams will use it. The genie 00:45:00.659 --> 00:45:03.320 is out of the bottle now, isn't it? I think the 00:45:03.320 --> 00:45:06.079 bigger question is whether your organization 00:45:06.079 --> 00:45:10.280 is ready for what comes next. And how are you 00:45:10.280 --> 00:45:15.059 managing this transition? As always, techtalksnetwork 00:45:15.059 --> 00:45:17.940 .com, you'll find 4 ,000 interviews, lots of 00:45:17.940 --> 00:45:20.900 ways of getting in touch with me there. And please, 00:45:21.019 --> 00:45:23.159 feedback, I'd love to hear from you. A quick 00:45:23.159 --> 00:45:26.760 thank you to NordLayer for supporting the podcast 00:45:26.760 --> 00:45:29.440 and helping me make these daily conversations 00:45:29.440 --> 00:45:31.980 possible. And if you are listening and you're 00:45:31.980 --> 00:45:35.659 responsible for security or IT, you will know 00:45:35.659 --> 00:45:38.619 the reality. The reality that most of your risk 00:45:38.619 --> 00:45:42.510 now sits inside SaaS apps. and browser activity. 00:45:43.010 --> 00:45:45.730 That gap is exactly what NordLayer is addressing 00:45:45.730 --> 00:45:49.530 with its new business browser. So instead of 00:45:49.530 --> 00:45:52.889 bolting security on from the outside, it builds 00:45:52.889 --> 00:45:56.530 it directly into the browser itself. This means 00:45:56.530 --> 00:45:59.889 you can control access, monitor activity, enforce 00:45:59.889 --> 00:46:04.630 policies and reduce shadow IT all from one single 00:46:04.630 --> 00:46:07.900 place. And most importantly, It does it without 00:46:07.900 --> 00:46:11.139 adding deployment headaches or complex onboarding. 00:46:11.480 --> 00:46:13.800 You get things like browser -based data loss 00:46:13.800 --> 00:46:17.219 prevention, SAS access control and zero trust 00:46:17.219 --> 00:46:20.139 browsing, but delivered in a way that your team 00:46:20.139 --> 00:46:23.440 can actually use. So if you've been trying to 00:46:23.440 --> 00:46:26.039 simplify your stack while improving visibility, 00:46:27.059 --> 00:46:30.780 please check it out at Nordlayer .com slash browser. 00:46:30.940 --> 00:46:33.380 But that's it for today, so time for me to check 00:46:33.380 --> 00:46:35.460 out. I'll be back again very soon with another 00:46:35.460 --> 00:46:37.679 guest, and hopefully you will meet me here, same 00:46:37.679 --> 00:46:39.679 time, same place. Bye for now.