WEBVTT 00:00.000 --> 00:10.513 The conclusion I've reached though is that if we looked like we were accidentally creating greater quality, we would respond not by achieving that quality, but by lowering our 00:10.513 --> 00:16.814 standards or increasing our haste until we manage to lower quality back to where it was before. 00:16.858 --> 00:25.953 Because the level of quality we have in software is not it's not the maximum we know how to achieve, or at least I don't think it's been in any code base I've been part of. 00:25.953 --> 00:28.014 It's basically it's lowers 00:28.074 --> 00:29.545 the conditions we'll accept. 00:29.545 --> 00:35.358 And if it gets lower and things start breaking, then people will get it so it's just above the waterline. 00:51.898 --> 00:56.479 In an Engineering with AI First, we're having a guest back for a second time. 00:56.479 --> 00:58.420 Most of our audience will remember Chris. 00:58.420 --> 01:02.361 When he was last on the show, he led with "Human in the Loop is dead". 01:02.581 --> 01:05.842 He's also been one of the most often downloaded episodes of all time. 01:05.842 --> 01:08.323 And I don't think it's just because he said something spicy. 01:08.323 --> 01:09.673 I I tried that with other guests. 01:09.673 --> 01:11.163 There's something else to it. 01:11.344 --> 01:14.524 We wanted to have him back today because of the book he's been writing. 01:14.525 --> 01:16.305 So let's get started. 01:16.305 --> 01:17.886 Chris, what's it called? 01:17.886 --> 01:18.506 What are you doing? 01:18.506 --> 01:19.916 What are you saying in this book? 01:19.986 --> 01:20.956 Right, it's awesome to be back. 01:20.956 --> 01:21.218 Okay. 01:21.218 --> 01:25.373 So the book that I'm writing is called Agentic Engineering at Scale. 01:25.373 --> 01:31.650 And basically the concept is I wanted to talk about all the stuff that is beyond personal productivity. 01:31.650 --> 01:39.595 Like there's a lot of personal hacking going on, but then a lot of people want to know how to get stuff done in groups of people or communities of people. 01:39.595 --> 01:48.719 maybe I can I can kind of say how I've divided things up in the book because I you know I wanted to I want to make it work for people who have real problems in real organizations. 01:48.719 --> 01:59.183 Not because I don't find it really fascinating, you know, how you set up your right agents MD or exactly how you code as an individual, but well, selfishly that stuff's changing 01:59.183 --> 01:59.793 really quickly. 01:59.793 --> 02:04.705 So very hard to put into a book in a way that will not be obsolete in a month's time. 02:04.705 --> 02:06.736 But secondly, I think the the gap. 02:07.072 --> 02:16.284 that a lot of people I'm seeing ask are people who come in after hacking on the weekend, getting all sorts of stuff done, and then find that they just don't know how to make that 02:16.284 --> 02:21.060 thing or that magic happen or apply in their workplace. 02:21.060 --> 02:21.491 ~ so 02:21.491 --> 02:22.692 that's what I'm trying to tackle. 02:22.692 --> 02:30.111 Okay, so this is not so much for the person who's vibe coded something and and now wants to get it in the app store. 02:30.111 --> 02:36.718 This is more for the the the plucky engineer who wants to to help out the team he's on, he or she. 02:36.922 --> 02:47.691 Yeah, so ~ I I guess the so as a I'm a consultant, I work for ThoughtWorks, and so often we give advice to our clients about how to do better in their software engineering. 02:47.691 --> 02:56.518 And that normally means because they have to be big enough companies to wanna pay externals to give them opinions, that they're trying to get large things done with groups 02:56.518 --> 02:57.800 of people coordinating. 02:57.800 --> 03:04.385 And so that doesn't it doesn't decompose into just everyone go off into their own corner and vibe code their own app. 03:04.385 --> 03:05.435 There has to be 03:05.435 --> 03:06.207 Right. 03:06.282 --> 03:09.885 have to care about quality that maybe people don't care about as much when they're prototyping. 03:09.885 --> 03:13.687 They have to care about managing priorities and specs and things like that. 03:13.687 --> 03:16.809 They have to worry about teams of people working together. 03:16.809 --> 03:21.912 And maybe the biggest thing, which I opened the book with, which I think is a bit of a contrast, is they have to worry about legacy code. 03:21.912 --> 03:31.059 So a lot of people you hear, you see on LinkedIn or whatever, they're basically talking about things that they started last night and maybe you've got a hundred GitHub stars 03:31.059 --> 03:31.937 already. 03:31.937 --> 03:32.349 Mm-hmm. 03:32.349 --> 03:44.395 not thinking about here's this system that pays a thousand people's wages, how do I work with that in a way that respects it and doesn't break it, basically? 03:44.395 --> 03:45.437 Yeah, yeah, yeah. 03:45.437 --> 03:46.057 Okay. 03:46.057 --> 03:46.839 Cool. 03:46.839 --> 03:48.242 what what was your process like for the book? 03:48.242 --> 03:51.936 Did you research and talk to people and let let's hear that part? 03:51.936 --> 04:00.178 Well the selfishly the the kind of the the fortunate thing is that my job consists of coming up with opinions in this area and working with clients to solve their problems. 04:00.178 --> 04:03.849 So I do get a bit of synergy out of my personal thinking time and my work. 04:03.849 --> 04:12.421 So a lot a lot of what I ~ write about is maybe a synthesis of my own experience across many different clients, which is one of the privileges of being a consultant that you get 04:12.421 --> 04:14.782 to dip your toe into many different waters. 04:14.792 --> 04:17.453 but also the collective wisdom of my colleagues at ThoughtWorks. 04:17.453 --> 04:18.504 So I'll you I 04:18.504 --> 04:24.616 do a lot of sharing of drafts and asking of questions on the internal chats and and things like that. 04:24.616 --> 04:31.449 I mean of course also I I do a lot of reading to inform or try to make my book semi informed ~ as well. 04:31.449 --> 04:31.619 but 04:31.619 --> 04:36.807 really it's trying to channel the experience that people have already started to have in these in this area. 04:36.807 --> 04:37.368 Yeah. 04:37.368 --> 04:38.541 It's no easy thing writing a book. 04:38.541 --> 04:43.482 There turns out to be a a fair amount of work, ~ in in in getting the writing down, 04:43.482 --> 04:45.687 publishing down, the all the stuff. 04:45.687 --> 04:46.467 Yeah. 04:46.599 --> 04:54.535 Well, a and especially I guess fascinating now is I I don't know whether people will be starting to assume that books have been written themselves by AI. 04:54.535 --> 05:03.262 I've certainly had at least one article that I wrote by hand a while back have someone report to Reddit, I think, for being a AI written. 05:03.262 --> 05:11.819 ~ I'm not quite I well, I'm not quite sure what caused that and I don't particularly blame someone for thinking that because I guess there are there is a lot of mixture of AI 05:11.819 --> 05:14.171 content or AI inflected content or 05:14.171 --> 05:15.222 or something out there. 05:15.222 --> 05:23.309 But it is kind of a an interesting thought that I might be just putting quite a bit of effort and quite a bit of part of my life into compiling a hundred thousand words and then 05:23.309 --> 05:26.702 have someone assume that they ~ they could do it themselves with fable. 05:26.702 --> 05:28.315 ~ I guess we'll see though. 05:28.315 --> 05:28.785 What was it? 05:28.785 --> 05:30.625 The two, three sentence prompt, basically. 05:30.625 --> 05:32.026 sound something like that, yeah. 05:32.297 --> 05:32.906 Yeah. 05:32.906 --> 05:33.908 That's all it was. 05:33.908 --> 05:34.478 Yeah. 05:34.478 --> 05:36.449 No, ~ it doesn't work that way, for sure. 05:36.449 --> 05:43.053 I it's funny though, to your to your point, there are certain tropes that like just hit me in a certain way right now. 05:43.053 --> 05:45.004 Like when if it's like if if 05:45.004 --> 05:54.268 What actually matters here is I'm like, okay, yeah, that was yeah, but that was probably you know, it was anthropic model, it was Claude that you were talking to that generated 05:54.268 --> 05:54.538 that. 05:54.538 --> 06:00.933 And you know, you everybody at first was trained to look for the em-dash instead of the hyphen because like who even knows how to type that? 06:00.933 --> 06:03.694 But ~ but it now it's it's certain other things. 06:03.694 --> 06:06.356 But now I wonder, like, is it inflecting our speech? 06:06.356 --> 06:11.129 Is it it's picking up on what we are doing and then it's doing it and then we're gonna follow it. 06:11.129 --> 06:11.920 I I don't know. 06:11.920 --> 06:13.981 So which came first in all of that? 06:14.283 --> 06:19.445 Yeah, well I have actually an dash problem because the way I found easiest to write is dictation. 06:19.445 --> 06:20.845 ~ maybe because 06:20.845 --> 06:25.087 through conversations like this I find it easier to maintain the flow while speaking. 06:25.147 --> 06:25.687 Yet 06:25.687 --> 06:32.910 while speaking and then reading a transcript, in speech we have a lot of, you know, meaningful pauses that lead on to the next thing we say. 06:32.910 --> 06:41.163 So I've naturally, when I get my raw speech transcribed, it has quite a few m-dashes based on the the software I'm using. 06:41.163 --> 06:47.228 So I'm now faced with the paradox of do I leave them in because that's authentically what I came up with, or do I try 06:47.228 --> 06:53.944 to write some kind of AI tool to get rid of the dashes to pretend that it's more natural than actually 06:53.944 --> 06:57.667 is so I'm I'm a bit confused as to how how I should do that that at the moment. 06:57.667 --> 07:00.129 ~ but I'm certainly stouting myself. 07:01.542 --> 07:05.586 These these are these are the problems that we now have as a society to solve. 07:06.328 --> 07:07.129 Yeah. 07:07.129 --> 07:07.680 Cool. 07:07.680 --> 07:13.877 Well obviously you don't want to give away the ending or or you know or any of that. 07:14.499 --> 07:15.371 Yeah. 07:15.371 --> 07:16.232 I can I can kind 07:16.232 --> 07:18.684 of g maybe engage with the key the key arguments of it. 07:18.684 --> 07:20.515 I think I think if someone got to the end of a technical book 07:20.515 --> 07:25.769 and only only kind of s found out right at the end what it was, I'm not sure they'd actually invest as much time. 07:25.769 --> 07:26.569 Yeah. 07:26.850 --> 07:28.352 Well, but I'm I'm also I'm wondering 07:28.352 --> 07:35.131 if there's a story in there that you know, to your point, a lot of this comes from, stories real and synthesized is there a story you would tell out of this? 07:35.494 --> 07:37.814 I think there's an I have a narrative of order of concepts. 07:37.814 --> 07:46.552 the big overarching theme is that control theory is a really useful way of thinking about how to get coding agents to do good work 07:46.552 --> 07:47.191 Mm. 07:47.191 --> 07:53.571 and that if you're gonna take that seriously, it's not just a matter of an individual. 07:53.571 --> 07:58.934 coming up with the kind of ~ guards, guardrails, checks, sensors and things like that. 07:58.934 --> 08:04.308 It's actually something that you have to start sharing across your organization, which leads to a whole bunch of 08:04.308 --> 08:08.951 problems because we already found it kind of difficult to do platform engineering. 08:08.951 --> 08:18.717 But I think I have become convinced that if you're going to achieve with agentic engineering at scale, so just getting the book title in there, ~ another another kind of 08:18.847 --> 08:23.180 branding plug, ~ you will have to start thinking about sharing 08:23.234 --> 08:28.898 common expectations of what go good code looks like in a way that coding agents can access 08:28.898 --> 08:29.611 Mm. 08:29.611 --> 08:36.204 at scale and then you'll have to worry about the knowledge management and the human collaboration that gets to the consensus on those artifacts. 08:36.204 --> 08:36.678 Right. 08:36.678 --> 08:48.324 a team that I serve ~ has taken to having a a confluence page that that sort of is a is a little bit of a a starting point around like, okay, please don't just go write your own 08:48.324 --> 08:49.715 crazy terraform willy-nilly. 08:49.715 --> 08:55.699 We we use a YAML file that our platform team knows how to ingest, here's here's the ropes, there's the bathroom kind of stuff. 08:55.699 --> 09:03.359 ~ and that way, since it's in the confluence, like ~ the first thing we do in ClaudMD is go read it and it's like, okay, now none of us have to tell it again. 09:03.359 --> 09:05.025 You know, and we can kinda keep it up to date. 09:05.025 --> 09:05.566 Yeah. 09:05.566 --> 09:10.169 So I'm imagining that kind of thing, but you may mean other more nuanced things in addition. 09:10.293 --> 09:19.390 So I I I divide up the different kinds of and I'll say harness for now and maybe you can substitute in whatever word is popular at given times. 09:19.390 --> 09:23.052 You know, we can we can refresh this as we get different jargon as we go along. 09:23.083 --> 09:26.685 but I divide harnesses into four kinds and I think you need 09:26.685 --> 09:27.996 a balanced diet. 09:28.056 --> 09:36.970 So ~ maybe I think the two most important distinctions of different kinds of harness is whether it's something that tells you yes or no. 09:36.970 --> 09:44.506 So whether it's giving information back to the agent and the agent has to make its own decision or whether it actually gives a red or a green signal. 09:44.506 --> 09:49.630 So a unit test keep tells you no, it's not good enough, or yes, it is good enough. 09:49.630 --> 09:58.807 But something like using play wright from the browser gives the agent access to look at the site you're working on, but it doesn't tell the agent whether it's acceptable or not. 09:58.807 --> 10:00.529 The agent has to kind of apply the judgment. 10:00.529 --> 10:03.111 So I think that's kind of that's one of the distinctions. 10:03.111 --> 10:05.963 The second distinction, just a link to what I said about 10:05.963 --> 10:10.556 control theory is ~ feed forward versus feedback. 10:10.556 --> 10:15.358 So maybe I should unpack that because feed forward is a somewhat unfortunate nilogism. 10:15.358 --> 10:23.102 But feedback is I've done a bit of the system, let me use this to perceive it and work out what's going on. 10:23.102 --> 10:30.873 Either to tell me whether yes, no, if it's a judgment based thing, or just to let me get information or context about what's going on, kind of observability. 10:30.873 --> 10:33.098 So that's got a feedback which people are familiar with. 10:33.098 --> 10:35.617 Feed forward is effectively 10:35.617 --> 10:36.678 guidance. 10:36.678 --> 10:47.343 So it's you know, in control theory, feed forward is the stuff that tells you where you should be aiming for without you having to give it a go and and evaluate where you are. 10:47.343 --> 10:57.669 So if you kind of think of those two things as axes, because I'm a consultant, everything becomes a quadrant, that you get kind of things that are normative, tell you yes or no 10:57.669 --> 11:05.673 that are feedback, things that are normative that are feed forward, and you get the informative versions of feed for forward and feedback as well. 11:05.697 --> 11:07.908 And so that combination I think is really important. 11:07.908 --> 11:14.240 So to your point about guidance, I would call that a guide if you have something about the golden path and your confluence. 11:14.240 --> 11:16.822 So that's informative feed forward. 11:16.822 --> 11:19.163 So the agent can use that to do a better job. 11:19.163 --> 11:27.686 It's not going to tick off the agent for getting it right or wrong, and it's not going to allow the agent to evaluate what it's done, but it is actually one of the important things 11:27.926 --> 11:31.316 in order to to have a good system of input to the agent. 11:31.352 --> 11:37.747 And I and I would say a lot of people who are struggling at the moment tend to be overindexing on one kind of harness or the other. 11:37.747 --> 11:38.868 So they might be trying to lean 11:38.868 --> 11:40.951 entirely on feed forward. 11:40.951 --> 11:45.355 I'm not gonna give the agent any way to mark its own homework or understand what it's done. 11:45.355 --> 11:48.820 I'm just gonna tell it what good is and hope that it listens. 11:48.820 --> 11:49.921 And it might listen, 11:49.921 --> 11:53.583 but it might also go off track and not be able to self-correct. 11:53.583 --> 11:54.424 But alternatively, 11:54.424 --> 11:59.848 I see people focusing far too much on feedback, which is leads towards brute force kind of 11:59.900 --> 12:00.531 situation. 12:00.531 --> 12:05.245 So they don't give the agent guidance on what a good design looks like. 12:05.245 --> 12:09.699 So they don't give it the agent a design system or they don't give it a templated infrastructure for your company. 12:09.699 --> 12:13.060 They just say go for it and like see how you go. 12:13.060 --> 12:22.324 And that results in cycles, which is good, but too many cycles and it doesn't converge on a good answer as quick as it would if you'd started it near to where it should have been. 12:22.948 --> 12:27.617 And don't think these things learn on the job in the same way or as quickly as humans do. 12:27.617 --> 12:30.693 So yeah, that sounds like it could be particularly painful. 12:30.693 --> 12:31.754 Yeah. 12:31.886 --> 12:40.371 Well, yeah, learning on the job or just, online learning in general is, some people would say is one of the big architectural problems with the current generation of LLMs. 12:40.371 --> 12:41.012 ~ and 12:41.012 --> 12:44.023 it's actually, you know, it's not what the everyday public thinks is happening. 12:44.023 --> 12:49.826 The everyday public assumes that if they ask a question of an LLM, the L LM will learn at the same time because that's what we do. 12:49.826 --> 12:50.326 But as we 12:50.326 --> 12:57.947 know, if we you know if we want to feed information we've learnt back in, we have to do it explicitly with some kind of agent memory mechanism. 12:57.947 --> 12:58.417 Exactly. 12:58.417 --> 12:58.767 Yes. 12:58.767 --> 13:04.851 We have this business of memory saving now and and those kinds of things, which is kind of trying to approximate it. 13:04.851 --> 13:09.424 I'm not sure it's exactly the same thing, but it 'cause that just results in more feed forward. 13:09.424 --> 13:10.914 But maybe that's fine, I guess. 13:10.914 --> 13:14.717 But i you've got this concept of like the context window is only so big though, right? 13:14.717 --> 13:15.037 So 13:15.037 --> 13:19.279 I assume all this feed forward is eating into that budget in the end. 13:19.466 --> 13:29.800 So I guess progressive context disclosure, maybe an overly bureaucratic term, is what people usually say for trying to stage the input so that the agent doesn't just read the 13:29.800 --> 13:31.161 whole world through a straw. 13:31.161 --> 13:42.405 So for example, if you're giving your agent ADRs, you should be giving it an index page with clear kind of titles so that it knows when it should actually go one step further. 13:42.405 --> 13:47.907 And when I've seen people profile the context usage of their agent. 13:47.907 --> 13:54.552 accidental overreading of context is probably the number one thing that burns tokens. 13:54.552 --> 14:03.478 They didn't really realize it, but they've got a kind of a series of hyperlinks that the agent is preloading because it's been encouraged to go from the agent MD to this file to 14:03.478 --> 14:06.400 your Confluence to another page on Confluence to a manual. 14:06.400 --> 14:14.245 and it it does that maybe on every session or almost every session and that causes a bunch of irrelevant stuff to yeah junk up the context 14:14.245 --> 14:15.360 window like you say. 14:15.360 --> 14:16.103 Yeah, yeah, yeah. 14:16.103 --> 14:18.511 And it gets dumber as the context window fills for sure. 14:18.511 --> 14:20.978 So that this is to be discouraged. 14:21.059 --> 14:31.758 I don't know about your experience, but I think the common figure that people say is that if you're more than fifty percent full, ~ it starts to have trouble with knowing what to 14:31.758 --> 14:32.598 pay attention to. 14:32.598 --> 14:33.139 So you start 14:33.139 --> 14:34.571 to get erratic results. 14:34.571 --> 14:36.011 Yeah, right. 14:36.052 --> 14:41.516 I I often think of it as it doesn't know what to forget to to keep room in the in the but but yeah, you're right. 14:41.516 --> 14:44.223 It's also the just the problem of what should I pay attention to? 14:44.223 --> 14:45.779 This thing's got so much in it now. 14:45.779 --> 14:46.659 Yeah. 14:46.760 --> 14:47.116 Nice. 14:47.116 --> 14:56.683 in my mind I it's it it manifests in maybe early on in your agents MD or even in your session you told it, please remember this, this, this, do not do this, this, this. 14:56.683 --> 15:03.417 And once you get more than fifty percent full, you start at the playing Russian roulette with whether those instructions are still going 15:03.417 --> 15:05.360 to be brought to mind at any one time. 15:05.360 --> 15:05.820 Yeah. 15:05.820 --> 15:14.814 Through the magic of time the time travel that podcasting brings, people listening to this episode, if they listen to the last one, it was with Birgitta Bockler also at 15:14.814 --> 15:19.726 Thoughtworks, ~ and she talked about how we say these are guardrails. 15:19.726 --> 15:22.927 Those are the things that keep your car on the highway, but 15:23.248 --> 15:27.129 sometimes your car just goes straight through and the guardrails are just gone in this world. 15:27.129 --> 15:29.370 So are they guardrails at all? 15:29.370 --> 15:33.152 Yeah, feedback and feed forward definitely seem like better words for what's happening there. 15:33.152 --> 15:33.455 Yep. 15:33.455 --> 15:38.270 and to be clear, I partly got those terms literally from Brigita, who I talked about this before. 15:38.270 --> 15:42.685 So I I don't want to claim to have superseded or or improved necessarily on her thinking. 15:42.685 --> 15:49.712 I think she's done a really good job of distinguishing between different kinds of things that we we have in harness engineering and spec driven development. 15:49.712 --> 15:50.112 Definitely. 15:50.112 --> 15:51.755 She's she's given us some more words. 15:51.755 --> 15:54.829 This is so exciting about ~ about your book, Chris. 15:54.829 --> 15:56.602 what can you say about like when it comes out? 15:56.602 --> 15:59.776 And i is this gonna be just Dead Tree or or I don't know. 15:59.776 --> 16:00.917 Will there be a movie made? 16:00.917 --> 16:02.279 Like do I get to play the starring role? 16:02.279 --> 16:02.900 Probably not. 16:02.900 --> 16:03.550 But 16:04.031 --> 16:08.843 Yeah, I mean I guess ~ I Dead Tree is not the only format that you would release a book in these days. 16:08.843 --> 16:10.614 I do have a promise of that being the case. 16:10.614 --> 16:14.925 I have a an O'Reilly animal for the front cover, which is Australian fauna, which I asked for. 16:14.925 --> 16:24.414 ~ they didn't they didn't ~ they didn't agree to f get me a deadly Australian animal, so they found one of the cute ones that is not deadly, but still that's that's pretty cool 16:24.414 --> 16:25.747 for for kind of a 16:25.747 --> 16:27.178 eco-nationalist like myself. 16:27.178 --> 16:29.970 ~ it's supposed to come out in Q1 next year. 16:29.970 --> 16:32.682 I'm about to say something that I'll probably regret. 16:32.682 --> 16:35.524 I'm a little bit ahead of schedule in terms of the writing. 16:35.524 --> 16:44.824 ~ though I've discovered it's a really good embodiment of ~ the difficulties with improving the speed of coding, not necessarily improving the speed of the machine. 16:44.824 --> 16:51.609 I'm working in a collaborative effort with a bunch of people who do other things in their day job with publishing and they have other schedule. 16:51.609 --> 16:52.750 So me 16:53.046 --> 16:58.481 dictating or typing slightly faster doesn't necessarily alter when the end product is, much like using a 16:58.481 --> 17:01.514 coding agent doesn't necessarily get you live faster in a real project. 17:01.514 --> 17:11.571 But yeah, the the schedule is ~ Q one next year, but the early release is in the works at the moment, so that'll be on the O'Reilly platform in electronic form with I think the 17:11.571 --> 17:15.179 first four out of twelve chapters soon. 17:15.179 --> 17:16.059 Cool. 17:16.061 --> 17:21.418 Aren't you gonna like keep us keep us waiting to figure out which Australian charismatic fauna that we've chosen? 17:21.418 --> 17:22.398 just assumed you wouldn't know. 17:22.398 --> 17:24.920 So that I really like to use the the ~ obscure ones. 17:24.920 --> 17:34.886 the animal is a honey possum, which is basically the size of a mouse, but it has an especially long nose because it's design well, I was about to say designed, I didn't mean 17:34.886 --> 17:35.036 that. 17:35.036 --> 17:44.933 ~ it's iterated towards being able to suck out honey from banksia and protea flowers, which are these kind of like enormous flowers of Australian flora. 17:44.933 --> 17:47.844 So yeah, it looks pretty it looks pretty cute is the idea. 17:47.844 --> 17:49.255 And yeah, I don't think 17:49.291 --> 17:53.624 I know of any fatalities at all involving them, but I guess we can we can always hope. 17:53.624 --> 17:54.505 Yeah. 17:54.587 --> 17:54.948 Yeah. 17:54.948 --> 17:56.418 Tiny possum basically. 17:56.418 --> 17:57.986 there's an analogy in there somewhere. 17:57.986 --> 18:00.868 Th the long straw something, I don't know. 18:00.868 --> 18:01.729 Yeah. 18:01.729 --> 18:02.059 Yeah. 18:02.059 --> 18:02.519 Yeah, I mean 18:02.519 --> 18:07.762 I I I I think ~ I I've learned obviously well I I've done a lot of technical reviewing in the past. 18:07.762 --> 18:13.834 So I have I've been, you know, reviewer of a lot of books is my first time actually being the the in the driver rather than the backseat driver. 18:13.834 --> 18:22.678 ~ but one of the things I learned is that, you know, publishing companies want to be publishing companies, not people who spend all their time explaining to adults why they 18:22.678 --> 18:24.609 can't have the elephant or whatever. 18:24.609 --> 18:29.843 So, you know, in the end they don't try to come up with really direct mappings of 18:29.843 --> 18:34.098 cover animal to subject matter because I think that ends up being just like coupling 18:34.098 --> 18:36.832 in software, that ends up being ~ 18:36.832 --> 18:39.849 a bit onerous even though it might seem neat at the time. 18:39.849 --> 18:44.217 Yeah, a bad idea for your publishing process to try to find like the exact right 18:44.217 --> 18:47.056 change what chapter seven's about and they have to like get you a new animal. 18:47.056 --> 18:47.508 I don't know. 18:47.508 --> 18:48.574 Like it yeah, it doesn't 18:48.574 --> 18:49.454 Right. 18:49.495 --> 18:50.135 That too. 18:50.135 --> 18:51.176 That too. 18:51.195 --> 18:51.877 This is cool. 18:51.877 --> 18:52.527 Super exciting. 18:52.527 --> 18:55.259 so I'm wondering did you did you check out the radar? 18:55.259 --> 18:56.364 Like what were your thoughts? 18:56.364 --> 19:02.019 it was a lot of fun for me to go back and and review what you and other guests had said, ~ and put this together. 19:02.019 --> 19:03.660 ~ yeah, what were your thoughts? 19:03.660 --> 19:13.654 to start with, thank you for putting it together because I I I think ~ my kind of meta observation is that we're using words in really incommensurate different ways. 19:13.654 --> 19:23.444 So all your work you're doing of kind of trying to organize the thoughts to actually start the process of us at least being able to even argue at each other rather than past each 19:23.444 --> 19:25.124 other, I think is a pretty important function. 19:25.124 --> 19:26.985 So thank you for doing that. 19:26.985 --> 19:29.088 ~ I think for me I was quite attracted 19:29.088 --> 19:35.863 Perhaps because I was interpreting things through my own lens, thro the the things that were, I guess, talking about feedback in some way. 19:35.863 --> 19:38.205 I saw a lot of things through that lens. 19:38.205 --> 19:45.396 and that maybe in to tie it back to the work I was doing of of trying to reconcile feedback and feed forward, is that I I I can't 19:45.396 --> 19:54.672 say I scientifically remember exactly the proportion of things, but I wondered whether we were focusing too much on the inner loop and not enough on the guidance of how to get to 19:54.672 --> 19:57.814 close to what the answer is before we ~ iterate. 19:57.814 --> 20:06.950 So I kind of I I liked it both for helping me to explore the different ways people have thought about those things, but also, you know, I was kind of in a way using it to map out 20:06.950 --> 20:09.251 what parts of the conversation we haven't had yet. 20:09.341 --> 20:09.842 Got it. 20:09.842 --> 20:17.777 Now when you say interloop versus guidance, I'm not sure I can exactly like I could make three guesses as to what you mean, but but help me fill it in. 20:17.777 --> 20:24.661 Well, I I I'm probably not even using my own terminology in a standard way, but what I meant specifically was the experience of an individual developer. 20:24.661 --> 20:32.607 they've got the coding agent, they've got things set up, and they want to give the agent a way to tell whether it's doing a good job. 20:32.607 --> 20:34.739 I think I I seem to remember quite a few of those. 20:34.739 --> 20:43.455 So in my mind, that's very that's in actually one of the most useful quadrants of the one of the the four, but it's ~ feedback and normative. 20:43.455 --> 20:45.236 So how do I get a linter? 20:45.281 --> 20:47.652 To tell the agent that it did wrong and it has to fix it? 20:47.652 --> 20:52.925 Or how do I you know how do I otherwise control the agent? 20:52.925 --> 21:03.571 I think that is really important because anything that's in there is part of the discretionary process that you can set the agent off to do, to tie it back to the earlier 21:03.571 --> 21:04.131 comment. 21:04.131 --> 21:08.214 I'm not sure I phrased it exactly this way, but ~ of human in the in the loop being dead. 21:08.214 --> 21:12.818 But if you want the human in the loop to not be there, you need to give 21:12.818 --> 21:18.693 enough scaffolding that the agent can try, fail, try better, fail better, succeed. 21:18.693 --> 21:19.313 And a lot, 21:19.313 --> 21:28.751 I think, of our attention has been naturally at that question of in the small, including your guests, because that's a problem we have to solve before we can even earn the right 21:28.751 --> 21:33.248 to encounter other problems about how we get to good results. 21:33.248 --> 21:33.808 Yeah. 21:33.808 --> 21:38.011 I'm seeing interesting like patterns in terms of how to make that safer, right? 21:38.011 --> 21:47.918 and whether it's like, the Swamp Club, which I I learned about only like a few days ago and and still need to dive into, but ~ it's apparently aimed at sort of DevOps scripting 21:47.918 --> 21:58.037 and helping the agent become well, repeatable ~ essentially I assume that this means it's going to become a tool that the agent uses instead of like letting the agent GPU it and 21:58.037 --> 21:59.068 there 'cause it might do it at 21:59.068 --> 22:00.150 the second 22:00.150 --> 22:00.941 Wait at a different time. 22:00.941 --> 22:09.758 ~ but but yeah, tools like that, tools like code mods, you know, like building something that goes and does the refactoring for you so that it doesn't have to imagine how to do it 22:09.758 --> 22:10.378 each time. 22:10.378 --> 22:15.872 These not only save on tokens, but also make it safer for us not to be in the loop, I think. 22:16.662 --> 22:21.344 Yes, I think if don't use inference to solve something unless you have to is I 22:21.344 --> 22:22.705 think a reasonable rule. 22:22.705 --> 22:34.000 Although I must also say that like I think some of our enthusiasm for these tools is partially due to wanting to retain the kind of control we have when we use tools like 22:34.000 --> 22:36.291 refactoring tools and compilers. 22:36.291 --> 22:36.581 So 22:36.581 --> 22:43.474 ~ I've certainly heard a lot of people talk about how they think it's a problem that agents are not deterministic. 22:43.722 --> 22:48.157 And so their solution to that is well, how do I introduce determinism much in the process? 22:48.157 --> 22:55.196 So I completely agree actually that if a code mod can do it, that's a more efficient way of doing it and it's a less error prone way of doing things. 22:55.196 --> 22:59.861 But I don't I think it's a strength in a way that agents are non deterministic. 22:59.861 --> 23:01.233 It's just a a strength that 23:01.233 --> 23:03.937 we are not used to exploiting. 23:03.937 --> 23:04.507 Right. 23:04.507 --> 23:13.525 I I I in when I'm working in with organizations and I'm trying to make sure that the engineering staff in that organization has access to good models, I often find myself 23:13.525 --> 23:16.717 saying, I just want my smart friends to have the good tools. 23:16.837 --> 23:18.798 But I feel like this also applies to the agent. 23:18.798 --> 23:26.902 I want my smart agent friends to have access to the good tools, whether it's Swamp Club or Code Mods or Linters, obviously. 23:26.902 --> 23:28.683 But you know, sort of all of this category. 23:28.683 --> 23:32.925 And I guess these are all in the feed well, many of these, not all, are in the feedback. 23:33.087 --> 23:33.557 side. 23:33.557 --> 23:35.409 But some of this is is it almost something different. 23:35.409 --> 23:43.728 Like if we're expecting the agent to reach for that tool in the same way that it would reach for L S or Bash or whatever, is that feedback or feed forward? 23:43.728 --> 23:45.459 That or is that something in the middle? 23:46.129 --> 23:52.673 so I think it might be not I don't want to introduce three dimensions because ~ even even in electronic books you can't really represent it. 23:52.673 --> 23:58.486 But I think I think the kind of mech mechanistic versus the inferential is a a little bit of another distinction because I think you 23:58.486 --> 24:03.638 could I think you could introduce it for for other things in in terms of the way the agent invokes it. 24:03.638 --> 24:11.293 but I I I think that a lot of the advantage to the it's not just that it's mechanical, it's that you'd need the agent to do less work. 24:11.293 --> 24:19.125 If you can express the possibilities of what you want the agent to do, is what effectively is a smaller language or a smaller set of levers. 24:19.125 --> 24:23.616 Basically, you're asking for less information to be outputted by the agent to make a good choice. 24:23.616 --> 24:30.198 So you're not asking it to wade through a billion invalid possibilities to find the one good one. 24:30.433 --> 24:41.071 It kind of matches up with the idea of the golden path from Netflix and that you mentioned earlier, which is that if you give it a more constrained set of options of how it can act. 24:41.107 --> 24:44.648 you're putting less of a burden on it, you're making it more likely it'll choose well. 24:44.750 --> 24:51.288 I think I'm learning that the story I told about go to confluence to find out how we deploy is something Netflix talks about called the Golden Path. 24:51.288 --> 24:52.526 Get you wanna tell that story? 24:52.526 --> 25:01.532 well, sorry, I think I think it was originally they taught the the the concept of the golden path was so so Netflix were ~ earlier adopter of microservices, expanding very 25:01.532 --> 25:02.633 rapidly, ~ 25:02.633 --> 25:05.175 aiming to have really high talent density. 25:05.175 --> 25:12.630 But they also then realized they didn't want everyone to be spending their high talent density in building infrastructure from scratch, right? 25:12.630 --> 25:12.861 So 25:12.861 --> 25:17.558 so how how do you square the kind of anarchist freedom principle with the kind of 25:17.558 --> 25:20.590 centralization urge to have consistent platform. 25:20.590 --> 25:21.090 The way they 25:21.090 --> 25:30.005 did it, and it's obviously been copied by others because they're an admired company, have done many interesting things with tech, is that they have a golden path that says if 25:30.005 --> 25:35.308 you're using Java and you're using this AWS service, then we support that really well. 25:35.308 --> 25:37.099 We document it really well. 25:37.099 --> 25:39.140 But it's not you can't leave the path. 25:39.140 --> 25:40.660 It's that if you're on the path, 25:40.660 --> 25:44.923 you get first class service across all of our tooling and other things. 25:44.923 --> 25:47.724 So ~ you know, it's a it's a way of 25:47.734 --> 25:54.668 in a way lowering the team's cognitive load because they don't have to evaluate every container running service that AWS has for themselves. 25:54.668 --> 26:00.012 They know that this is the one or these are the two that Netflix supports and that they'll get great tooling out of. 26:00.012 --> 26:07.700 But if they really do have a unique need, they're not constrained from going off road, basically. 26:07.700 --> 26:14.584 yeah, yeah, yeah, yeah, I I can remember these conversations coming out of Netflix, you know, sort of five, ten-ish years ago. 26:14.584 --> 26:17.225 Like I think I was in Cincinnati, so that would make it ten years ago. 26:17.225 --> 26:19.267 ~ at a at a guess. 26:19.267 --> 26:26.961 and yeah, that that sort of conversation about like, yeah, you can use the you know, this framework instead of that what framework, but you know, as you say, it's it's not the 26:26.961 --> 26:28.492 golden path, it's not as well supported. 26:28.492 --> 26:30.284 And that's that's a neat way to split that. 26:30.284 --> 26:36.127 It seems like interestingly, ~ in my own observations with my own clients, we're kinda not doing that with AI. 26:36.127 --> 26:41.009 And maybe that's ~ like a little bit of a fear about the non determinism and the security impact. 26:41.009 --> 26:46.251 I am aware of companies that have already seen well, I mean, we've all had the Shy Haludworm, you know, warrior's, you 26:46.251 --> 26:51.034 know, come in and and and w inject scripts into our systems and so on. 26:51.034 --> 26:51.794 So 26:51.794 --> 26:57.219 May I get why people would, you know, let's let's not go with the anarchy path this time. 26:57.219 --> 26:59.890 But but I'm just dreaming this up right now. 26:59.890 --> 27:01.433 Does that match your experience? 27:01.433 --> 27:10.330 Or are do you see organizations which are trying that, hey, let's experiment, let's allow experimentation, but this is the golden path versus what I'm seeing, which is more like 27:10.330 --> 27:12.873 no, there this is the path, you know, blinders on. 27:12.873 --> 27:13.673 Yeah. 27:13.753 --> 27:21.517 So I don't think organizations have got to grips how they can, as a collective, ha exercise any kind of opinion one way or the other. 27:21.517 --> 27:22.758 Is my is my experience. 27:22.758 --> 27:23.948 So ~ 27:23.948 --> 27:33.824 teams build their own skills and share them among the team is actually the most common level of collaboration I've seen, with exceptions of maybe some specialist services like 27:33.824 --> 27:41.667 token gateways or whatever, most of the time there is no collaboration of agents working to do things. 27:41.667 --> 27:43.550 So it's the equivalent of completely 27:43.550 --> 27:45.081 Atomize teams. 27:45.081 --> 27:50.763 The the most I've seen is that some companies do have systems for sharing skills. 27:50.803 --> 27:56.725 So someone will come up with a really good code review skill and then they'll share it with other teams. 27:56.725 --> 28:00.167 Most commonly it'll get passed, maybe unofficially. 28:00.167 --> 28:04.569 There are some organizations where there's like a central repository for them. 28:04.569 --> 28:11.662 But even at that stage, and you know, we're talking about really basic stuff, not infrastructure templates or design systems like you're kind of hinting at with the golden 28:11.662 --> 28:12.032 path. 28:12.032 --> 28:12.958 They're just 28:12.958 --> 28:15.689 Here's a way to review a pull request or whatever. 28:15.689 --> 28:22.812 ~ They get put on a central place, but it tends to rot at about the same rate as a wiki page as well. 28:22.812 --> 28:24.643 ~ Unfortunately. 28:24.643 --> 28:25.768 So ~ 28:25.768 --> 28:32.606 in terms of the choice, does the organization say agents must code in exactly this way versus encouraging the agent? 28:32.606 --> 28:42.721 I think actually by default, we're not restricting the agent at all, not because organizations have decided that they wish to see self organization right reign, but 28:42.721 --> 28:52.489 just because the governance functions of technical organizations haven't got traction on how to exercise their function with agents. 28:52.489 --> 28:52.990 Now I think 28:52.990 --> 29:04.017 actually we're going to see that happen and in a very clumsy way soon, where, you know, there will be a set of guidance that your agent is expected to load and the CTO will be 29:04.017 --> 29:10.791 able to like change it on the weekend and where previously the CTO might have wanted you to start writing unit tests or 29:10.963 --> 29:16.157 move only to outside in tests or adopt this latest thing that they've heard. 29:16.398 --> 29:25.016 They previously had to kind of rely on this chain of influence of telling the head of architecture and doing all hands and then if necessary people could just ignore them and 29:25.016 --> 29:26.307 get on with their life. 29:26.748 --> 29:33.875 But I think technical authoritarianism will actually be a a risk of having shared harness artifacts because it will 29:33.875 --> 29:36.597 be possible to directly inject 29:36.597 --> 29:40.261 You know, your your prompt injector will be your CTO, maybe is another way of putting it. 29:40.261 --> 29:40.841 And if they have 29:40.841 --> 29:42.443 great judgment, maybe that's good. 29:42.443 --> 29:48.679 But I I can't say that, you know, being senior always gives you infallible tech judgment necessarily. 29:48.685 --> 29:57.451 I I the immediate flashback for me is working in one of those organizations where the entire company, and in this case three hundred thousand people, the exact same JIRA 29:57.451 --> 29:58.831 template across 29:58.831 --> 30:05.155 all projects, whether they were accounting or finance or software development or and yeah, so authoritarianism gone wrong. 30:05.155 --> 30:06.736 But maybe this will be better. 30:06.736 --> 30:08.337 Maybe fingers crossed. 30:08.337 --> 30:08.683 Yeah. 30:08.683 --> 30:10.544 think some level of standardization is good. 30:10.544 --> 30:12.215 That is actually the Netflix insight right? 30:12.215 --> 30:13.825 It's better that kinda 30:13.825 --> 30:18.767 we're a Java shop or kind of we're a dotnet shop or kinda we're a Ruby shop because that does 30:18.767 --> 30:21.138 help us collaborate more effectively. 30:21.138 --> 30:21.598 But the 30:21.598 --> 30:26.919 excessive standardization is kind of pretending the world is simpler than it is. 30:26.919 --> 30:33.235 I think it is very relevant because we're we're seeing ways of scaling decisions with agents that we haven't seen before. 30:33.235 --> 30:34.293 And so 30:34.293 --> 30:34.796 Yeah. 30:34.796 --> 30:45.425 stomping over the local resilience of people taking context to into account and localizing decision with their own thing, that that might actually be stomped on a little bit by 30:45.425 --> 30:48.097 having organizational level harness engineering. 30:48.097 --> 30:54.022 And so I think we will see over correction and then recorrection in that area. 30:54.410 --> 30:55.210 Cool. 30:55.210 --> 31:02.956 So yeah, I one of the things you one of the things that you responded when I asked you about the radar, ~ you said, Well, Kyle, we didn't hear your voice. 31:02.956 --> 31:06.509 ~ and and true, that it w that was somewhat by design. 31:06.509 --> 31:15.675 ~ it's funny though, ~ on review when I was doing the work of putting together the radar, I thought to myself, hmm, because what I did is I fed, you know, the transcripts into 31:15.675 --> 31:20.988 Claude, and Claude comes out and says, you know, Ahmed said this Ahmed said that, and Chris Ford said this and Chris said that. 31:21.229 --> 31:21.955 But 31:21.955 --> 31:26.527 It was giving them to me like by episode, but I'm not sure it actually knew who it was speaking. 31:26.527 --> 31:28.968 So some of the things that it would attribute to a guest, 31:28.968 --> 31:30.188 it might have been me that said it. 31:30.188 --> 31:36.352 ~ and so like certain things that I recognize as like, yeah, that's me that says that, is like AI is an amplifier. 31:36.352 --> 31:37.962 I'm not trying to claim that one. 31:37.962 --> 31:42.434 Like probably somebody said that to me first, but I know I say it frequently. 31:43.335 --> 31:44.185 Yeah, no 31:44.185 --> 31:45.396 doubt, no doubt. 31:45.396 --> 31:48.551 So there is there is definitely some of Kyle in there. 31:48.551 --> 31:50.172 and I I think that is one of the things. 31:50.172 --> 31:54.727 If if your team is fast at making messes today, it's going to be faster at making messes tomorrow. 31:54.727 --> 31:56.828 That's definitely something that I said more than once. 31:56.828 --> 31:58.459 Yeah, thanks for checking out the radar. 31:58.459 --> 31:59.280 Thanks for being part of it. 31:59.280 --> 32:00.717 ~ appreciate that. 32:00.717 --> 32:03.022 I'm trying to think about what's changed since you were on. 32:03.022 --> 32:05.045 It looks like it was back in April. 32:05.045 --> 32:11.022 ~ and I guess, you know, the the you were saying when we prepped the the March of the models, right? 32:11.022 --> 32:17.590 Fable in in the paid world, ~ and and of course like Kimi Three and and others in the non paid world. 32:17.651 --> 32:22.547 ~ yeah, I mean what it it's been it's been an interesting time since since April for sure. 32:22.595 --> 32:31.593 Yeah, I would say that the number one thing that's changed for me in my professional context is that the amount of people freaking out over costs has ramped up. 32:31.593 --> 32:37.799 So, you know, we did have that faddish idea of token maxing at one point, right? 32:37.799 --> 32:42.353 Where maybe, you know, literally consuming more of a resource was considered virtuous. 32:42.353 --> 32:43.604 So we'll have a leaderboard. 32:43.604 --> 32:47.147 I don't think that lasted very long and I guess it was a little bit of a stunt. 32:47.147 --> 32:48.428 But we're seeing 32:48.448 --> 32:52.529 At least I'm certainly seeing in my professional life organizations go the other way. 32:52.529 --> 33:01.483 So, you know, that unlike us when we have our personal subscription, organizations are on, pay per token API plans of different kinds. 33:01.544 --> 33:14.267 And even from the extreme of how do I make sure that no one in my organization accidentally spins up a recursive sorcerer's apprentice agent job and costs me $50,000 33:14.267 --> 33:18.106 through to just widespread reported people 33:18.106 --> 33:27.572 Having had a budget set and then blowing that budget roughly, I think quite commonly in one week out of a month is about the common ratio between what people expected and how 33:27.572 --> 33:28.543 long it takes. 33:28.543 --> 33:28.893 This 33:28.893 --> 33:38.209 is causing a lot of consternation because people have remembered that we don't know how to measure software productivity, software development productivity. 33:38.209 --> 33:39.149 So 33:39.710 --> 33:43.412 when the costs were low, or maybe it was a few 33:43.664 --> 33:50.504 teams who were trying things out and certainly they might have been prosecuting the C-suite's agenda because the C suite was interested in getting better at AI. 33:50.504 --> 33:51.969 It was kind of fine. 33:51.969 --> 34:05.996 But as more people blow more budgets, ~ people who are responsible for being able to defend ROI of things are finding that they can't easily work out whether it's worth paying 34:05.996 --> 34:08.117 that much in tokens or not. 34:08.157 --> 34:12.825 and so that's a little bit of a kind of parties over moment in a lot of places. 34:12.825 --> 34:13.215 Yeah. 34:13.215 --> 34:14.886 which is which is interesting. 34:14.886 --> 34:26.088 I can remember hearing, you know, ~ C suite folks from various AI companies going on podcasts and saying, you know, if your engineers aren't using so many tokens per day, you 34:26.088 --> 34:26.708 know, 34:26.708 --> 34:28.717 you need to worry about that engineer's productivity. 34:28.717 --> 34:31.049 It's like, wow, that was brilliant, you know. 34:31.970 --> 34:34.670 I mean on some level there is maybe some value to it. 34:34.670 --> 34:35.860 I mean I I kinda get it. 34:35.860 --> 34:41.558 In our in our own organization, you know, so and so has a reputation of using a lot of tokens and no one's sure why. 34:41.558 --> 34:42.365 And 34:42.668 --> 34:45.549 other people, it's like, well, you're not using any, so are you learning fast enough? 34:45.549 --> 34:54.432 It's not a useless metric, but ~ certainly like like you say, we we don't know how to be healthy about our metrics yet, I don't think, in software development. 34:54.432 --> 34:58.113 So all metrics can go toxic ~ and problematic. 34:58.113 --> 35:01.114 But but it really has been interesting seeing that swing around. 35:01.114 --> 35:05.021 You know, people like my mother is sending me articles saying, AI is getting expensive. 35:05.021 --> 35:06.902 Maybe you should worry about getting into AI now. 35:06.902 --> 35:08.442 It's like, well, yes, but 35:08.843 --> 35:13.869 I mean, lots of professions, the equipment is more expensive than the wages. 35:13.869 --> 35:18.492 And in those professions, people argue about your raise less. 35:18.553 --> 35:22.746 So the problem we have as engineers is is we are the biggest cost, and while we're the 35:22.746 --> 35:29.162 biggest cost, it is worthwhile for people to argue the most about whether there's a slight increase in salary. 35:29.162 --> 35:36.163 If we did get to the stage where Jensen, who I think you're kind of alluding to, ~ you know, is having people using hundreds of thousands of 35:36.163 --> 35:45.484 tokens a year, then it becomes less worth the more we worry about the token cost, the less proportionately people might worry about the human wage costs. 35:45.484 --> 35:53.993 So I'm I I'm not sure whether it's a dystopia for those of us who are already skilled software engineers, though it it might make the economics weird for for others. 35:54.062 --> 35:54.532 Yeah. 35:54.532 --> 35:55.453 Yeah. 35:55.533 --> 36:03.646 Well well and you raise the the Jensen spector Well, to your point I raise the Jensen Spector but but to name him, I I keep thinking to myself, as long as he can make this 36:03.646 --> 36:10.212 cheaper, as long as he can keep making the chips better for less money, you know, ~ then we are going to be fine. 36:10.212 --> 36:11.773 ~ yeah. 36:11.773 --> 36:14.615 Which almost kinda brings me to another question. 36:14.615 --> 36:19.249 ~ so I've I've had this analogy bouncing around my brain for a little while and I haven't 36:19.249 --> 36:20.926 managed to talk to anybody about it. 36:20.926 --> 36:23.053 And so I'm gonna run it past you. 36:23.216 --> 36:23.898 So you 36:23.898 --> 36:24.391 Just between 36:24.391 --> 36:26.756 you and me and and our ~ thousands 36:26.756 --> 36:27.441 of listeners, yeah. 36:27.441 --> 36:28.732 Yeah, yeah, exactly. 36:28.732 --> 36:33.526 ~ and and it is the analogy of FedEx and the fax machine. 36:33.526 --> 36:40.258 ~ and so some listeners will know the story, but for for those who don't, I'll tell it. 36:40.258 --> 36:48.884 it's my understanding that when the fax machine, the humble fax machine, right, this essentially black and white printer combined with scanner, combined with modem. 36:48.964 --> 36:56.766 that you know can sit on a table and can be plugged into a phone line and plugged into a power line, they can now call each other and send documents back and forth. 36:56.766 --> 37:05.939 And you can drop, you know, a letter in the scanner side and it will scan it and then it will call somebody up and it will get printed remotely on the other side because print 37:05.939 --> 37:07.049 jobs are essentially data. 37:07.049 --> 37:08.759 And so we can move those over phone lines. 37:08.759 --> 37:13.270 And it as I understand it, FedEx when they first saw these, thought, this is great. 37:13.431 --> 37:15.921 You know, Chris is in Barcelona, Kyle's in Toronto. 37:15.921 --> 37:18.616 We can put a fax machine in each office. 37:18.616 --> 37:23.059 If Chris wants us to send something to Kyle really, really quickly, we can fax it. 37:23.059 --> 37:24.609 And it won't look as nice. 37:24.609 --> 37:28.682 It won't be the original, but we can get it to Kyle for so much less. 37:28.682 --> 37:31.203 And like maybe we don't have to charge all that less. 37:31.203 --> 37:33.584 And when fax machines first came out, they were 10 grand a pop. 37:33.584 --> 37:37.366 So this maybe seemed like it was gonna be a thing. 37:37.366 --> 37:39.768 But what they didn't realize is that they would end up being 80. 37:39.768 --> 37:47.565 ~ and so pretty soon everybody would have a fax machine, you know, and and big companies almost immediately realized well, yeah, we should just do. 37:47.565 --> 37:49.566 Instead of paying FedEx to do this. 37:49.566 --> 37:55.444 And eventually medium-sized companies, small companies, people, and then eventually the internet, and it was all all over. 37:55.988 --> 37:59.549 I wonder if this might apply to AI. 37:59.549 --> 38:02.220 You know, companies like Tiiny AI are building these 38:02.220 --> 38:06.111 tiny little things that you can now run a reasonable model in. 38:06.211 --> 38:14.393 And the better and better the algorithms get around compacting models down and you know and reducing the amount, making it more efficient for tokens to do their work or for 38:14.393 --> 38:15.573 models to do their work. 38:15.573 --> 38:15.974 yeah. 38:15.974 --> 38:19.536 Like will this be the kind of thing that, you know, we're just gonna run it locally? 38:19.536 --> 38:23.029 At that point, that's really gonna change the economics of this entire game. 38:23.029 --> 38:23.839 So but what do you 38:23.839 --> 38:24.060 think? 38:24.060 --> 38:24.790 Like am I dreaming? 38:24.790 --> 38:26.471 Like what what do you think? 38:26.483 --> 38:29.345 Well, I I can certainly come up with a direct parallel from what I've seen. 38:29.345 --> 38:39.504 So the problem there was that FedEx were thinking about competing against their competitors as though their competitors didn't also have fax machines, or also their 38:39.504 --> 38:40.324 customers didn't also. 38:40.324 --> 38:45.769 So they're kind of imagining a world where they could compare everything else saying static and then moving forward. 38:45.769 --> 38:54.416 I do actually see that quite a lot in the software world where companies kind of simultaneously going, ~ code is code is cheap with coding agents, that's great. 38:54.416 --> 38:55.729 But they're still pretending that 38:55.729 --> 38:58.321 If that were true that code is valuable and that's 38:58.321 --> 38:59.761 not an equilibrium that can hold. 38:59.761 --> 39:05.996 So, you know, companies are thinking, okay, I can have higher margins because I can do what I used to do much more cheaply. 39:05.996 --> 39:07.037 That's awesome. 39:07.037 --> 39:11.820 But they're may they're thinking the fallacy that they're operating in a static world and they're comparing 39:11.820 --> 39:19.085 their offering to what their competitors or substitutes were able to do before coding agents became popular. 39:19.085 --> 39:23.147 So I think there's definitely a lot of like equilibrium settling that will need to happen. 39:23.148 --> 39:25.529 It would definitely be ironic though that if the 39:25.813 --> 39:37.089 you know, if the large L L providers having, you know, done the R and D and received a lot of capital to eat the world with L L Ms, were out competed by 39:37.089 --> 39:41.825 locally artisanally home grown run things on everyone's laptop. 39:41.825 --> 39:43.319 ~ but I guess we'll see. 39:43.319 --> 39:53.897 models are good enough and fast enough, once they're smarter than us, you know, like and and we can run it on, you know, our phone, ~ then does it matter if it's 10x smarter than 39:53.897 --> 39:55.358 us or a hundred X smarter than us? 39:55.358 --> 39:56.449 Like maybe. 39:56.449 --> 39:57.872 and and this may not be for a while. 39:57.872 --> 39:59.921 I mean things have changed fast in the 39:59.921 --> 40:02.524 last two years, but maybe not this fast. 40:02.524 --> 40:08.618 But so yeah, if but if it did happen, if it did get to the point where most people could get just fine results. 40:08.678 --> 40:13.181 ~ on their phone or on their PC, on their Mac, laptop, whatever it is. 40:13.181 --> 40:14.234 I don't know. 40:14.234 --> 40:16.551 Like I wonder what we're paying other people for the tokens for. 40:16.551 --> 40:17.341 Yeah. 40:17.440 --> 40:19.722 I guess so will our just fine remain static? 40:19.722 --> 40:28.518 Because that that may sound a little bit like I the thing that Bill Gates didn't say about was it I don't know, like a it should be enough for anyone, right? 40:28.518 --> 40:29.248 I don't think he actually said 40:29.248 --> 40:33.981 that, but but but that's kind of saying, okay, well that amount of intelligence should be just fine. 40:33.981 --> 40:36.153 I can't see I can't see why anyone would need more than that. 40:36.153 --> 40:46.780 And I think I do have a little bit of faith slash pessimism, I'm not sure what emotion to have, about our ability to find ways to waste, consume 40:47.582 --> 40:51.477 at least desire the additional capacity if it if it exists in the world. 40:51.477 --> 40:52.258 I think the idea 40:52.258 --> 40:53.639 of us going, Yeah, that's fine. 40:53.639 --> 41:00.106 We've had enough, ~ unfortunately is not something that our civilization has done very often in the past. 41:00.207 --> 41:01.107 So we'll see. 41:01.107 --> 41:02.771 the hot thing is right now. 41:02.771 --> 41:06.401 Whatever the thing that used to be hot, we want to optimize the heck out of that thing. 41:06.401 --> 41:07.672 But whatever's hot right now. 41:07.672 --> 41:11.366 talking about climate change, but you would no you were talking about more metaphorical, but yeah. 41:13.529 --> 41:14.736 That's a whole other podcast. 41:14.736 --> 41:15.977 Yeah, exactly. 41:15.977 --> 41:16.487 Fair enough. 41:16.487 --> 41:18.718 I w I I I won't try to sidetrack there. 41:18.718 --> 41:29.667 But yeah, I I I th I think there there will be desire to increase consumption and things becoming cheaper increases consumption is the Jevons paradox that has suddenly become hot 41:29.667 --> 41:30.807 ~ again, right? 41:30.807 --> 41:31.308 So 41:31.308 --> 41:32.949 ~ it will be interesting to see. 41:32.949 --> 41:39.533 But I maybe there's a stage that we could consider in between the two of running locally and using the frontier 41:39.533 --> 41:43.393 model providers, which is there are a lot of companies that are using 41:43.393 --> 41:57.759 their cloud hosting providers to run models for their own in in house inference as a way of either an insurance policy against rising token costs from you know anthropic open AI 41:57.759 --> 41:58.979 Google, 41:58.979 --> 42:10.636 or even just a way of trying to do it cheaper because as long as they have the ability to capacity plan and manage utilization, they can just take the margin of the 42:10.636 --> 42:12.669 cloud of the the token provider out of that. 42:12.669 --> 42:21.681 So they so I think companies are definitely making they're trying to make sure that they have levers so that they aren't dependent on a supplier that might decide to massively 42:21.681 --> 42:22.750 hike its costs. 42:22.750 --> 42:24.862 Yeah, well, or or foreign government. 42:24.862 --> 42:26.899 Like we we had a guest on Ahmed 42:26.899 --> 42:29.811 Misbah and he's he's working on Arabic LLMs, right? 42:29.811 --> 42:31.283 And I think it's it's a couple of things. 42:31.283 --> 42:39.279 One is, you know, the language and being able to to yeah, there's an interesting computer science thing about what happens when we teach at other languages besides English. 42:39.279 --> 42:43.984 ~ but I think more to that, it's also just sort of sovereignty, you know. 42:43.984 --> 42:50.389 ~ and you know, Canada, where I live, ~ we have one of our telcos is standing up data centers. 42:50.389 --> 42:52.330 know, with NVIDIA GPUs in them. 42:52.330 --> 42:55.270 We're not we're not creating our own GPUs, but 42:55.330 --> 42:58.951 but the idea of a sovereign AI, we do have ~ local model companies. 42:58.951 --> 43:01.452 You know, there's a Canadian model ~ I'm not gonna remember the name. 43:01.452 --> 43:02.569 That's gonna pain me. 43:02.569 --> 43:06.110 but yeah and so the idea that we can't all be beholden. 43:06.110 --> 43:13.592 We we saw what happens when the government of the US decides to tell Anthropic, I'm sorry your model is too something 43:13.592 --> 43:15.903 and so you have to turn it off now. 43:16.503 --> 43:18.920 I'm not sure we all want to be beholden to that in the end. 43:18.920 --> 43:19.417 Yeah. 43:19.417 --> 43:28.233 Yeah, if I understand correctly, there's rumblings in China of actually maybe not wanting people to use the Chinese models as freely in the West, ~ the way the way that's 43:28.233 --> 43:28.603 happening. 43:28.603 --> 43:29.543 So, you know, there could be 43:29.543 --> 43:31.305 controls in that direction as well. 43:31.305 --> 43:32.895 Or, you know, I mean I live in Europe 43:32.895 --> 43:43.652 and ~ not wanting to depend on external parties, be they North American or Asian for critical infrastructure is a really hot 43:43.652 --> 43:45.063 topic in Europe as well. 43:45.063 --> 43:46.844 Yeah, yeah, yeah, for sure. 43:46.844 --> 43:48.685 So I think that's gonna be an element of it. 43:48.685 --> 43:55.718 And you've you've almost we're almost into like globalism now and you know, should you know, should countries depend on one another or should we be isolationists? 43:55.718 --> 43:58.589 Like a we're definitely trying to solve the world's problems here, Chris. 43:58.589 --> 43:59.349 Yeah. 44:00.717 --> 44:01.299 Yep, yep. 44:01.299 --> 44:02.463 We might need a a third 44:02.463 --> 44:05.102 a third podcast to to completely finish the job, but you never know. 44:05.102 --> 44:05.458 Yeah. 44:05.458 --> 44:09.161 My my only counterpoint to your argument around Jevon's paradox and we always 44:09.161 --> 44:10.197 want more of a thing. 44:10.197 --> 44:11.473 I mean, yes, you're right. 44:11.473 --> 44:12.714 And I could see it going that way. 44:12.714 --> 44:15.546 Like it certainly has gone that way with say bandwidth. 44:15.546 --> 44:22.482 Like why for instance, you know, my first ~ cool cable modem was nine hundred and sixty kilobit per second. 44:22.482 --> 44:24.223 That's almost a megabyte, Chris. 44:24.223 --> 44:24.974 my god. 44:24.974 --> 44:29.918 A T one is a three thousand dollar a month thing, and this is almost that and it only cost me a hundred and fifty. 44:29.918 --> 44:31.648 And now I've got a gigabit connection. 44:31.648 --> 44:34.125 And if someone offered me a ten gigabit, I would take it. 44:34.125 --> 44:36.727 What I knew what I'm doing with the other nine gigabits, I don't know. 44:36.727 --> 44:37.967 But I would take it, right? 44:37.967 --> 44:39.748 So maybe it goes that way. 44:39.768 --> 44:47.352 On some level, it's gone that way because it was commoditized and because it was cheap, and because we could keep saying yes at this at roughly the same hundred bucks a month. 44:47.733 --> 44:52.035 I think the argument on the fax side was with fax similes, we it didn't really matter. 44:52.035 --> 44:57.508 We didn't need to be able to send, you know, 7,000 page faxes in thy in three seconds. 44:57.508 --> 44:58.719 That was never of value. 44:58.719 --> 45:03.591 ~ that basically what the fax was doing was like, look, I got a wet signature, you know. 45:03.939 --> 45:06.630 well it isn't wet, but I got a signature, you know. 45:06.630 --> 45:09.271 And that's that's mostly what it was doing in the end. 45:09.271 --> 45:14.662 ~ you could send the other 25 pages in a Word doc in an email eventually, you know. 45:14.662 --> 45:17.533 And so Good Enough had a different bound. 45:17.533 --> 45:24.195 ~ and and because and I think prices were were something about what kept that in check as well. 45:24.195 --> 45:24.575 Like 45:24.575 --> 45:32.740 if ~ we if we wanted to send, you know, that 7,000 page document in two seconds, we eventually did that on our gigabit line 45:32.740 --> 45:34.731 Computers because again, that's where the commodity was. 45:34.731 --> 45:37.112 That's where you could commodify and bring the price down. 45:37.112 --> 45:39.994 So I don't so yes, Jevon's paradox, yes. 45:39.994 --> 45:48.148 I mean, you know, I'm I'm a big electrification fan, and there is definitely a threat that the more we electrify, the more we use electricity, which might just mean we have to bring 45:48.148 --> 45:51.300 the coal plants back on, and that's not what I'm after. 45:51.380 --> 45:53.261 So I get it, I'm with you. 45:53.261 --> 45:54.896 But I think there's another side of it though. 45:54.896 --> 46:00.735 As long as we can keep optimizing the hardware and we can stay just ahead of the efficiency curve. 46:00.843 --> 46:02.417 And we can commodify, commodify. 46:02.417 --> 46:07.557 Well, eventually I will have that ten gigabit connection ~ to my house and it'll only be a hundred bucks a month. 46:07.557 --> 46:09.831 They probably already have it in Korea, you know. 46:09.952 --> 46:10.455 So 46:10.455 --> 46:16.048 I think you're definitely right to think about diminishing marginal returns and what people will actually want. 46:16.048 --> 46:20.831 I think that that that that's true and no no company is kind of owed a living by the world. 46:20.831 --> 46:28.635 ~ so yeah, I think you you do you make a good point and there may be people certainly part of the supply chain that are routed around like ~ like 46:28.635 --> 46:29.580 FedEx was. 46:29.580 --> 46:30.820 Yeah, yeah, yeah. 46:30.820 --> 46:31.591 It's interesting times. 46:31.591 --> 46:37.036 maybe the thing that I could add on to that idea about to what degree people will be willing to pay more for stuff. 46:37.036 --> 46:45.292 I think I believe that that attribute of humanity might make it very difficult to improve the quality of software using coding agents. 46:45.353 --> 46:56.296 So I think I do believe that if you were to do a great job with all the different kind of harnesses, the feed forward, the feedback, guardrails and whatever, that in theory, 46:56.296 --> 47:03.622 someone who then maintains the same amount of energy and discipline can do a better job than if they were coding by hand in many cases. 47:03.622 --> 47:05.753 So I don't you I I I I don't 47:05.814 --> 47:11.085 subscribe to kind of a slop narrative to that implies that coding agent will automatically do a worse worse job. 47:11.085 --> 47:13.475 I think you can do a better job with coding agent, especially 47:13.475 --> 47:18.437 if, for example, you invest some of the time you save in improving the harness or doing more review. 47:18.477 --> 47:28.990 The conclusion I've reached though is that if we looked like we were accidentally creating greater quality, we would respond not by achieving that quality, but by lowering our 47:28.990 --> 47:35.291 standards or increasing our haste until we manage to lower quality back to where it was before. 47:35.335 --> 47:44.430 Because the level of quality we have in software is not it's not the maximum we know how to achieve, or at least I don't think it's been in any code base I've been part of. 47:44.430 --> 47:46.491 It's basically it's lowers 47:46.551 --> 47:48.022 the conditions we'll accept. 47:48.022 --> 47:53.635 And if it gets lower and things start breaking, then people will get it so it's just above the waterline. 47:53.635 --> 48:00.439 So there there's a there's a paradox called Wirth's it's Wirth's law or Wirth's paradox, the after the computer science. 48:00.439 --> 48:04.271 And that explains why using a computer doesn't get faster. 48:04.701 --> 48:08.423 Like you think it should get faster to use a computer because computers are getting faster. 48:08.423 --> 48:09.844 How could that how could that not be the case? 48:09.844 --> 48:10.245 Well, 48:10.245 --> 48:19.131 because the faster the computer gets, the more layers of crap we put on our stack until it slows just at the point where we can bear it. 48:19.131 --> 48:26.936 You would think, you know, I've heard people say many times, Well, hang on a second, can we just stop developing more weird layers of abstraction for like one generation of 48:26.936 --> 48:28.577 hardware and just take the win? 48:28.577 --> 48:30.358 Wouldn't that wouldn't everyone love that? 48:30.358 --> 48:34.589 And maybe they would, but certainly the revealed preference is that they don't really 48:34.589 --> 48:38.469 give a crap of that and that they're willing, at least us as software developers are willing to 48:38.469 --> 48:40.171 to to fritter away. 48:40.171 --> 48:49.963 So I think we might have the same quality equilibrium problem where even if the techniques are available to produce higher quality software with better modularity, with more 48:49.963 --> 48:52.746 stability, with fewer bugs with coding agents, 48:52.746 --> 48:53.214 Yeah. 48:53.214 --> 48:58.465 if we achieve that, we'll say, well, that means we can take a little bit less care next time. 48:58.465 --> 49:04.417 That means that we can go a little bit faster next time until the quality drops to the level that people were kind of 49:04.842 --> 49:06.731 Grudgingly accepting anyway. 49:06.731 --> 49:09.131 So I think there's I think there's an equilibrium force there. 49:09.131 --> 49:09.701 I like it. 49:09.701 --> 49:12.243 If I if I borrow the analogy for a minute and 49:12.243 --> 49:20.848 and riff on it, then what's coming to mind for me, ~ and I I've just now heard of Wirth's law, so I'm I'm probably gonna get it wrong, but I think it is sort of a counterpoint, I 49:20.848 --> 49:22.549 guess, to to to Moore's Law. 49:22.549 --> 49:31.093 And I always thought, like, yes, you know, you know, the ~ w CPUs get faster and faster and faster, so why don't they get slower and and I I always figured it was sort of two two 49:31.093 --> 49:34.555 debts in the budget, ~ two things that that was going to. 49:34.555 --> 49:35.406 Maybe maybe three. 49:35.406 --> 49:37.957 Like maybe one is like, okay, you know. 49:38.299 --> 49:42.953 faster can software can go faster, so we don't have to performance optimize as much. 49:42.953 --> 49:47.617 So that's like going to the margins of the software development companies in theory. 49:47.617 --> 49:50.049 They didn't have to spend as much money on performance optimization. 49:50.049 --> 49:52.229 So I think that's a one bucket that it's going to. 49:52.229 --> 49:55.504 I think another bucket that it's going to is like shiny features. 49:55.504 --> 50:02.409 Like you and I right now are in different countries of the world and yet we're having a very high resolution conversation as though we were not. 50:02.510 --> 50:05.722 That was not possible on my 486 DX33. 50:05.722 --> 50:07.013 It just was not. 50:07.033 --> 50:08.794 ~ and so that's that's like a budget. 50:08.794 --> 50:12.827 But I think there's this other budget, which is like technical architectural debt. 50:12.827 --> 50:15.218 Like Linux still has POSIX in it. 50:15.519 --> 50:16.680 and so so does Windows. 50:16.680 --> 50:21.973 Like, because for a minute we all thought that we needed this layer of translation called POSIX, 50:21.973 --> 50:24.805 which from what I can tell didn't ever do anything, right? 50:24.805 --> 50:27.440 Like no one ever actually got any value out of POSIX. 50:27.440 --> 50:28.158 I'm probably wrong. 50:28.158 --> 50:29.409 Probably someone did. 50:29.409 --> 50:31.951 But all the computers still have a POSIX layer in them. 50:31.951 --> 50:35.463 So I figured like if Morris Law ever did stop. 50:35.479 --> 50:38.133 That it would hopefully the budget would come out of that. 50:38.133 --> 50:45.183 Like we would sort of go back to our technical debt, these old abstractions that don't matter anymore, and like let's strip them out of the operating system. 50:45.183 --> 50:47.125 Are we gonna get a couple of milliseconds back? 50:47.125 --> 50:47.596 I don't know. 50:47.596 --> 50:50.330 But I ~ so how does that apply to this? 50:50.330 --> 50:51.712 Not not sure. 50:51.712 --> 50:53.003 But what do you think? 50:53.134 --> 50:59.796 Well, I think well I think it it it it's a neat example because it works for both, you know, technical debt for quality and for speed. 50:59.956 --> 51:08.158 I I I I guess I think we'll find our ways to screw up software in different and imaginative ways in the future is maybe my my conclusion. 51:08.158 --> 51:08.868 So it 51:08.868 --> 51:18.640 will be possible well was always possible possible with coding agent, it will be possible to maybe refactor or remove code where we have a clear idea of what the outcome should be, 51:18.640 --> 51:21.025 but the human labor to do it would be 51:21.025 --> 51:26.728 gargantuan but we can specify the problem and put a harness around it really clearly, sure, then we can go ahead and do it. 51:26.728 --> 51:27.548 ~ 51:27.989 --> 51:33.512 still you have to worry about compatibility and distribution and things like that that would be difficult if people are 51:33.512 --> 51:38.515 using POSIX compatibility as as as as something as the interface that wouldn't be as easy. 51:38.715 --> 51:48.641 so I it's it's not that the the same specific subsystems that slow us down or exactly the same anti-patterns that cause quality problems. 51:48.779 --> 51:49.580 will be present. 51:49.580 --> 51:58.756 So maybe in the future C based memory bugs won't exist anymore because we'll use coding agents to rewrite everything in Rust and and that'll be done. 51:58.996 --> 52:00.627 But I think there'll be a substitution 52:00.627 --> 52:04.720 of other quality problems that we will tolerate up until 52:04.720 --> 52:11.705 our level of ~ acceptability of our employers slash the patience of our users and just at the point 52:11.705 --> 52:16.018 before they throw the laptop down in disgust, we will 52:16.018 --> 52:17.909 we will step in and just 52:17.943 --> 52:20.594 Just keep it just good enough. 52:20.594 --> 52:28.656 so I I I I know this is a kind of a pessimistic take, but I'm just trying to imagine the organizations I've seen tolerating software improving. 52:28.656 --> 52:38.206 And I just can't say 'cause if there was a bunch of engineers that came in and said, Okay, great news, software is twice as reliable, let's say, I'm talking about kind of bugs and 52:38.206 --> 52:45.268 things like that as it's been before, the you know, a lot of the response of organizations will be, Well, why did you spend so much time on it then? 52:45.268 --> 52:46.908 ~ and 52:47.337 --> 52:48.417 Unless unless 52:48.417 --> 53:01.622 maybe as consumers we can demand our share of the surplus introduced by coding agents to be paid to us in increased stability or performance, ~ I think it will get ~ eaten maybe 53:01.622 --> 53:07.702 by like many other things into the you know, into the profits of the company that is sitting at the right point in the value chain. 53:07.702 --> 53:08.752 Yeah, a hundred percent. 53:08.752 --> 53:16.274 I I can remember, and this is now twenty years plus this is pre-thought works days for me, so I won't name anybody, but it's it's nobody in my life currently. 53:16.375 --> 53:22.026 Going to the CEO and this is when I was first discovering test driven development and unit testing, all of this stuff. 53:22.026 --> 53:26.057 And I remember going to the CEO and saying, Boss, we're gonna start unit testing everything. 53:26.057 --> 53:27.292 Quality's gonna get better. 53:27.292 --> 53:29.148 I it might slow things down a little bit. 53:29.148 --> 53:30.378 I don't know. 53:30.418 --> 53:34.419 And he's like, I'm not sure that quality matters that much, Kyle. 53:34.419 --> 53:35.540 No, no word of the lie. 53:35.540 --> 53:36.564 And he's not wrong. 53:36.564 --> 53:40.129 You know, they invest or put money in because they want to see money out. 53:40.530 --> 53:41.282 He wasn't wrong. 53:41.282 --> 53:43.576 Now, luckily the unit test sped us up. 53:43.576 --> 53:48.000 It didn't slow us down because, you know, it just helped f cut things off before they launched. 53:48.000 --> 53:54.572 the there is the quality is free, which I think definitely does apply to a certain kinds of quality and unit testing 53:54.572 --> 53:55.013 is great. 53:55.013 --> 53:57.298 It's definitely a quality is free phenomenon, 53:57.298 --> 53:57.561 yeah. 53:57.561 --> 53:58.391 Yeah, yeah, yeah, yeah. 53:58.391 --> 53:59.301 But not all of them are. 53:59.301 --> 54:05.744 The other thing that's coming up for me is they originally like internet lines were sort of like the same telco lines. 54:05.744 --> 54:16.268 ~ and telco lines were built for telco levels of reliability that needed to be there because they were centrally managed and circuit switched instead of packet switched. 54:16.308 --> 54:24.862 And people started just saying, Well, screw it, just make it an Ethernet line between two cities, you know, and it obviously can't be the same copper coax you 54:24.862 --> 54:26.704 you'd end up having RF issues and so on. 54:26.704 --> 54:36.442 But but still like drop the telco levels of reliability that telco expected because the internet was architected not to care as much about reliability. 54:36.442 --> 54:43.728 And that that caused, I can remember buying a T1 line in Kingston, Ontario, and it was three thousand dollars a month. 54:43.728 --> 54:49.192 And they, you know, I got a quote for, you know, for basically the same mega and a half from somebody else. 54:49.192 --> 54:51.624 And it was like, I don't know, $300 a month. 54:51.624 --> 54:53.781 And I remember calling up my bell rep and he was like 54:53.781 --> 54:56.964 Well, you could, but the quality is gonna be terrible. 54:56.964 --> 54:57.814 But Chris it wasn't. 54:57.814 --> 55:04.290 Like it was fine, you know, like maybe ~ the the likelihood that that there would be a problem. 55:04.290 --> 55:14.128 Well, that's not even true, because later I I don't know if it's the same year, but a bell technician dropped a a wrench in an open router and took out like most of Toronto for a 55:14.128 --> 55:14.818 couple of hours. 55:14.818 --> 55:18.652 So even my three thousand dollar T one line, and that's when I sort of gave up and said, Okay, this is dumb. 55:18.652 --> 55:22.104 ~ so I do think there is something like that going. 55:22.142 --> 55:31.707 Around quality where there's this sort of like b if we if we we have to just understand now that everybody's gonna be doing four things at once, everybody's got a an app that 55:31.707 --> 55:37.500 they just put in the marketplace, everybody's got a startup that they launched, we're gonna have more things, right? 55:37.500 --> 55:39.551 Did as much care go into each one of them? 55:39.551 --> 55:44.284 Probably not, because you could do it faster, which is why you did it in the first place. 55:44.284 --> 55:45.425 But is that a bad thing? 55:45.425 --> 55:46.395 Maybe not, right? 55:46.395 --> 55:51.560 Like it might be like that hundred and fifty dollar T one line where it it really was okay. 55:51.560 --> 55:52.104 In the end. 55:52.104 --> 55:52.455 Yeah. 55:52.455 --> 55:53.345 Some of the quality 55:53.345 --> 55:57.911 serves humans' needs better better than if if it was high quality in a purist way. 55:57.911 --> 55:59.300 That could be true. 55:59.300 --> 56:00.201 Yeah. 56:00.242 --> 56:00.592 Yeah. 56:00.592 --> 56:02.424 And that's not to say it'll only go well. 56:02.424 --> 56:07.398 I'm I'm sure in places it will go terribly and will tell interesting stories and it'll be funny. 56:07.978 --> 56:16.632 Yes, well I guess, you know, with all sorts of feedback loops, if it takes a long time to figure out that you've done something bad, then 56:16.632 --> 56:20.440 you're much less likely to gracefully adjust, you know. 56:20.440 --> 56:21.314 A steering wheel of the 56:21.314 --> 56:25.716 car where you can where you ~ can correct course correct instantly is fine. 56:25.716 --> 56:33.959 If you were trying to drive a car with a steering wheel that had a lag of, I don't know, let's say a second, your chances of getting through any you know, if seconds not very 56:33.959 --> 56:34.369 long, but 56:34.369 --> 56:36.659 your chances of getting through traffic 56:36.659 --> 56:38.270 unscathed would be very low. 56:38.270 --> 56:38.909 And so 56:38.909 --> 56:48.483 I might the analogy I'm making is that for things like, I don't know, accidentally lowering the quality of the software in your state or poisoning the well of open source, 56:48.483 --> 56:50.831 for example, if it takes a 56:50.831 --> 56:56.136 a while before the implications of our actions to come back and bite us, that doesn't 56:56.136 --> 57:01.298 like it we're not necessarily going to take holistic long term rational decisions. 57:01.298 --> 57:01.748 We might 57:01.748 --> 57:06.500 find that we've messed things up before before we we know it. 57:06.752 --> 57:11.212 I think it's it's it's generally thought of that we are bad at seeing those kinds of things as a species. 57:11.212 --> 57:11.772 In fact. 57:11.772 --> 57:12.814 Yeah. 57:12.916 --> 57:21.832 Yeah, well that maybe if I could relate it to kind of ~ my my theme, I think we talk a lot in terms of feed forward about how we respond in a society. 57:21.832 --> 57:23.223 ~ we should do this. 57:23.223 --> 57:24.854 Okay, everybody, let's let's do this. 57:24.854 --> 57:28.206 We we imagine that is how as human beings we make decisions. 57:28.206 --> 57:41.295 But I think, you know, feedback and things going wrong and being forced to do things when there's no alternative is much more how things change at a social level over time. 57:41.295 --> 57:42.846 And so if we don't have the 57:42.940 --> 57:51.944 if we're in a context where we don't have good feedback and maybe this relates to the climate change question the message before, if we you know, if our if our actions and our 57:51.944 --> 58:00.978 con our consequences are separated by enough distance, ~ you know, it it might take us a a while to to to change what we do. 58:01.086 --> 58:01.797 Yeah. 58:01.797 --> 58:02.698 Fingers crossed. 58:02.698 --> 58:04.559 We're gonna need all the luck we can get right now. 58:04.559 --> 58:05.087 Yeah. 58:05.087 --> 58:05.790 so. 58:06.641 --> 58:08.192 Chris, thank you so much. 58:08.192 --> 58:10.704 This has been great having you on a second time. 58:11.114 --> 58:11.475 Thank you. 58:11.475 --> 58:20.841 I I feel like I was you got pessimistic, Chris, this time round in in in various ways, but ~ I'm definitely really enjoyed the conversation and ~ hopefully we've increased the 58:20.841 --> 58:22.714 quality of our ignorance together. 58:24.372 --> 58:25.314 Amen.