1 00:00:00,000 --> 00:00:06,200 Welcome to the Azure Security Podcast, 2 00:00:06,200 --> 00:00:09,400 where we discuss topics relating to security, privacy, 3 00:00:09,400 --> 00:00:13,520 reliability, and compliance on the Microsoft Cloud Platform. 4 00:00:13,520 --> 00:00:16,000 Hi, everybody. Welcome to Episode 42. 5 00:00:16,000 --> 00:00:18,600 This week, it is myself and Mark. 6 00:00:18,600 --> 00:00:20,720 We also have a guest, Dave Lubash, 7 00:00:20,720 --> 00:00:22,520 who's here to talk to us about Azure Monitor, 8 00:00:22,520 --> 00:00:25,840 both Gladys and Sarah are on vacation. 9 00:00:25,840 --> 00:00:27,720 But before we get to our guests, 10 00:00:27,720 --> 00:00:29,880 why don't we take a lap around the news? 11 00:00:29,880 --> 00:00:31,680 Mark, why don't you kick things off? 12 00:00:31,680 --> 00:00:34,520 Yeah, the big thing that I wanted to make sure folks 13 00:00:34,520 --> 00:00:36,120 are aware of this week, 14 00:00:36,120 --> 00:00:39,200 actually happened just a day or two ago, 15 00:00:39,200 --> 00:00:42,000 is the release of the Zero Trust Commandments. 16 00:00:42,000 --> 00:00:44,640 For those that are familiar with a little bit of the history, 17 00:00:44,640 --> 00:00:46,880 this is essentially the updated version or 18 00:00:46,880 --> 00:00:50,440 the replacement for the original Jericho Commandments, 19 00:00:50,440 --> 00:00:52,080 that essentially set out 20 00:00:52,080 --> 00:00:56,720 some really clear specific truths around security 21 00:00:56,720 --> 00:00:58,760 and what security need to be modernized. 22 00:00:58,760 --> 00:01:02,400 These are heavily linked to and based on those, 23 00:01:02,400 --> 00:01:05,080 but it's a modern one for the age of 24 00:01:05,080 --> 00:01:08,520 Cloud and how things have changed in the past decade or so. 25 00:01:08,520 --> 00:01:11,640 Really proud of this work that participated 26 00:01:11,640 --> 00:01:13,920 very actively in the open group work around this. 27 00:01:13,920 --> 00:01:15,680 That just got published by the open group. 28 00:01:15,680 --> 00:01:18,080 There's just to cover really quickly, 29 00:01:18,080 --> 00:01:21,280 Validate Trust Explicitly is one of the first ones, 30 00:01:21,280 --> 00:01:24,320 which is very much the exact opposite of 31 00:01:24,320 --> 00:01:27,240 assuming trust is to actually validate it and build it 32 00:01:27,240 --> 00:01:29,840 based on explicitly validated trust. 33 00:01:29,840 --> 00:01:33,000 Maybe like modern work because of effectively 34 00:01:33,000 --> 00:01:35,440 IT and the business units and 35 00:01:35,440 --> 00:01:37,560 the users have a vote in today's world, 36 00:01:37,560 --> 00:01:39,480 and so how do you incorporate that in? 37 00:01:39,480 --> 00:01:43,760 There's some interesting elements that we get to put in there around. 38 00:01:43,760 --> 00:01:46,560 One of the things we realized is how important it is to have 39 00:01:46,560 --> 00:01:49,000 the accountability land in the right place. 40 00:01:49,000 --> 00:01:53,680 If you essentially hold your CISO accountable for decisions they didn't make, 41 00:01:53,680 --> 00:01:55,600 you're not going to get the best decisions 42 00:01:55,600 --> 00:01:58,360 by the people that are making them or by the CISO. 43 00:01:58,360 --> 00:02:01,160 You want to make sure that you're keeping 44 00:02:01,160 --> 00:02:04,120 accountability for risk where all the other risk is, 45 00:02:04,120 --> 00:02:07,800 which is oftentimes on the asset owners in the business. 46 00:02:07,800 --> 00:02:10,440 That's one of the key pieces there. 47 00:02:10,440 --> 00:02:14,800 Pervasive security, this was directly tied to the original Jericho form. 48 00:02:14,800 --> 00:02:16,720 Securing assets by value, 49 00:02:16,720 --> 00:02:18,560 don't waste your time on the stuff that doesn't matter, 50 00:02:18,560 --> 00:02:21,520 the proverbial cafeteria menu. 51 00:02:21,520 --> 00:02:23,600 Spend it on the things that actually make 52 00:02:23,600 --> 00:02:25,600 a difference to the organization, 53 00:02:25,600 --> 00:02:29,040 very asset-centric, data-centric type of approach, 54 00:02:29,040 --> 00:02:30,840 simple and sustainable. 55 00:02:30,840 --> 00:02:34,720 Otherwise, you get lost in complexity and you end up 56 00:02:34,720 --> 00:02:36,960 not having effective security we found. 57 00:02:36,960 --> 00:02:39,840 The attackers have time to figure out the weaknesses there, 58 00:02:39,840 --> 00:02:42,440 so make sure your people can understand and manage it. 59 00:02:42,440 --> 00:02:45,280 Utilize lease privilege, no surprise there. 60 00:02:45,280 --> 00:02:46,960 Improve continuously. 61 00:02:46,960 --> 00:02:50,280 This is a critical thing we found as everything is changing, 62 00:02:50,280 --> 00:02:54,920 business models, Cloud, threat actors, security capabilities. 63 00:02:54,920 --> 00:02:57,560 You really have to have a program built around 64 00:02:57,560 --> 00:02:59,840 continuous learning, continuous improvement. 65 00:02:59,840 --> 00:03:01,760 Then making informed decisions, 66 00:03:01,760 --> 00:03:03,120 you got to make it based on data. 67 00:03:03,120 --> 00:03:04,600 That means you have to gather the data, 68 00:03:04,600 --> 00:03:06,080 you have to use the data, 69 00:03:06,080 --> 00:03:09,320 and you have to constantly look to do we need to get more data 70 00:03:09,320 --> 00:03:11,880 in or using the data we have, right? 71 00:03:11,880 --> 00:03:15,560 But yeah, I'm really proud about how these came out. 72 00:03:15,560 --> 00:03:18,120 I highly encourage people to check those out. 73 00:03:18,120 --> 00:03:19,840 We got the link in the show notes. 74 00:03:19,840 --> 00:03:23,680 Yeah, a few things peaked my interest over the last few weeks. 75 00:03:23,680 --> 00:03:26,880 The first one is in a product that's near and dear to my heart, 76 00:03:26,880 --> 00:03:28,680 and that's Azure SQL DB. 77 00:03:28,680 --> 00:03:31,680 There's now the ability to enable just 78 00:03:31,680 --> 00:03:34,000 Azure Active Directory authentication. 79 00:03:34,000 --> 00:03:37,040 Historically, SQL Server supported 80 00:03:37,040 --> 00:03:39,360 through three authentication mechanisms starting up with 81 00:03:39,360 --> 00:03:41,520 the original SQL Server authentication, 82 00:03:41,520 --> 00:03:43,520 where all the authentication and the identities were 83 00:03:43,520 --> 00:03:45,400 managed by SQL Server. 84 00:03:45,400 --> 00:03:46,560 That's been around forever. 85 00:03:46,560 --> 00:03:49,280 I mean, that's literally been around forever since I 86 00:03:49,280 --> 00:03:51,640 first started working with SQL Server back in, 87 00:03:51,640 --> 00:03:54,800 dare I say, OS 2 and Landman days. 88 00:03:54,800 --> 00:03:57,800 It's essentially not really changed that much. 89 00:03:57,800 --> 00:04:02,120 Then with Windows, we added Windows authentication, 90 00:04:02,120 --> 00:04:03,440 that included Kerberos. 91 00:04:03,440 --> 00:04:06,560 But now we have Azure Active Directory as an ability. 92 00:04:06,560 --> 00:04:07,760 That's been around for a while, 93 00:04:07,760 --> 00:04:12,240 but now you can actually require Azure SQL DB only use 94 00:04:12,240 --> 00:04:14,560 Azure Active Directory, 95 00:04:14,560 --> 00:04:18,000 which is really great to see because now you've got 96 00:04:18,000 --> 00:04:20,040 a Cloud-native solution, 97 00:04:20,040 --> 00:04:22,440 you've now got a Cloud-native authentication, 98 00:04:22,440 --> 00:04:26,240 and backing that as an authorization scheme as well. 99 00:04:26,240 --> 00:04:28,640 Also available, I don't pretend to be an expert here, 100 00:04:28,640 --> 00:04:30,080 I wish Sarah was here, 101 00:04:30,080 --> 00:04:34,000 but AKS support for secret store CSI driver. 102 00:04:34,000 --> 00:04:37,000 So we're announcing the general availability within 103 00:04:37,000 --> 00:04:40,120 AKS, the Azure Kubernetes Service support for 104 00:04:40,120 --> 00:04:43,000 the secret store container storage interfaces, 105 00:04:43,000 --> 00:04:44,960 that CSI driver. 106 00:04:44,960 --> 00:04:49,760 Storing secrets has always been a huge Achilles heel for 107 00:04:49,760 --> 00:04:51,080 all applications. 108 00:04:51,080 --> 00:04:54,360 So it's great to see that there is a standardized way, 109 00:04:54,360 --> 00:04:56,520 that's now generally available for 110 00:04:56,520 --> 00:04:59,360 support on Azure Kubernetes Service. 111 00:04:59,360 --> 00:05:01,320 There's another one that piqued my interest as well, 112 00:05:01,320 --> 00:05:03,320 but soon as we have Dave here, 113 00:05:03,320 --> 00:05:04,480 when we get around to him, 114 00:05:04,480 --> 00:05:07,200 I think he's probably better to talk about it than me. 115 00:05:07,200 --> 00:05:09,400 But we now have in general availability, 116 00:05:09,400 --> 00:05:13,160 Log Analytics Workspace Insights in Azure Monitor. 117 00:05:13,160 --> 00:05:16,920 The cynic in me thinks that sounds like Azure Monitor for 118 00:05:16,920 --> 00:05:18,680 Azure Monitor, but I don't know. 119 00:05:18,680 --> 00:05:20,520 So when we get around to Dave, 120 00:05:20,520 --> 00:05:24,280 I'll ask him to shed some light on that particular item. 121 00:05:24,280 --> 00:05:25,800 We've also published, 122 00:05:25,800 --> 00:05:29,960 Mark Rosinovic published a blog post on some of 123 00:05:29,960 --> 00:05:33,960 the key foundations for Azure confidential computing. 124 00:05:33,960 --> 00:05:37,680 We talked about these a little bit last time, 125 00:05:37,680 --> 00:05:40,480 with some of the announcements around 126 00:05:40,480 --> 00:05:44,680 specific VM types that support things like SGX, 127 00:05:44,680 --> 00:05:46,120 the Software Guard Extensions, 128 00:05:46,120 --> 00:05:49,480 that are one of the linchpins for confidential computing. 129 00:05:49,480 --> 00:05:51,600 Confidential computing is ultimately 130 00:05:51,600 --> 00:05:54,040 encryption of data while it's in use. 131 00:05:54,040 --> 00:05:55,400 I mean, there's more to it than that, 132 00:05:55,400 --> 00:05:57,240 but that's one of the most important parts. 133 00:05:57,240 --> 00:06:00,520 So I would definitely recommend that anyone who's 134 00:06:00,520 --> 00:06:01,760 interested in that topic, 135 00:06:01,760 --> 00:06:03,440 go and read this blog post. 136 00:06:03,440 --> 00:06:06,960 Another item is in Azure Logic Apps. 137 00:06:06,960 --> 00:06:09,760 The next item is in Azure Logic Apps. 138 00:06:09,760 --> 00:06:11,520 Azure Logic Apps now supports 139 00:06:11,520 --> 00:06:13,280 a managed identity for providing 140 00:06:13,280 --> 00:06:16,560 applications that support Azure Active Directory. 141 00:06:16,560 --> 00:06:19,480 So for example, if you've got, 142 00:06:19,480 --> 00:06:24,240 say, Azure SQL DB or Blob Storage, 143 00:06:24,240 --> 00:06:26,960 you can now use that managed identity in 144 00:06:26,960 --> 00:06:31,120 a Logic App to both authenticate and to 145 00:06:31,120 --> 00:06:34,360 provide authorization mechanisms to that particular resource. 146 00:06:34,360 --> 00:06:35,720 I've touched on this many times, 147 00:06:35,720 --> 00:06:36,960 but I'm going to do it again because I think this is 148 00:06:36,960 --> 00:06:38,240 critically important. 149 00:06:38,240 --> 00:06:44,000 We're seeing three massive areas of improvement across Azure. 150 00:06:44,000 --> 00:06:45,520 Different products were different stages, 151 00:06:45,520 --> 00:06:47,440 but we're well along the way. 152 00:06:47,440 --> 00:06:50,480 That is, customer managed key support for 153 00:06:50,480 --> 00:06:53,040 encryption of data at rest depending on whatever the services. 154 00:06:53,040 --> 00:06:56,400 The vast majority of services support that now. 155 00:06:56,400 --> 00:06:58,360 Another one is the use of 156 00:06:58,360 --> 00:07:01,080 for pass services is using private endpoints. 157 00:07:01,080 --> 00:07:03,440 Then the third one is making sure 158 00:07:03,440 --> 00:07:06,280 the services that connects as a client to 159 00:07:06,280 --> 00:07:09,040 another service have support for managed identities. 160 00:07:09,040 --> 00:07:11,160 The nice thing about managed identities is that 161 00:07:11,160 --> 00:07:13,080 the credential is not managed by you, 162 00:07:13,080 --> 00:07:14,440 it's managed by Azure. 163 00:07:14,440 --> 00:07:16,320 So when the process starts, 164 00:07:16,320 --> 00:07:20,200 the credential is automatically used by Azure. 165 00:07:20,200 --> 00:07:21,560 So you don't have to worry about 166 00:07:21,560 --> 00:07:23,800 storing and protecting credentials. 167 00:07:23,800 --> 00:07:27,280 I'm a huge fan of managed identities. 168 00:07:27,280 --> 00:07:29,640 It helps remove one of the biggest stumbling blocks is, 169 00:07:29,640 --> 00:07:31,320 okay, so where do you store the credential and 170 00:07:31,320 --> 00:07:32,880 how do you protect that credential? 171 00:07:32,880 --> 00:07:34,800 That is a non-issue when it 172 00:07:34,800 --> 00:07:36,520 comes to managed identities. 173 00:07:36,520 --> 00:07:38,880 The other item I want to bring up, 174 00:07:38,880 --> 00:07:41,640 I don't even know how to broach this one. 175 00:07:41,640 --> 00:07:43,400 So earlier in November, 176 00:07:43,400 --> 00:07:47,520 there was an issue where some Windows 11 features 177 00:07:47,520 --> 00:07:49,720 failed to run correctly, they wouldn't work, 178 00:07:49,720 --> 00:07:50,840 they wouldn't load. 179 00:07:50,840 --> 00:07:54,200 The reason was because a certificate had expired. 180 00:07:54,200 --> 00:07:55,480 So as you're probably aware, 181 00:07:55,480 --> 00:07:58,320 I think just about everything in Windows is digitally signed. 182 00:07:58,320 --> 00:08:00,120 When we start the process up, 183 00:08:00,120 --> 00:08:01,440 we got to check the signature and do 184 00:08:01,440 --> 00:08:03,440 all the usual X.509 stuff. 185 00:08:03,440 --> 00:08:07,000 Well, if the certificate has expired, what do you do? 186 00:08:07,000 --> 00:08:09,120 Well, the correct thing to do is to 187 00:08:09,120 --> 00:08:10,600 fail the operation altogether and 188 00:08:10,600 --> 00:08:13,000 that's exactly what happened here. 189 00:08:13,000 --> 00:08:17,160 This is an ongoing issue with X.509 this and 190 00:08:17,160 --> 00:08:20,480 X.509 that is management of certificates. 191 00:08:20,480 --> 00:08:22,760 So the real lesson here that came out of this 192 00:08:22,760 --> 00:08:25,120 is as an industry as a whole, 193 00:08:25,120 --> 00:08:27,840 this is still an area that we've got to really focus on. 194 00:08:27,840 --> 00:08:29,680 If you're managing certificates, 195 00:08:29,680 --> 00:08:31,600 then you need to concern yourself with 196 00:08:31,600 --> 00:08:34,200 the correct life cycle of those certificates as well. 197 00:08:34,200 --> 00:08:35,040 Because- 198 00:08:35,040 --> 00:08:36,720 In case you need a reminder that 199 00:08:36,720 --> 00:08:39,840 it's people process and technology altogether. 200 00:08:39,840 --> 00:08:43,680 Yeah, I was wondering when you would pipe up about that. 201 00:08:43,680 --> 00:08:44,960 You're absolutely right. 202 00:08:44,960 --> 00:08:46,760 I mean, at the end of the day, 203 00:08:46,760 --> 00:08:49,200 X.509 certificates are fantastic. 204 00:08:49,200 --> 00:08:50,400 I love the technology. 205 00:08:50,400 --> 00:08:52,640 I might be the only person in the world who does, 206 00:08:52,640 --> 00:08:53,960 but I really do. 207 00:08:53,960 --> 00:08:56,760 But man, when certificates aren't managed properly, 208 00:08:56,760 --> 00:09:00,400 things like this happen and it's not uncommon. 209 00:09:00,400 --> 00:09:04,600 All right. So that brings to an end the news. 210 00:09:04,600 --> 00:09:07,600 With that, let's turn our attention to our guest. 211 00:09:07,600 --> 00:09:09,960 This week, we have Dave Lubash, 212 00:09:09,960 --> 00:09:12,720 who's here to talk to us about Azure Monitor. 213 00:09:12,720 --> 00:09:14,800 Dave, why don't you spend a moment and just 214 00:09:14,800 --> 00:09:17,080 introduce yourself to our listeners? 215 00:09:17,080 --> 00:09:23,440 Hi, I'm David Lubash and I have worked at Microsoft for 23 years. 216 00:09:23,440 --> 00:09:27,240 Most of that time in the compliance and 217 00:09:27,240 --> 00:09:32,520 security space for shipping products the last seven years in 218 00:09:32,520 --> 00:09:35,720 Azure starting in with application insights. 219 00:09:35,720 --> 00:09:41,360 You had a mention about the Log Analytics Workspace Insights 220 00:09:41,360 --> 00:09:43,200 and how that's releasing. 221 00:09:43,200 --> 00:09:44,640 That is very meta. 222 00:09:44,640 --> 00:09:46,000 You were right, Mike, 223 00:09:46,000 --> 00:09:50,080 where you said that is insights about logs. 224 00:09:50,080 --> 00:09:52,480 What we've done is we've gone and made 225 00:09:52,480 --> 00:09:54,800 curated experiences around 226 00:09:54,800 --> 00:09:57,240 many different resources in Azure, 227 00:09:57,240 --> 00:09:59,560 many of the ones you were just talking about, 228 00:09:59,560 --> 00:10:03,440 Azure SQL DB, containers, Key Vault. 229 00:10:03,440 --> 00:10:07,840 We work with the owning teams and we build 230 00:10:07,840 --> 00:10:10,320 these rich curated experiences 231 00:10:10,320 --> 00:10:12,440 across both the data that they can give us 232 00:10:12,440 --> 00:10:16,120 and then both the views that you can get on top of that. 233 00:10:16,120 --> 00:10:19,240 This most recent one with Log Analytics is actually 234 00:10:19,240 --> 00:10:23,680 the insights you get on your own logging experience 235 00:10:23,680 --> 00:10:25,960 across all of Azure Monitor. 236 00:10:25,960 --> 00:10:30,640 Yeah, I'm glad we had you all lined up to discuss that. 237 00:10:30,640 --> 00:10:31,960 Again, I was like, man, 238 00:10:31,960 --> 00:10:35,040 this looks like a layer on top of Azure Monitor 239 00:10:35,040 --> 00:10:36,800 to monitor or something. 240 00:10:36,800 --> 00:10:41,120 I don't know. Anyway, I'm glad I wasn't too completely off the mark. 241 00:10:41,120 --> 00:10:43,640 The topic, I mean, Azure Monitor, right? 242 00:10:43,640 --> 00:10:47,640 So it's always been a point of confusion. 243 00:10:47,640 --> 00:10:49,680 I'm going to be tooting on this with you with customers. 244 00:10:49,680 --> 00:10:51,080 You'll see customers say, 245 00:10:51,080 --> 00:10:53,720 oh, you know, Log Analytics and then you'll hear say, 246 00:10:53,720 --> 00:10:55,640 people say, well, as your monitor. 247 00:10:55,640 --> 00:10:58,240 So what have we got going on here? 248 00:10:58,240 --> 00:11:02,560 I mean, is Azure Monitor something that sits on top of Log Analytics? 249 00:11:02,560 --> 00:11:03,760 What is it? 250 00:11:03,760 --> 00:11:08,680 So Azure Monitor, to really a large extent, 251 00:11:08,680 --> 00:11:10,400 maybe about three or four years ago, 252 00:11:10,400 --> 00:11:14,160 we created that to try to help reduce the confusion. 253 00:11:14,160 --> 00:11:17,760 So first off, we had many ways to monitor, right? 254 00:11:17,760 --> 00:11:21,120 We had the old OMS system center. 255 00:11:21,120 --> 00:11:24,480 We have Log Analytics came out of that. 256 00:11:24,480 --> 00:11:26,360 We had the application insights, 257 00:11:26,360 --> 00:11:32,080 which is really your curated experience on top of monitoring your applications. 258 00:11:32,080 --> 00:11:37,120 We had all of these external facing different products. 259 00:11:37,120 --> 00:11:42,200 First, we wanted to bring them all together under the Azure Monitor brand, 260 00:11:42,200 --> 00:11:45,640 but we also brought the teams together internally. 261 00:11:45,640 --> 00:11:50,000 So that first part was we brought these teams together so that we could look and we could go, 262 00:11:50,000 --> 00:11:55,760 wow, we have three different experiences built on top of logs. 263 00:11:55,760 --> 00:12:00,240 Let's make one better experience on top of logging, 264 00:12:00,240 --> 00:12:05,360 and similarly around alerting and the UX on top of this, 265 00:12:05,360 --> 00:12:07,240 UX on top of metrics. 266 00:12:07,240 --> 00:12:12,480 We took these teams across 267 00:12:12,480 --> 00:12:15,800 multiple geographies and multiple teams, 268 00:12:15,800 --> 00:12:22,000 multiple areas, brought them together in an attempt to make one better experience 269 00:12:22,000 --> 00:12:24,440 and also to reduce some of that confusion. 270 00:12:24,440 --> 00:12:27,400 So Azure Monitor is the collection of 271 00:12:27,400 --> 00:12:35,160 capabilities that enables us to have a better unified experience on top of logs, 272 00:12:35,160 --> 00:12:38,920 metrics, alerting going forward. 273 00:12:38,920 --> 00:12:42,200 I mean, I want to make sure I completely understand this. 274 00:12:42,200 --> 00:12:43,120 I'm not going to be wrong. 275 00:12:43,120 --> 00:12:44,880 I mean, I use Azure Monitor every single day, 276 00:12:44,880 --> 00:12:49,880 but I don't pretend to know all the moving parts nor the machinations underneath it. 277 00:12:49,880 --> 00:12:53,440 It's just like magic, stuff just happens. 278 00:12:53,440 --> 00:12:56,080 But at the end of the day, 279 00:12:56,080 --> 00:13:00,040 is it fair to say that Azure Monitor is a layer, 280 00:13:00,040 --> 00:13:02,960 like is a way of viewing the contents of log analytics. 281 00:13:02,960 --> 00:13:08,600 So different products can feed into log analytics and then Azure Monitor can be used to 282 00:13:08,600 --> 00:13:14,840 extract intelligence out of those logs as well as perhaps even raising alerts. 283 00:13:14,840 --> 00:13:18,600 That might be fair from one point of view. 284 00:13:18,600 --> 00:13:23,920 The other view is when we created and started combining 285 00:13:23,920 --> 00:13:26,520 all of these capabilities under Azure Monitor, 286 00:13:26,520 --> 00:13:32,400 we knew we couldn't just go delete the experience that people think of as log analytics, 287 00:13:32,400 --> 00:13:36,480 and the experience people think of as application insights, 288 00:13:36,480 --> 00:13:39,520 because these had many customers already. 289 00:13:39,520 --> 00:13:45,480 So we've built on top of log analytics as part of Azure Monitor. 290 00:13:45,480 --> 00:13:50,080 So instead of thinking that Azure Monitor allows you to view your logs, 291 00:13:50,080 --> 00:13:56,000 you should think of that log analytics is just the logging capability 292 00:13:56,000 --> 00:13:58,560 that is now built into Azure Monitor. 293 00:13:58,560 --> 00:14:02,000 It combines the logging capability that was already built into 294 00:14:02,000 --> 00:14:06,560 application insights and some of the other products. 295 00:14:06,560 --> 00:14:09,240 Right. So you brought up application insights and so on. 296 00:14:09,240 --> 00:14:18,120 So is there a general guidance around where app insights is used versus say? 297 00:14:18,120 --> 00:14:20,360 Sure. That's a great question. 298 00:14:20,360 --> 00:14:23,240 As a security center, those are the other tools like that. 299 00:14:23,240 --> 00:14:30,160 So when you're sitting on top of a resource that creates logs, 300 00:14:30,160 --> 00:14:37,680 VMs, compute, Windows, laptops, 301 00:14:37,680 --> 00:14:42,560 these types of resources already create logs. 302 00:14:42,560 --> 00:14:53,800 You can also then improve those logging experiences by both filtering on the device itself. 303 00:14:53,800 --> 00:14:59,040 So you reduce the number of logs coming in to the right level. 304 00:14:59,040 --> 00:15:03,320 But when you're looking at a resource that already creates logs, 305 00:15:03,320 --> 00:15:05,160 that's from what we would consider, 306 00:15:05,160 --> 00:15:08,520 oh, okay, let's we put the agent on there, 307 00:15:08,520 --> 00:15:10,800 either the new agent, 308 00:15:10,800 --> 00:15:14,400 the one of the older OMS log analytics agents, 309 00:15:14,400 --> 00:15:22,280 the agent sits in that resource and then routes logs to log analytics. 310 00:15:22,280 --> 00:15:25,560 Then, but there are also types of resources, 311 00:15:25,560 --> 00:15:31,640 including applications that you also want to collect data from. 312 00:15:31,640 --> 00:15:36,000 So in the context of when application insights is used, 313 00:15:36,000 --> 00:15:42,600 is you're coming to your application and you're using an SDK to instrument 314 00:15:42,600 --> 00:15:48,320 the SDK to send data to what used to be an application insights endpoint. 315 00:15:48,320 --> 00:15:52,800 Yet we're looking as we converge this platform, 316 00:15:52,800 --> 00:15:57,760 it's an Azure Monitor endpoint, your log show weapon in a very similar, 317 00:15:57,760 --> 00:16:02,440 they're in the same set of workspaces so that you can then look and say, 318 00:16:02,440 --> 00:16:08,280 oh, I have this workspace set here and it has my application logs, 319 00:16:08,280 --> 00:16:12,280 my resource logs, all of that in one place. 320 00:16:12,280 --> 00:16:16,720 Then you have your curated views on top of those logs. 321 00:16:16,720 --> 00:16:18,800 Now you mentioned Sentinel, 322 00:16:18,800 --> 00:16:25,080 and some of the other Microsoft Cloud security products as they're going through 323 00:16:25,080 --> 00:16:27,960 some of their own convergence and branding. 324 00:16:27,960 --> 00:16:38,760 Those products typically sit with their extension to the log analytics agent on the resource. 325 00:16:38,760 --> 00:16:47,560 So what you're getting is the Sentinel team has done the work to try to keep the logs 326 00:16:47,560 --> 00:16:49,280 logging at the right level. 327 00:16:49,280 --> 00:16:57,040 So you're both getting the right number of logs because it's really easy to turn on 328 00:16:57,040 --> 00:17:03,160 and make it so that you're collecting more telemetry or you're spending more money on 329 00:17:03,160 --> 00:17:09,080 telemetry than you would ever spend on the actual compute resource itself. 330 00:17:09,080 --> 00:17:14,360 Yes. I mean, I guess the way I'm thinking about this and tell me if I'm on the right track here is that, 331 00:17:14,360 --> 00:17:17,000 whenever you look at an architecture diagram, 332 00:17:17,000 --> 00:17:21,400 you have the foundational stuff at the bottom and I feel like Azure Monitor is kind of that. 333 00:17:21,400 --> 00:17:23,720 It's like the trucking company that gets it there, 334 00:17:23,720 --> 00:17:27,320 not necessarily all the analytics and what's being shipped and whatnot. 335 00:17:27,320 --> 00:17:32,600 It's just make sure that the package gets there and the logs get out to all the right locations. 336 00:17:32,600 --> 00:17:36,440 In both the agent and the cloud service piece, 337 00:17:36,440 --> 00:17:42,560 and it sounds like the log analytics and then the Sentinel and the app insights 338 00:17:42,560 --> 00:17:47,960 essentially then stack on top of that and create that value added insights and, 339 00:17:47,960 --> 00:17:50,160 hey, your application is running slow or hey, 340 00:17:50,160 --> 00:17:52,280 you have a security attack going on. 341 00:17:52,280 --> 00:17:54,840 I mean, it feels like that's the configuration. 342 00:17:54,840 --> 00:17:56,280 Is that the right way to think about it? 343 00:17:56,280 --> 00:17:58,680 I think so, but it two layers. 344 00:17:58,680 --> 00:18:02,560 I mean, using the package distribution, 345 00:18:02,560 --> 00:18:10,160 maybe it's more the distribution center because on the one layer at the bottom, 346 00:18:10,160 --> 00:18:14,480 you very frequently at the lower level at the resource level, 347 00:18:14,480 --> 00:18:19,160 the Azure Monitor has worked with the resource teams, 348 00:18:19,160 --> 00:18:23,400 whether that's Sentinel, Azure SQL DB, 349 00:18:23,400 --> 00:18:26,960 Key Vault in order to collect the right data. 350 00:18:26,960 --> 00:18:30,240 That data then gets routed to what you would, 351 00:18:30,240 --> 00:18:33,440 your analogy would be like the distribution center. 352 00:18:33,440 --> 00:18:34,960 Then on top of that, 353 00:18:34,960 --> 00:18:41,320 the views get built and those then tend to frequently come back to the teams that 354 00:18:41,320 --> 00:18:43,680 help provide that data in the first place. 355 00:18:43,680 --> 00:18:48,600 What you would see from Sentinel is both at the lower level, 356 00:18:48,600 --> 00:18:52,080 helping collect the right data and then at the upper level, 357 00:18:52,080 --> 00:18:55,000 ensuring that the right data is displayed, 358 00:18:55,000 --> 00:18:56,560 including of course, 359 00:18:56,560 --> 00:19:00,040 the right alerts are built in on top of that data. 360 00:19:00,040 --> 00:19:01,960 So you're just not alerting there. 361 00:19:01,960 --> 00:19:04,360 How do we do alerting in Azure Monitor? 362 00:19:04,360 --> 00:19:07,400 What sort of things can I A, alert on and B, 363 00:19:07,400 --> 00:19:09,080 how do I get notified? 364 00:19:09,080 --> 00:19:11,920 Sure. So to start with, 365 00:19:11,920 --> 00:19:16,040 when you look at the data that we've ingested into Azure Monitor, 366 00:19:16,040 --> 00:19:19,400 it's really all comes down to two types, 367 00:19:19,400 --> 00:19:22,840 it's logs and its metrics. 368 00:19:22,840 --> 00:19:26,160 The logs are essentially the raw logs that have been 369 00:19:26,160 --> 00:19:32,800 indexed from any of the types of both log analytics and insights and gesting pieces. 370 00:19:32,800 --> 00:19:37,800 Then we also on top of that have the capability to 371 00:19:37,800 --> 00:19:42,480 create these curated metrics on top of this. 372 00:19:42,480 --> 00:19:44,760 What these do is in some cases, 373 00:19:44,760 --> 00:19:49,600 they're just think of that as a part of the log itself, 374 00:19:49,600 --> 00:19:57,440 but maybe with a bit of more of a time series pre-aggregation on top of it. 375 00:19:57,440 --> 00:20:02,840 So now for alerting, you can build alerting on top of metrics, 376 00:20:02,840 --> 00:20:06,200 on top of the logs themselves. 377 00:20:06,200 --> 00:20:14,280 Then again, going back to that convergence story that we've been going through these last four or five years, 378 00:20:14,280 --> 00:20:17,280 it was three different, 379 00:20:17,280 --> 00:20:24,520 four different teams at Microsoft who were building alert capabilities on top of different types of data. 380 00:20:24,520 --> 00:20:28,360 Now we really have one larger, 381 00:20:28,360 --> 00:20:35,320 more focused team that's job it is to build alerting on top of the entire platform. 382 00:20:35,320 --> 00:20:38,480 So you mentioned before about pre-aggregation and that sort of stuff. 383 00:20:38,480 --> 00:20:39,320 So make sure I get this right. 384 00:20:39,320 --> 00:20:40,640 So if I go to Azure Monitor, 385 00:20:40,640 --> 00:20:43,800 I've noticed that on the pane on the left-hand side there, 386 00:20:43,800 --> 00:20:47,080 there are some products and I think they call them like insights or something. 387 00:20:47,080 --> 00:20:48,880 I don't like Key Vault stuff. 388 00:20:48,880 --> 00:20:50,720 So I can click on that Key Vault option, 389 00:20:50,720 --> 00:20:53,640 it will give me all this really nicely laid out, 390 00:20:53,640 --> 00:20:59,720 really actionable charts and telemetry like showing access failures to Key Vault, 391 00:20:59,720 --> 00:21:01,400 the number of puts, the number of gets, 392 00:21:01,400 --> 00:21:05,000 the number of key creation and so on and so forth. 393 00:21:05,000 --> 00:21:10,320 So is that where you guys have worked with Key Vault, for example, 394 00:21:10,320 --> 00:21:18,680 to say, okay, we can display your critical information as part of Azure Monitor in one pane, 395 00:21:18,680 --> 00:21:23,040 as opposed to having to go into Key Vault necessarily and start messing around inside of Key Vault. 396 00:21:23,040 --> 00:21:25,160 You can sort of display it inside of Azure Monitor. 397 00:21:25,160 --> 00:21:26,640 The short answer is yes. 398 00:21:26,640 --> 00:21:28,800 Then of course, additionally, 399 00:21:28,800 --> 00:21:31,440 whether it's Key Vault or storage, 400 00:21:31,440 --> 00:21:35,160 we've also tend to make a more, 401 00:21:35,160 --> 00:21:42,960 I would say, maybe a more combined or comprehensive story because instead of looking at 402 00:21:42,960 --> 00:21:46,360 my one storage account or my one Key Vault, 403 00:21:46,360 --> 00:21:51,720 I'm looking at a collection of storage accounts, a collection of, 404 00:21:51,720 --> 00:21:58,520 so I'm looking across a broader spectrum so I can get an idea of how all of these resources are behaving, 405 00:21:58,520 --> 00:22:03,560 whether it's across a geo worldwide. 406 00:22:03,560 --> 00:22:13,160 So it just gives that better comprehensive view across the entire set of resources you're operating in your tenant. 407 00:22:13,160 --> 00:22:15,480 I love that view because I use Key Vault all the time. 408 00:22:15,480 --> 00:22:21,040 So having that one little view with absolutely no effort required for me looking across all my Key Vaults, 409 00:22:21,040 --> 00:22:23,160 and it's like a hierarchy as well. 410 00:22:23,160 --> 00:22:27,920 So I can see like an aggregation of the data or I can see drill down into an individual Key Vault. 411 00:22:27,920 --> 00:22:32,040 If I see that there's one being problematic, it's actually really, really nice. 412 00:22:32,040 --> 00:22:37,440 So my guess is what you're doing is just taking some really low level telemetry from the Key Vaults, 413 00:22:37,440 --> 00:22:43,240 and then aggregating it and then making it look pretty so that I can actually understand what the heck is going on. 414 00:22:43,240 --> 00:22:47,720 Yeah. That's what we've been working through these last, 415 00:22:47,720 --> 00:22:54,880 I would say about two years as we started this insights journey across almost, 416 00:22:54,880 --> 00:23:00,480 I mean, long term, of course, let's do it for all of the resources in Azure, 417 00:23:00,480 --> 00:23:06,560 but starting with the ones that customers are most interested in first, 418 00:23:06,560 --> 00:23:13,680 things like containers and VMs and Key Vault and Azure SQL. 419 00:23:13,680 --> 00:23:18,560 One of the things that I always worry about and I don't know, maybe I'm too paranoid as a security guy, 420 00:23:18,560 --> 00:23:25,720 is what happens if sensitive data like PII or PHI, probably more PII than that, 421 00:23:25,720 --> 00:23:32,040 but sensitive stuff, even passwords, gets logged into the system accidentally. 422 00:23:32,040 --> 00:23:36,840 How would someone handle cleaning that up and finding them? 423 00:23:36,840 --> 00:23:44,840 So first of all, if you did find something in a log that you didn't expect to be there or was accidentally logged, 424 00:23:44,840 --> 00:23:53,040 and this does happen, the view on that is, of course, you should think of it as not necessarily exposed, 425 00:23:53,040 --> 00:23:58,080 but it is something that does need to be rotated because you have put it probably into a system 426 00:23:58,080 --> 00:24:04,480 that you haven't maybe put the same RBAC controls on top of as production, 427 00:24:04,480 --> 00:24:11,000 VMs or other resources, and yet at the same time, in those other resources, 428 00:24:11,000 --> 00:24:17,240 if someone did have access, they may not even be able to see something like a plain text password. 429 00:24:17,240 --> 00:24:24,400 So in the accidentally logging scenario, we can delete individual rows. 430 00:24:24,400 --> 00:24:30,080 We have purge functionality so that customers can come through and say, 431 00:24:30,080 --> 00:24:34,720 oh, this data is not supposed to be in this log. 432 00:24:34,720 --> 00:24:37,040 I need, of course, to do the three steps. 433 00:24:37,040 --> 00:24:44,200 One, stop sending it because you need to stop that bleeding. 434 00:24:44,200 --> 00:24:48,480 And then second, go through and maybe look at, oh, do I need to rotate these 435 00:24:48,480 --> 00:24:54,160 or do I already have some rotation plans in place for these secrets? 436 00:24:54,160 --> 00:25:01,720 And then finally, executing the purge commands against the data that has shown up in these logs. 437 00:25:01,720 --> 00:25:04,200 Who's we? Is we Microsoft or is we somebody? 438 00:25:04,200 --> 00:25:07,240 I'm sorry. We, in this context, the customer. 439 00:25:07,240 --> 00:25:16,440 The customer themselves has full control over their data and can operate the purge API to delete this. 440 00:25:16,440 --> 00:25:25,080 I was just saying that, and also we Microsoft, because we've had this experience many times 441 00:25:25,080 --> 00:25:30,800 at Microsoft where teams come to us and say, we've found some secrets here. 442 00:25:30,800 --> 00:25:33,720 What do I need to do to clean it up? 443 00:25:33,720 --> 00:25:42,280 Hang on a minute. I'm going to take Mark's paranoia and raise it with a bit more paranoia. 444 00:25:42,280 --> 00:25:48,600 So Mark has a valid point, right? If we have some sensitive data being logged, 445 00:25:48,600 --> 00:25:51,680 how do we scrub it? How do we clean it up? 446 00:25:51,680 --> 00:25:55,920 However, how do we stop someone from covering their tracks? 447 00:25:55,920 --> 00:26:01,360 Because they realize that something bad happened and now they want to go and delete stuff from the logs. 448 00:26:01,360 --> 00:26:03,440 How do we prevent that? 449 00:26:03,440 --> 00:26:05,640 Or is that the question you were going to ask as well, Mark? 450 00:26:05,640 --> 00:26:07,480 Yeah, you stole my question. 451 00:26:07,480 --> 00:26:14,600 That's all you meant. We're both paranoia. So we're on the same track then. Okay. Good to have big paranoia. 452 00:26:14,600 --> 00:26:26,360 So every Azure resource has a logging story where they need to send the audit actions to the audit logs. 453 00:26:26,360 --> 00:26:31,440 Now, interestingly, the audit logs are of course handled by Azure Monitor directly. 454 00:26:31,440 --> 00:26:37,480 The view that customers are getting for audit logs are also part of 455 00:26:37,480 --> 00:26:42,480 that at least one of the Azure Monitor experiences across all other resources. 456 00:26:42,480 --> 00:26:49,480 I'm worried that somebody is going to go and delete part of this logging. 457 00:26:49,480 --> 00:26:51,720 Well, there's a couple of things here. 458 00:26:51,720 --> 00:27:00,480 One, the RBAC controls on the purge command are more restricted, 459 00:27:00,480 --> 00:27:02,720 so that you do have to have purge. 460 00:27:02,720 --> 00:27:08,720 I forget the exact name of the control itself, 461 00:27:08,720 --> 00:27:14,720 but basically you need to be on the permissions to operate a purge command. 462 00:27:14,720 --> 00:27:17,080 Once you have those permissions, 463 00:27:17,080 --> 00:27:18,360 when you do operate that, 464 00:27:18,360 --> 00:27:26,200 that is of course stored in the same audit log because you executed this purge. 465 00:27:26,200 --> 00:27:28,800 That purge is stored. 466 00:27:28,800 --> 00:27:31,520 Additionally then because the purge is stored, 467 00:27:31,520 --> 00:27:35,080 that's stored in an immutable storage so that, 468 00:27:35,080 --> 00:27:37,680 for example, our own purge controls 469 00:27:37,680 --> 00:27:40,040 went in to allow you to purge that data. 470 00:27:40,040 --> 00:27:42,120 It's immutable at that point. 471 00:27:42,120 --> 00:27:46,200 Ultimately, it all boils down to so many other things. 472 00:27:46,200 --> 00:27:48,080 As any cloud system for that matter, 473 00:27:48,080 --> 00:27:50,480 it all boils down to the RBAC controls that are on 474 00:27:50,480 --> 00:27:52,600 the specific resource being protected. 475 00:27:52,600 --> 00:27:53,920 It makes sense. 476 00:27:53,920 --> 00:27:56,240 She's interesting. 477 00:27:56,240 --> 00:28:01,680 If I delete a row from a log, 478 00:28:01,680 --> 00:28:05,080 there is an entry written to another log that says, 479 00:28:05,080 --> 00:28:07,400 Michael deleted this row from the log. 480 00:28:07,400 --> 00:28:08,640 Yes. 481 00:28:08,640 --> 00:28:12,360 All right. It's a little bit like the Windows Event log. 482 00:28:12,360 --> 00:28:14,760 If you clear the security log, 483 00:28:14,760 --> 00:28:18,440 there's an event written straight away which is Michael cleared the security log. 484 00:28:18,440 --> 00:28:20,120 If you try and delete that, 485 00:28:20,120 --> 00:28:25,320 there's another entry written that says Michael tried to enter the security log. 486 00:28:25,320 --> 00:28:26,960 There's always that record there. 487 00:28:26,960 --> 00:28:28,240 Even if something is deleted, 488 00:28:28,240 --> 00:28:30,600 there's a record that shows that it's been deleted regardless, 489 00:28:30,600 --> 00:28:33,640 which is absolutely critically important, I think. 490 00:28:33,640 --> 00:28:39,920 But where does Azure Data Explorer fit in all this Azure monitor stuff? 491 00:28:39,920 --> 00:28:44,480 Going back years on how we built these systems, 492 00:28:44,480 --> 00:28:49,360 the Azure Data Explorer and the Azure Monitor teams, 493 00:28:49,360 --> 00:28:54,800 we started down this journey together. 494 00:28:54,800 --> 00:29:02,640 They built a fantastic platform for handling large amounts of data, 495 00:29:02,640 --> 00:29:06,360 and building really great queries on top of that. 496 00:29:06,360 --> 00:29:10,760 That became our back-end for logs. 497 00:29:10,760 --> 00:29:17,800 The back-end that we use internally is built on top of Azure Data Explorer. 498 00:29:17,800 --> 00:29:21,400 That was first used just for application insights, 499 00:29:21,400 --> 00:29:25,480 and then as we converge the next round of platforms, 500 00:29:25,480 --> 00:29:28,480 then was started to use by Log Analytics. 501 00:29:28,480 --> 00:29:35,880 The way it fits in is you're operating the log capabilities from Azure Monitor, 502 00:29:35,880 --> 00:29:40,800 is we're operating that entirely on Azure Data Explorer. 503 00:29:40,800 --> 00:29:49,080 It's the same KQL language with maybe a couple of commands that are restricted, 504 00:29:49,080 --> 00:29:54,000 because it is this multi-tenant platform. 505 00:29:54,000 --> 00:29:59,120 But that's how and why the queries are as fast as they are, 506 00:29:59,120 --> 00:30:06,160 and how you can go ahead and bypass the curated experience 507 00:30:06,160 --> 00:30:09,800 and go directly to your own query experience. 508 00:30:09,800 --> 00:30:12,880 You brought up an interesting word there, which was multi-tenant. 509 00:30:12,880 --> 00:30:16,520 Is there a solution here for customers who, 510 00:30:16,520 --> 00:30:21,720 for whatever purposes, require the use of a single tenant logging environment? 511 00:30:21,720 --> 00:30:29,560 Sure. In addition to storing the data in the multi-tenant Azure Data Explorer, 512 00:30:29,560 --> 00:30:35,040 customers can come in and in the context really of from logs only, 513 00:30:35,040 --> 00:30:40,880 they create their own Azure Data Explorer, 514 00:30:40,880 --> 00:30:45,240 and then we push their workspaces into that. 515 00:30:45,240 --> 00:30:48,080 Now, they do this on a per-region basis, 516 00:30:48,080 --> 00:30:54,000 and so they can have as many workspaces as they want in that one region, 517 00:30:54,000 --> 00:30:59,400 that one Azure Data Explorer that they've configured. 518 00:30:59,400 --> 00:31:02,520 What about with Log Analytics itself? 519 00:31:02,520 --> 00:31:06,720 I heard that there's a single-tenant version of Log Analytics as well. 520 00:31:06,720 --> 00:31:08,120 Yeah, that's sorry, 521 00:31:08,120 --> 00:31:11,560 and maybe I didn't quite explain that accurately. 522 00:31:11,560 --> 00:31:16,520 In the Log Analytics API, you would go through in that context 523 00:31:16,520 --> 00:31:20,040 and create the dedicated cluster. 524 00:31:20,040 --> 00:31:24,120 It is an ADX cluster, 525 00:31:24,120 --> 00:31:29,760 but it is, I guess, still paid for as part of Log Analytics, 526 00:31:29,760 --> 00:31:34,560 still managed for the most part directly through Log Analytics. 527 00:31:34,560 --> 00:31:37,000 Is that a special version of Log Analytics? 528 00:31:37,000 --> 00:31:40,600 Is that something you can opt in for when you create 529 00:31:40,600 --> 00:31:41,920 a new Log Analytics workspace? 530 00:31:41,920 --> 00:31:46,080 How does that manifest itself at a practical level? 531 00:31:46,080 --> 00:31:48,760 Yes. At a practical level, 532 00:31:48,760 --> 00:31:51,720 you could see it as for the most part, 533 00:31:51,720 --> 00:31:56,640 I'm creating a new workspace that I'm going to host on top of this platform. 534 00:31:56,640 --> 00:32:01,720 It's a little more nuanced in that a customer could have 535 00:32:01,720 --> 00:32:05,480 an existing workspace sitting in the multi-tenant platform, 536 00:32:05,480 --> 00:32:09,200 and after a while, they're looking at their data and going, 537 00:32:09,200 --> 00:32:11,080 I have a lot of this data. 538 00:32:11,080 --> 00:32:14,440 Some of it, I really want to, 539 00:32:14,440 --> 00:32:17,080 because maybe it's a little more sensitive, 540 00:32:17,080 --> 00:32:20,160 or maybe I want a little more control, 541 00:32:20,160 --> 00:32:24,280 and maybe even be able to manage the costs a little differently. 542 00:32:24,280 --> 00:32:30,560 They can create a new workspace or create the new dedicated cluster, 543 00:32:30,560 --> 00:32:33,440 associate the workspace with that. 544 00:32:33,440 --> 00:32:37,360 We don't typically move the data, 545 00:32:37,360 --> 00:32:43,400 because you might be leaving behind hundreds of petabytes of data. 546 00:32:43,400 --> 00:32:48,320 Instead, the new data would go to the new cluster, 547 00:32:48,320 --> 00:32:56,080 because these logs tend to have age-out times of 30 days, 90 days, two years. 548 00:32:56,080 --> 00:33:00,160 The data would sit in the old cluster as well, 549 00:33:00,160 --> 00:33:03,280 but that older data would then gradually age out, 550 00:33:03,280 --> 00:33:07,760 and of course, all new data is landing in the new space. 551 00:33:07,760 --> 00:33:10,960 Queries are merged on top of this, 552 00:33:10,960 --> 00:33:16,760 so the customer is then able to see data from both portions. 553 00:33:16,760 --> 00:33:18,000 One of the things, I mean, 554 00:33:18,000 --> 00:33:20,320 I've heard the term Azure Monitor Essentials. 555 00:33:20,320 --> 00:33:23,520 Can you explain what that means in the context of 556 00:33:23,520 --> 00:33:27,000 Azure Monitor all up and any premium options and whatnot? 557 00:33:27,000 --> 00:33:31,720 When we started monitoring around the company, 558 00:33:31,720 --> 00:33:36,360 around Microsoft and providing monitoring solutions for Microsoft Teams, 559 00:33:36,360 --> 00:33:37,720 and additionally, of course, 560 00:33:37,720 --> 00:33:42,520 building these premium offerings that we were for OMS, 561 00:33:42,520 --> 00:33:45,600 Log Analytics, Application Insights. 562 00:33:45,600 --> 00:33:48,720 One of our teams, 563 00:33:48,720 --> 00:33:55,160 the team that own this almost free experience built on top of Azure, 564 00:33:55,160 --> 00:33:58,680 that is required for operating your resources in Azure. 565 00:33:58,680 --> 00:34:03,960 These are your diagnostic logs and your audit logs experiences, 566 00:34:03,960 --> 00:34:09,320 auto-scale, some alerting on top of all of that. 567 00:34:09,320 --> 00:34:11,600 These pieces were being built by, 568 00:34:11,600 --> 00:34:15,960 of course, a slightly different team than the teams building for 569 00:34:15,960 --> 00:34:18,760 Log Analytics and Application Insights. 570 00:34:18,760 --> 00:34:27,360 They also didn't have the need to bill because these are the essential pieces. 571 00:34:27,360 --> 00:34:30,240 When we joined all these teams together, 572 00:34:30,240 --> 00:34:36,920 we said, let's still keep this separate in order to help people 573 00:34:36,920 --> 00:34:40,280 understand where we are on our journey. 574 00:34:40,280 --> 00:34:43,200 The Azure Monitor Essentials pieces are 575 00:34:43,200 --> 00:34:50,520 the free or mostly free components that are built on top of diagnostic logging, 576 00:34:50,520 --> 00:34:55,960 audit logging, and the data that's collected as part of that. 577 00:34:55,960 --> 00:34:57,800 Then the teams then, of course, 578 00:34:57,800 --> 00:35:01,040 as we've merged together and worked on these pieces, 579 00:35:01,040 --> 00:35:08,160 we've then combined hopefully and made our improvements for both UX, 580 00:35:08,160 --> 00:35:13,080 and both the scale of data that we're able to handle. 581 00:35:13,080 --> 00:35:16,760 That's really where Azure Monitor Essentials fits in. 582 00:35:16,760 --> 00:35:21,240 Then the premium offerings are still the ones that we'd also had before, 583 00:35:21,240 --> 00:35:24,160 with application insights and Log Analytics. 584 00:35:24,160 --> 00:35:28,200 These are the offerings where because, 585 00:35:28,200 --> 00:35:33,240 in many cases, significant capacity in Azure is used to operate them, 586 00:35:33,240 --> 00:35:38,360 we do need to pass along those costs to the customer. 587 00:35:38,360 --> 00:35:41,240 If I think about this as the monitoring essentials, 588 00:35:41,240 --> 00:35:46,720 these are core basic logging features that are just naturally part of the platform. 589 00:35:46,720 --> 00:35:50,560 Then any of the crank it up to 11 and 590 00:35:50,560 --> 00:35:54,320 huge storage requirements and whatnot, essentially, 591 00:35:54,320 --> 00:35:57,120 those are the more the premium paid ones. 592 00:35:57,120 --> 00:35:59,840 Yes. That's correct. 593 00:35:59,840 --> 00:36:07,200 One thing that always interests me is how large customers especially use Log Analytics. 594 00:36:07,200 --> 00:36:11,000 I mean, we're dealing with often petabytes of data, 595 00:36:11,000 --> 00:36:15,800 gigabytes if not more of ingestion on a regular basis. 596 00:36:15,800 --> 00:36:19,560 What are some of the things that some of the customers have told you about the platform, 597 00:36:19,560 --> 00:36:23,120 and some of the practices that they've been practicing themselves? 598 00:36:23,120 --> 00:36:28,320 We do have some best practices documents published as well now, 599 00:36:28,320 --> 00:36:34,360 but just to go through some of the challenges that we've had with some large customers, 600 00:36:34,360 --> 00:36:37,120 particularly when it is petabytes of data. 601 00:36:37,120 --> 00:36:44,040 When we first started integrating with some of the additional teams, 602 00:36:44,040 --> 00:36:45,320 including Sentinel, 603 00:36:45,320 --> 00:36:51,400 I would say a very common onboarding experience for a large customer was, 604 00:36:51,400 --> 00:36:57,040 I enabled Sentinel or the Shkurti tooling at the time. 605 00:36:57,040 --> 00:37:00,840 I enabled this, I turned everything on, 606 00:37:00,840 --> 00:37:04,760 and then, oh, wow, I got bill shock. 607 00:37:04,760 --> 00:37:10,560 I looked at this bill and it was more than I was spending on the compute being monitored. 608 00:37:10,560 --> 00:37:14,840 I think we've done a pretty good job of improving that. 609 00:37:14,840 --> 00:37:16,640 A lot of feedback from customers, 610 00:37:16,640 --> 00:37:20,320 and that's that initial onboarding experience, 611 00:37:20,320 --> 00:37:22,560 particularly with the larger customers. 612 00:37:22,560 --> 00:37:28,320 We did work with them to help them both manage their bill and ensure that we have 613 00:37:28,320 --> 00:37:33,880 our own internal learning now so that when we see a new customer coming in and 614 00:37:33,880 --> 00:37:35,960 incurring very high costs, 615 00:37:35,960 --> 00:37:39,000 we'll immediately send them emails saying, 616 00:37:39,000 --> 00:37:41,920 hey, was this expected? 617 00:37:41,920 --> 00:37:47,320 Some of the other challenges I think we've had with large customers over the years, 618 00:37:47,320 --> 00:37:52,880 particularly as we've merged these teams that support 619 00:37:52,880 --> 00:37:55,880 internal monitoring and external monitoring is, 620 00:37:55,880 --> 00:38:01,720 external customers have almost a different set of access rules 621 00:38:01,720 --> 00:38:04,400 than maybe what we would see at Microsoft. 622 00:38:04,400 --> 00:38:11,840 Really, it's been about understanding how these customers might do their deployments, 623 00:38:11,840 --> 00:38:15,120 and a customer would say a thousand engineers, 624 00:38:15,120 --> 00:38:23,080 you'll find out that they have only maybe 10 or 15 of them are allowed to 625 00:38:23,080 --> 00:38:26,720 touch production with deployments. 626 00:38:26,720 --> 00:38:31,280 These types of access to the portal, 627 00:38:31,280 --> 00:38:35,640 we built one experience that we're expecting, 628 00:38:35,640 --> 00:38:41,040 this is how you jit and this is how you get your access, 629 00:38:41,040 --> 00:38:42,960 and it's fantastic. 630 00:38:42,960 --> 00:38:47,960 Then we turn around and we find that a large customer would really be 631 00:38:47,960 --> 00:38:51,000 operating on Azure very differently. 632 00:38:51,000 --> 00:38:55,760 We did have to make changes both in what we've supported through our back controls, 633 00:38:55,760 --> 00:39:02,120 so that maybe you didn't see the same read-write access directly to the resource in 634 00:39:02,120 --> 00:39:07,360 order to still be able to go in and have access and controls on top of 635 00:39:07,360 --> 00:39:12,520 dashboards on top of maybe tweaking and alert. 636 00:39:12,520 --> 00:39:18,400 It's been an interesting journey as we built out Azure Monitor and as we 637 00:39:18,400 --> 00:39:20,720 onboard some of these larger customers. 638 00:39:20,720 --> 00:39:26,440 Another point you alluded to at the start of the broadcast on how do you deal with 639 00:39:26,440 --> 00:39:28,720 credentials and MSI, 640 00:39:28,720 --> 00:39:33,680 that's been part of this most recent round of maybe the last three, 641 00:39:33,680 --> 00:39:40,920 four or five months is how to make sure that we enable the same capabilities 642 00:39:40,920 --> 00:39:47,160 of monitoring using MSI rather than using maybe credentials and moving around 643 00:39:47,160 --> 00:39:54,240 certs because certain management is a problem both internally and externally. 644 00:39:54,240 --> 00:39:58,680 It really becomes a challenge as we look all up. 645 00:39:58,680 --> 00:40:01,960 During the week, you and I were talking about some of the improvements that have 646 00:40:01,960 --> 00:40:06,320 been made around security and app insights around AAD. 647 00:40:06,320 --> 00:40:07,920 What's going on there? 648 00:40:07,920 --> 00:40:12,320 I was looking for the actual document for this itself today. 649 00:40:12,320 --> 00:40:17,880 Application Insights has some new SDKs that are in preview. 650 00:40:17,880 --> 00:40:23,000 These SDKs should be GA-ing in the first half of 2022. 651 00:40:23,000 --> 00:40:30,280 They allow use of AAD authentication to be included as part of the telemetry. 652 00:40:30,280 --> 00:40:36,560 Instead of today where most telemetry endpoints for application insights are unauthenticated, 653 00:40:36,560 --> 00:40:40,000 we can start collecting strictly authenticated data. 654 00:40:40,000 --> 00:40:46,360 Now, that is only for of course a portion of the types of data and that there are certainly 655 00:40:46,360 --> 00:40:52,360 continued to be types of applications that the entire ingestion telemetry experience 656 00:40:52,360 --> 00:41:01,960 is intended to capture the moment from the external customer first hits a website. 657 00:41:01,960 --> 00:41:07,000 You have a shopping portal, customer hits this, and well before they're logged in, you want 658 00:41:07,000 --> 00:41:11,880 to capture maybe what they put in there, what was their experience in their workflow, as 659 00:41:11,880 --> 00:41:16,360 they added items, looked at items, navigated through different pages. 660 00:41:16,360 --> 00:41:18,600 All of that is unauthenticated. 661 00:41:18,600 --> 00:41:23,960 You can still of course use an unauthenticated experience, but when you're operating data 662 00:41:23,960 --> 00:41:32,240 from your own servers, your applications running in your own context, your own VMs, having 663 00:41:32,240 --> 00:41:38,240 that authentication only flow provides an extra layer of security on top of your data. 664 00:41:38,240 --> 00:41:40,200 There are some customers that I work with. 665 00:41:40,200 --> 00:41:41,840 They'll be very excited to hear that. 666 00:41:41,840 --> 00:41:42,840 That's good news. 667 00:41:42,840 --> 00:41:48,240 Well, I think this brings things to an end, but before you go, we always ask our guest, 668 00:41:48,240 --> 00:41:52,680 is there any one final thought you would like to leave our listeners with? 669 00:41:52,680 --> 00:41:57,880 I think the final thought is maybe going back to that beginning what you were talking about 670 00:41:57,880 --> 00:42:03,480 and what is Azure Monitor and maybe some of the confusion around Azure Monitor and these 671 00:42:03,480 --> 00:42:04,680 different pieces. 672 00:42:04,680 --> 00:42:13,680 I think it really comes down to, View Azure Monitor is our convergence across both Microsoft 673 00:42:13,680 --> 00:42:16,800 internal monitoring and external monitoring. 674 00:42:16,800 --> 00:42:24,560 As we bring all of the monitoring capabilities into one space, and so instead of thinking 675 00:42:24,560 --> 00:42:31,960 of it as, am I using application insights or am I using log analytics, I'm using Azure 676 00:42:31,960 --> 00:42:39,400 Monitor to monitor all of my resources, all of my applications, and get the right rich 677 00:42:39,400 --> 00:42:43,200 curated experiences and insights on top of those. 678 00:42:43,200 --> 00:42:45,560 David, thank you so much for joining us this week. 679 00:42:45,560 --> 00:42:47,320 I really appreciate you taking the time. 680 00:42:47,320 --> 00:42:53,520 I know you're incredibly busy, but I also know Azure Monitor is such a cornerstone of 681 00:42:53,520 --> 00:42:58,920 management, ongoing use of our Azure platform. 682 00:42:58,920 --> 00:43:00,640 I think it's been great having you on here. 683 00:43:00,640 --> 00:43:05,880 Hopefully, for some of our listeners out there, I hope this has helped clean up some of the 684 00:43:05,880 --> 00:43:11,080 misconceptions or confusion about what Azure Monitor actually is. 685 00:43:11,080 --> 00:43:13,920 To our listeners out there, thank you so much for listening. 686 00:43:13,920 --> 00:43:14,920 Stay safe out there. 687 00:43:14,920 --> 00:43:15,920 We'll see you next time. 688 00:43:15,920 --> 00:43:19,280 Thanks for listening to the Azure Security Podcast. 689 00:43:19,280 --> 00:43:26,080 You can find show notes and other resources at our website azsecuritypodcast.net. 690 00:43:26,080 --> 00:43:31,240 If you have any questions, please find us on Twitter at Azure SecPod. 691 00:43:31,240 --> 00:43:45,520 Music is from ccmixter.com and licensed under the Creative Commons license.