1 00:00:00,000 --> 00:00:02,640 Today on our Tech for Business podcast, 2 00:00:02,640 --> 00:00:05,640 we're joined by Nate, our Director of Cyber Security, 3 00:00:05,640 --> 00:00:09,240 and Andrew, our Security Incident Response Team Lead. 4 00:00:09,240 --> 00:00:12,400 This month, we've been talking about incident scenarios 5 00:00:12,400 --> 00:00:14,480 and what you can do to avoid them. 6 00:00:14,480 --> 00:00:16,440 We started with close calls, 7 00:00:16,440 --> 00:00:19,940 we went into some email compromise and wire transfers, 8 00:00:19,940 --> 00:00:25,600 and today, we've got 10 different examples about ransomware. 9 00:00:25,600 --> 00:00:27,360 Before we really jump in, 10 00:00:27,360 --> 00:00:30,240 I'm going to throw my first two questions to Nate. 11 00:00:30,240 --> 00:00:33,600 First, what terminology do we need to know, 12 00:00:33,600 --> 00:00:35,800 what are those beautiful acronyms that we should know 13 00:00:35,800 --> 00:00:38,240 to understand today's podcast, 14 00:00:38,240 --> 00:00:42,280 and then what is kind of the number one thing 15 00:00:42,280 --> 00:00:44,920 or a couple of things a customer could do 16 00:00:44,920 --> 00:00:47,520 to stay off of this list? 17 00:00:47,520 --> 00:00:49,000 Yeah. 18 00:00:49,000 --> 00:00:53,360 The, in terms of different acronyms to remember here, 19 00:00:53,360 --> 00:00:58,360 so we're going to call out EDR nonstop through this, I'm sure. 20 00:00:58,360 --> 00:01:00,840 So endpoint detection response, 21 00:01:00,840 --> 00:01:04,320 it's a, you know, traditionally you had antivirus, 22 00:01:04,320 --> 00:01:07,000 now you have EDR, there's a lot of different acronyms, 23 00:01:07,000 --> 00:01:11,280 EDR, MDR, XDR, I'm not going to go into depth of those ones, 24 00:01:11,280 --> 00:01:14,600 but essentially what it does is it's bringing in behavioral 25 00:01:14,600 --> 00:01:18,480 monitoring into your traditional antivirus solution, 26 00:01:18,480 --> 00:01:20,920 trying to catch the unknown risks, 27 00:01:20,920 --> 00:01:23,480 something that even if it's not malicious software, 28 00:01:23,480 --> 00:01:28,600 but it behaves abnormally in a malicious way, 29 00:01:28,600 --> 00:01:31,000 it can still take action on that type of stuff. 30 00:01:31,000 --> 00:01:34,120 So that's going to be a big one, MFA, 31 00:01:34,120 --> 00:01:37,760 multi-factor authentication or two-factor authentication, 32 00:01:37,760 --> 00:01:41,280 depending on what you're used to hearing. 33 00:01:41,280 --> 00:01:43,840 We might bring up, you know, IC3, 34 00:01:43,840 --> 00:01:45,040 we talked about that a little bit 35 00:01:45,040 --> 00:01:47,880 on the email compromise wire transfer. 36 00:01:47,880 --> 00:01:50,240 It's the Internet Crime and Complaint Center, 37 00:01:50,240 --> 00:01:54,000 which is the FBI's reporting division of cyber crime. 38 00:01:57,120 --> 00:02:02,120 The, I might bring it up, RTOs, RPOs a little bit, 39 00:02:03,120 --> 00:02:07,640 so recovery time objectives, recovery point objectives, 40 00:02:07,640 --> 00:02:11,280 business continuity planning, all of that type of stuff 41 00:02:11,280 --> 00:02:13,960 is all going to be tied to your backups, 42 00:02:13,960 --> 00:02:17,320 how fast you can back things up, 43 00:02:17,320 --> 00:02:21,080 at how much data can you potentially lose throughout that. 44 00:02:21,080 --> 00:02:25,240 Those are the big ones that I can think of, 45 00:02:25,240 --> 00:02:26,680 off the top of my head. 46 00:02:26,680 --> 00:02:30,440 And then I would say the biggest things 47 00:02:30,440 --> 00:02:33,440 to try and stay off of this list is, 48 00:02:35,840 --> 00:02:38,040 EDR and multi-factor are going to be the two biggest ones 49 00:02:38,040 --> 00:02:39,240 that come to mind. 50 00:02:39,240 --> 00:02:41,880 Unfortunately, in most of these cases, 51 00:02:41,880 --> 00:02:45,800 you're going to find that having one or both 52 00:02:45,800 --> 00:02:48,520 of those solutions would have prevented a lot of this stuff 53 00:02:48,520 --> 00:02:50,320 from happening in the first place. 54 00:02:52,680 --> 00:02:54,920 Removing local admin permissions, 55 00:02:54,920 --> 00:02:58,960 otherwise there's also application of whitelisting 56 00:02:58,960 --> 00:03:02,000 is another one, so no new software can be introduced 57 00:03:02,000 --> 00:03:05,440 into your environment unless it's previously approved, 58 00:03:05,440 --> 00:03:08,040 because every one of these ransomware cases, 59 00:03:08,040 --> 00:03:11,320 the threat actor had to introduce new malicious software 60 00:03:11,320 --> 00:03:15,000 into the environment to encrypt data or steal data. 61 00:03:15,000 --> 00:03:16,960 And so if you have some type of application 62 00:03:16,960 --> 00:03:19,320 whitelisting tool in place, 63 00:03:19,320 --> 00:03:21,920 you're going to very quickly prevent yourself 64 00:03:21,920 --> 00:03:23,320 from being on this type of list. 65 00:03:25,560 --> 00:03:27,600 Sure, I don't know if you had anything else. 66 00:03:27,600 --> 00:03:32,320 I think one thing, DR disaster recovery. 67 00:03:32,320 --> 00:03:35,160 You're talking about with the RTL or RPO. 68 00:03:35,160 --> 00:03:36,760 So yeah, I think disaster recovery 69 00:03:36,760 --> 00:03:37,880 is the only other one that I think 70 00:03:37,880 --> 00:03:40,480 that we are going to be talking quite a bit about. 71 00:03:40,480 --> 00:03:45,480 For sure, and all the names and some information 72 00:03:46,160 --> 00:03:48,360 have been changed to protect the innocent, 73 00:03:48,360 --> 00:03:50,960 just saying our little legal spiel. 74 00:03:50,960 --> 00:03:53,640 But yeah, how do you want to start us off today? 75 00:03:55,240 --> 00:03:57,480 I guess I'll start off with the first one. 76 00:03:57,480 --> 00:04:00,000 This was a relatively recent incident. 77 00:04:01,000 --> 00:04:03,800 It was from a threat actor group called Akira. 78 00:04:04,840 --> 00:04:09,000 This client, they were compromised through their VPN, 79 00:04:09,000 --> 00:04:13,920 the Cisco VPN, very prevalent now in ransomware cases 80 00:04:13,920 --> 00:04:16,280 just because there are plenty of vulnerabilities 81 00:04:16,280 --> 00:04:19,160 that have come out for the Cisco and Cisco VPNs. 82 00:04:20,840 --> 00:04:22,720 What happened was, Tacker got in, 83 00:04:23,600 --> 00:04:26,000 they were able to move laterally, 84 00:04:26,000 --> 00:04:28,360 escalate their privileges due to weak password policies, 85 00:04:28,360 --> 00:04:32,000 overused admin, the same admin password 86 00:04:32,000 --> 00:04:34,640 for all the local devices. 87 00:04:34,640 --> 00:04:39,200 In this event, the attackers were able to encrypt 88 00:04:39,200 --> 00:04:41,680 their backup server, their backup server, 89 00:04:42,920 --> 00:04:45,160 essentially encrypted the backups off in the cloud. 90 00:04:45,160 --> 00:04:47,760 So essentially they ransomware themselves 91 00:04:47,760 --> 00:04:49,880 by having this server get encrypted. 92 00:04:51,280 --> 00:04:53,640 Important things for that to prevent. 93 00:04:53,640 --> 00:04:57,200 EDR would have been a big and important tool 94 00:04:57,200 --> 00:05:00,040 to have in place, making sure that you do not reuse 95 00:05:00,040 --> 00:05:03,280 local admin passwords, Microsoft Laps. 96 00:05:03,280 --> 00:05:07,480 That's a free tool that can be deployed, very easy to do. 97 00:05:08,840 --> 00:05:11,240 As far as other things, I would also say, 98 00:05:11,240 --> 00:05:13,480 making sure your VPN and all points of egress 99 00:05:13,480 --> 00:05:15,240 on your network are up to date. 100 00:05:15,240 --> 00:05:18,760 Firewalls, your VPNs, any sort of SD-WAN, 101 00:05:18,760 --> 00:05:21,280 anything like that, where somebody from the outside 102 00:05:21,280 --> 00:05:23,680 can come in, it's the most up to date as possible. 103 00:05:25,960 --> 00:05:30,960 Another event that happened was through a threat actor 104 00:05:30,960 --> 00:05:32,960 called Lockbit 2.0. 105 00:05:32,960 --> 00:05:36,760 Again, this was a compromise of a Cisco VPN, 106 00:05:36,760 --> 00:05:39,520 very similar in timeframe from the past one 107 00:05:39,520 --> 00:05:40,600 that we spoke about. 108 00:05:41,800 --> 00:05:44,720 This one, the client did have EDR, 109 00:05:44,720 --> 00:05:48,280 but they did not have EDR on all of their endpoints. 110 00:05:48,280 --> 00:05:53,120 So the attacker was able to compromise a device without EDR 111 00:05:53,120 --> 00:05:55,800 and then just hammer on every single other device 112 00:05:55,800 --> 00:06:00,800 until they were able to encrypt other devices. 113 00:06:02,240 --> 00:06:04,840 So again, making sure that your firmware is up to date 114 00:06:04,840 --> 00:06:07,280 on all externally facing devices, 115 00:06:07,280 --> 00:06:09,400 and making sure that you have full coverage 116 00:06:09,400 --> 00:06:11,360 of all the devices that come into your network 117 00:06:11,360 --> 00:06:15,280 that you own, make sure they have an antivirus solution, 118 00:06:15,280 --> 00:06:17,880 preferably EDR, but have some sort of 119 00:06:17,880 --> 00:06:19,840 software protection in place. 120 00:06:19,840 --> 00:06:24,840 Because while EDR did make their job harder, 121 00:06:24,840 --> 00:06:26,760 no tool is perfect. 122 00:06:26,760 --> 00:06:29,240 And as we've talked about the security onion, 123 00:06:29,240 --> 00:06:33,600 layering your defenses, EDR defended, 124 00:06:33,600 --> 00:06:35,240 but attackers found other ways 125 00:06:35,240 --> 00:06:37,240 because they had that foothold. 126 00:06:37,240 --> 00:06:39,120 Yeah, that one was super interesting 127 00:06:40,320 --> 00:06:42,600 just for some additional context behind that one, 128 00:06:42,600 --> 00:06:47,600 because in the EDR portal, we saw that the VPN, 129 00:06:49,280 --> 00:06:51,680 again, there was an account that was on that VPN 130 00:06:51,680 --> 00:06:52,520 that was compromised. 131 00:06:52,520 --> 00:06:54,640 So again, multi-factor would have been a nice way 132 00:06:54,640 --> 00:06:56,040 to help prevent that. 133 00:06:56,040 --> 00:07:01,040 But from the logs, we saw the Thread Actor attempt 134 00:07:02,200 --> 00:07:07,040 to escalate privileges to the servers 135 00:07:07,040 --> 00:07:08,440 over and over and over again. 136 00:07:08,440 --> 00:07:11,120 And so in the logs, we saw the same attempt 137 00:07:11,120 --> 00:07:14,520 to all the different servers that had EDR installed. 138 00:07:14,520 --> 00:07:16,120 And then the next logs that we saw 139 00:07:16,120 --> 00:07:18,400 was coming from an internal host, 140 00:07:18,400 --> 00:07:22,600 or workstation that did not have the EDR installed. 141 00:07:22,600 --> 00:07:24,640 And then they tried doing the same thing 142 00:07:24,640 --> 00:07:26,640 from that to the servers. 143 00:07:26,640 --> 00:07:31,080 And again, the EDR solution protected a lot of that stuff. 144 00:07:31,080 --> 00:07:35,880 What happened was that one internal device 145 00:07:35,880 --> 00:07:38,640 that they didn't have EDR deployed on 146 00:07:39,960 --> 00:07:41,400 had access to the file shares. 147 00:07:41,400 --> 00:07:44,000 And so it started trying to encrypt all that data. 148 00:07:44,000 --> 00:07:47,640 Now the EDR solution was able to start restoring 149 00:07:47,640 --> 00:07:49,320 a bunch of data as much as it could 150 00:07:49,320 --> 00:07:53,160 based off of, I'm not gonna get too much of the depth here, 151 00:07:53,160 --> 00:07:55,360 but on the way that that typically works, 152 00:07:55,360 --> 00:07:57,320 but it was able to start trying to restore 153 00:07:57,320 --> 00:07:58,840 as many files as it could. 154 00:08:00,040 --> 00:08:03,600 Unfortunately, they did have to start pulling from backups 155 00:08:03,600 --> 00:08:06,840 because there were some limitations of data coverage 156 00:08:06,840 --> 00:08:09,040 of what was acceptable. 157 00:08:09,040 --> 00:08:11,480 So this is again, where we're starting to talk 158 00:08:11,480 --> 00:08:14,000 about things like recovery time objectives 159 00:08:14,000 --> 00:08:17,560 and recovery point objectives is if, 160 00:08:17,560 --> 00:08:20,680 let's say you're taking a snapshot every 24 hours 161 00:08:20,680 --> 00:08:23,200 or every five hours, but you actually needed data 162 00:08:23,200 --> 00:08:27,440 within the last hour, you might still need to pull back 163 00:08:27,440 --> 00:08:32,440 from a backup or potentially update your backup frequency 164 00:08:33,080 --> 00:08:35,960 to help accommodate the way that the actual business needs 165 00:08:35,960 --> 00:08:36,800 are. 166 00:08:37,920 --> 00:08:40,080 But again, for me, that was a really, really interesting 167 00:08:40,080 --> 00:08:43,440 one seeing the tool continually block and block and block 168 00:08:43,440 --> 00:08:46,240 until it found that device that wasn't protected. 169 00:08:46,240 --> 00:08:51,240 So I guess if you're keeping track of 170 00:08:55,000 --> 00:08:58,520 or just watching through all these different podcasts, 171 00:08:58,520 --> 00:09:03,520 org eight in our list here or three today is one that 172 00:09:04,840 --> 00:09:08,640 this is a customer that was doing almost everything right. 173 00:09:08,640 --> 00:09:12,000 They didn't have EDR deployed when this happened, 174 00:09:12,000 --> 00:09:14,320 but they did have network monitoring. 175 00:09:14,320 --> 00:09:16,040 They had phenomenal backups. 176 00:09:16,040 --> 00:09:18,800 They had multi-factor widely rolled out 177 00:09:18,800 --> 00:09:20,800 across the organization. 178 00:09:20,800 --> 00:09:23,800 Unfortunately, their users did have local admin as well. 179 00:09:23,800 --> 00:09:28,280 And so what happened in this next case was that 180 00:09:28,280 --> 00:09:30,600 there was a group called Conti. 181 00:09:30,600 --> 00:09:34,680 So this is an interesting group that has now shut down. 182 00:09:34,680 --> 00:09:37,800 So this is a group that's historically composed 183 00:09:37,800 --> 00:09:40,320 a lot of the Russian and Ukrainian hackers, 184 00:09:40,320 --> 00:09:42,440 obviously with the war, they kind of split apart 185 00:09:42,440 --> 00:09:43,640 and everything like that. 186 00:09:43,640 --> 00:09:46,560 But essentially what happened was 187 00:09:46,560 --> 00:09:51,000 we found that there was a document in this environment, 188 00:09:51,000 --> 00:09:53,680 which was an Excel file with a malicious macro 189 00:09:53,680 --> 00:09:54,920 associated with it. 190 00:09:54,920 --> 00:09:57,880 So a user received that phishing email, 191 00:09:57,880 --> 00:10:00,560 clicked on the Excel document, was able to run that. 192 00:10:00,560 --> 00:10:03,920 And then that's what kicked off the ransomware incident. 193 00:10:05,880 --> 00:10:08,040 So the biggest thing was they didn't have EDR 194 00:10:08,040 --> 00:10:09,280 because we deployed that for them. 195 00:10:09,280 --> 00:10:12,280 And then that's what found all these remote connections 196 00:10:12,280 --> 00:10:15,280 in their environment out to the threat actor. 197 00:10:15,280 --> 00:10:19,120 It's what helped find the initial Excel file 198 00:10:19,120 --> 00:10:21,320 sitting in their download folder. 199 00:10:22,360 --> 00:10:23,840 But what we were able to do is 200 00:10:23,840 --> 00:10:28,840 because they had solid and tested backups, 201 00:10:28,840 --> 00:10:32,200 we were able to get their entire environment operational 202 00:10:32,200 --> 00:10:34,400 in less than four hours, 203 00:10:34,400 --> 00:10:36,520 let them continue doing the workday 204 00:10:36,520 --> 00:10:40,680 as while that environment was running, 205 00:10:40,680 --> 00:10:44,200 we were able to continue doing full restores and forensics 206 00:10:44,200 --> 00:10:45,680 without impacting the business. 207 00:10:45,680 --> 00:10:50,680 So again, if you start to pair all these different solutions 208 00:10:51,760 --> 00:10:54,360 in place and have a proper methodology 209 00:10:54,360 --> 00:10:55,920 to protecting the business, 210 00:10:57,240 --> 00:11:02,240 a ransomware incident can be extremely minor impact 211 00:11:02,240 --> 00:11:05,360 compared to the traditional impact. 212 00:11:05,360 --> 00:11:08,480 Andrew, what's the latest downtime on average 213 00:11:08,480 --> 00:11:10,400 for a ransomware incident? 214 00:11:10,400 --> 00:11:12,840 Two weeks, I think is the number, 215 00:11:12,840 --> 00:11:14,840 but it could be even longer now 216 00:11:14,840 --> 00:11:16,360 with how the attackers are doing their stuff. 217 00:11:16,360 --> 00:11:18,120 So for two weeks, downtime, 218 00:11:18,120 --> 00:11:21,720 like restoration of business kind of time. 219 00:11:21,720 --> 00:11:24,320 That's probably your least favorite news to tell a customer 220 00:11:24,320 --> 00:11:26,680 when we tell them that they're potentially 221 00:11:26,680 --> 00:11:28,280 gonna be down for two weeks, isn't it? 222 00:11:28,280 --> 00:11:32,000 Yeah, it changes the mood of the conversation 223 00:11:32,000 --> 00:11:34,080 as just, hey, this is an inconvenience to, 224 00:11:34,080 --> 00:11:35,840 hey, this could be business ending, 225 00:11:35,840 --> 00:11:40,840 especially businesses that run on slim margins. 226 00:11:41,160 --> 00:11:43,440 They don't have a lot of money coming in to pay everything. 227 00:11:43,440 --> 00:11:45,440 They depend on the ongoing business 228 00:11:45,440 --> 00:11:47,240 and being down for two weeks 229 00:11:47,240 --> 00:11:48,720 where they don't have that incoming business, 230 00:11:48,720 --> 00:11:50,760 it can end the business. 231 00:11:50,760 --> 00:11:52,640 And so that really changes the conversation 232 00:11:52,640 --> 00:11:54,440 if that they know it's, 233 00:11:54,440 --> 00:11:57,080 we shoot to have it up as soon as possible, 234 00:11:58,240 --> 00:12:00,840 but average downtime is about that two week. 235 00:12:00,840 --> 00:12:01,680 Yeah. 236 00:12:05,000 --> 00:12:07,160 I think that I'll kind of use that as a segue 237 00:12:07,160 --> 00:12:08,200 into the next organization, 238 00:12:08,200 --> 00:12:13,200 because this customer, they had great backups that worked. 239 00:12:13,480 --> 00:12:16,200 However, they didn't do a great job at testing them 240 00:12:16,200 --> 00:12:18,400 with a disaster recovery test 241 00:12:18,400 --> 00:12:21,680 to ensure that they could restore as quickly as possible. 242 00:12:22,600 --> 00:12:26,040 And so even though we were able to get the business 243 00:12:26,040 --> 00:12:30,120 operational and limping along in the first couple hours, 244 00:12:30,120 --> 00:12:34,400 they essentially were majorly impacted for two weeks. 245 00:12:34,400 --> 00:12:38,760 And so this was one where the customer had a Sonic wall, 246 00:12:38,760 --> 00:12:42,880 VPN that hadn't, it was actually end of life at the time, 247 00:12:42,880 --> 00:12:45,000 hadn't been updated in years, 248 00:12:45,000 --> 00:12:47,120 there's vulnerabilities on it, 249 00:12:47,120 --> 00:12:49,920 but they just kept using it and everything like that. 250 00:12:49,920 --> 00:12:52,520 You know, going back, we even told them, 251 00:12:52,520 --> 00:12:54,240 hey, this is end of life, you need to get rid of it. 252 00:12:54,240 --> 00:12:55,760 It's a security risk. 253 00:12:55,760 --> 00:12:58,800 Didn't upgrade it, therefore it led to this incident, 254 00:12:58,800 --> 00:13:01,480 but that VPN was compromised. 255 00:13:01,480 --> 00:13:04,960 And so with the backups, essentially what happened was, 256 00:13:04,960 --> 00:13:09,960 they had these backups keeping all of their different servers 257 00:13:09,960 --> 00:13:13,640 restored and again, testing them every single day 258 00:13:13,640 --> 00:13:16,280 that they could backup appropriately, 259 00:13:16,280 --> 00:13:21,280 but what went wrong in this case was that this customer had, 260 00:13:21,880 --> 00:13:23,440 I think it was six different servers 261 00:13:23,440 --> 00:13:25,960 that all had to inter-communicate with each other 262 00:13:25,960 --> 00:13:28,640 to provide the total solution. 263 00:13:28,640 --> 00:13:33,080 And what happened there was the appliance 264 00:13:33,080 --> 00:13:37,240 that was running those backups and backup recoveries 265 00:13:37,240 --> 00:13:40,400 could only spin up three or four of them. 266 00:13:40,400 --> 00:13:43,640 And so they had a couple servers that you could not run. 267 00:13:43,640 --> 00:13:45,040 And so then you had to spend time 268 00:13:45,040 --> 00:13:47,640 doing full restores on those ones 269 00:13:47,640 --> 00:13:51,240 rather than running them in a virtualized state. 270 00:13:51,240 --> 00:13:53,120 Therefore the business is still down. 271 00:13:53,120 --> 00:13:56,880 And so that's what just dragged it on another couple days, 272 00:13:56,880 --> 00:14:01,880 which again, it was a significant amount of money lost 273 00:14:01,880 --> 00:14:05,800 in productivity for that organization. 274 00:14:08,160 --> 00:14:11,480 Customer also didn't have EDR implemented out there. 275 00:14:11,480 --> 00:14:14,240 So people are able to move laterally 276 00:14:14,240 --> 00:14:17,200 as new malicious software is being loaded in, 277 00:14:17,200 --> 00:14:20,760 trying to steal data outside of the organization. 278 00:14:20,760 --> 00:14:22,280 That's another big issue. 279 00:14:22,280 --> 00:14:26,160 They did have sensitive data about customers 280 00:14:26,160 --> 00:14:28,760 sitting on their network that was lost as well. 281 00:14:28,760 --> 00:14:32,720 So then you had to go through legal reporting off of that. 282 00:14:32,720 --> 00:14:37,720 And then I believe the last thing 283 00:14:38,240 --> 00:14:41,720 that I wanted to mention about that was, 284 00:14:45,480 --> 00:14:47,760 their organization was, 285 00:14:48,880 --> 00:14:49,960 sorry, I'm also trying to make sure 286 00:14:49,960 --> 00:14:52,680 that I'm not sharing a little too much info here. 287 00:14:54,560 --> 00:14:55,760 It was business altering. 288 00:14:55,760 --> 00:14:58,840 We'll just put it that way is they had to completely change 289 00:14:58,840 --> 00:15:01,800 how their business operated due to this incident, 290 00:15:01,800 --> 00:15:04,400 even though they were able to successfully restore. 291 00:15:04,400 --> 00:15:05,240 So. 292 00:15:09,840 --> 00:15:12,880 The, Kelsey and Ariel, 293 00:15:12,880 --> 00:15:14,600 I don't know if you have any questions along that. 294 00:15:14,600 --> 00:15:16,280 I've been talking a lot here. 295 00:15:19,480 --> 00:15:20,320 I was almost- 296 00:15:20,320 --> 00:15:22,400 Yeah, I'll let you, I've got kind of 297 00:15:24,000 --> 00:15:25,280 a little bit of a wrap up question. 298 00:15:25,280 --> 00:15:27,560 I mean, I'll ask that later, 299 00:15:27,560 --> 00:15:31,480 but this feels very stressful. 300 00:15:31,480 --> 00:15:33,560 If you're in this situation as a company, 301 00:15:33,560 --> 00:15:35,760 I mean, what do you do? 302 00:15:35,760 --> 00:15:39,920 Like, do you call your IT? 303 00:15:39,920 --> 00:15:43,200 Do you call the president and CEO? 304 00:15:43,200 --> 00:15:44,600 I mean, if you're in this situation 305 00:15:44,600 --> 00:15:45,720 and you're looking at your business 306 00:15:45,720 --> 00:15:47,440 potentially being down for two weeks, 307 00:15:47,440 --> 00:15:49,840 what does that kind of that first step look like? 308 00:15:51,640 --> 00:15:52,480 If I may. 309 00:15:52,480 --> 00:15:56,400 What's the first step on our incident response plan? 310 00:15:56,400 --> 00:15:57,240 Basically. 311 00:15:57,240 --> 00:15:59,080 Well, make sure you have an incident response plan. 312 00:15:59,080 --> 00:15:59,920 Yeah, exactly. 313 00:15:59,920 --> 00:16:01,480 That's really important. 314 00:16:01,480 --> 00:16:03,920 You never hope to have an incident, 315 00:16:03,920 --> 00:16:06,600 but make sure you have an incident response plan. 316 00:16:06,600 --> 00:16:09,320 Who is responsible for what applications? 317 00:16:09,320 --> 00:16:10,600 Who is the call tree? 318 00:16:10,600 --> 00:16:14,200 Who are the important people in the mix? 319 00:16:14,200 --> 00:16:16,040 When it comes to IT, when it comes to finance, 320 00:16:16,040 --> 00:16:19,440 when it comes to everything, 321 00:16:19,440 --> 00:16:22,320 it's important to know these things ahead of time. 322 00:16:22,320 --> 00:16:24,880 So then you know what to do 323 00:16:24,880 --> 00:16:26,880 because it's really no fun to get on a call 324 00:16:26,880 --> 00:16:29,320 and we have to learn everything about the network 325 00:16:29,320 --> 00:16:31,760 before we can actually start helping you. 326 00:16:32,920 --> 00:16:33,760 Yeah. 327 00:16:35,600 --> 00:16:39,440 The fact that came to mind was on our internal incident 328 00:16:39,440 --> 00:16:42,680 response playbook essentially, 329 00:16:42,680 --> 00:16:44,600 the first step that we have to find on that 330 00:16:44,600 --> 00:16:46,360 is take a deep breath, call your family 331 00:16:46,360 --> 00:16:48,920 and tell them you're gonna be late home 332 00:16:48,920 --> 00:16:51,160 and it's gonna be a long couple of weeks. 333 00:16:51,160 --> 00:16:53,640 It's just the nature of it, right? 334 00:16:53,640 --> 00:16:58,480 Is for Andrew and I on that last customer 335 00:16:58,480 --> 00:17:00,520 that I talked about, 336 00:17:00,520 --> 00:17:03,640 Andrew, I think you and I were up for almost three days straight 337 00:17:03,640 --> 00:17:07,200 for the first, yeah, the three days, 338 00:17:07,200 --> 00:17:11,280 just trying to get that customer operating and moving along. 339 00:17:11,280 --> 00:17:13,800 So, you know, we put in almost two weeks of time 340 00:17:13,800 --> 00:17:18,800 in the first three days and over the next two weeks, 341 00:17:18,800 --> 00:17:22,440 I couldn't tell you how many memes and gifts 342 00:17:22,440 --> 00:17:24,560 and everything we sent to each other 343 00:17:24,560 --> 00:17:28,640 just to keep ourselves sane during that engagement. 344 00:17:28,640 --> 00:17:32,040 But that was also one of my favorite times at CIT 345 00:17:32,040 --> 00:17:34,440 because I don't know if we were just so slap happy or what, 346 00:17:34,440 --> 00:17:35,280 but... 347 00:17:36,400 --> 00:17:38,280 There might have been a lack of sleep delirium 348 00:17:38,280 --> 00:17:39,840 or something like that. 349 00:17:39,840 --> 00:17:42,760 But that is the first thing for, you know, 350 00:17:42,760 --> 00:17:47,240 as you face these types of situations is take the breath, 351 00:17:47,240 --> 00:17:48,800 tell people that you're gonna be late. 352 00:17:48,800 --> 00:17:50,000 It's going to be stressful. 353 00:17:50,000 --> 00:17:51,480 We know that. 354 00:17:51,480 --> 00:17:54,120 Our job is to help reduce that stress, 355 00:17:54,120 --> 00:17:56,760 help provide the empathy, help provide the... 356 00:17:58,840 --> 00:18:01,920 The skill set and guidance to help you navigate 357 00:18:01,920 --> 00:18:02,960 that situation. 358 00:18:04,120 --> 00:18:06,800 You know, I guess the one big thing here, 359 00:18:06,800 --> 00:18:09,480 and I talked about it a little bit is, 360 00:18:09,480 --> 00:18:11,880 how do you prevent yourself from being on this list? 361 00:18:11,880 --> 00:18:14,520 And one of the things that come to mind is, 362 00:18:14,520 --> 00:18:17,480 if there are security recommendations that come out 363 00:18:17,480 --> 00:18:20,320 of either CIT, another provider, 364 00:18:20,320 --> 00:18:25,320 your own internal employees, please take it seriously, 365 00:18:25,560 --> 00:18:27,920 because oftentimes something like, 366 00:18:27,920 --> 00:18:29,720 we need to update our firewall. 367 00:18:29,720 --> 00:18:32,560 That seems minor, but we just talked about 368 00:18:32,560 --> 00:18:36,600 three different cases of the VPN being compromised 369 00:18:37,400 --> 00:18:39,640 due to outdated software or, you know, 370 00:18:39,640 --> 00:18:43,960 MFA not being implemented for a user accounts there. 371 00:18:43,960 --> 00:18:45,520 This is serious stuff. 372 00:18:45,520 --> 00:18:49,920 And then also is oftentimes what we see is that 373 00:18:49,920 --> 00:18:52,920 organizations say, I don't have the money to do that. 374 00:18:54,720 --> 00:18:58,480 It might only take you an hour or two of time 375 00:18:58,480 --> 00:18:59,560 to implement that. 376 00:19:01,160 --> 00:19:04,840 But the alternative to that is thousands and thousands 377 00:19:04,840 --> 00:19:09,400 of dollars of incident response or forensics responses, 378 00:19:09,400 --> 00:19:12,720 legal responses, everything like that. 379 00:19:12,720 --> 00:19:14,720 And it seems like organizations always seem 380 00:19:14,720 --> 00:19:16,480 to find the money afterwards. 381 00:19:16,480 --> 00:19:19,800 So being a little more proactive on it. 382 00:19:19,800 --> 00:19:21,560 The saying I like is, you know, 383 00:19:21,560 --> 00:19:24,120 pennies before an incident, dollars after, 384 00:19:25,000 --> 00:19:26,680 because, you know, if you pay some pennies here 385 00:19:26,680 --> 00:19:29,120 before something happens as a preventative measure, 386 00:19:29,120 --> 00:19:31,680 it's gonna help you not have to pay dollars later. 387 00:19:36,560 --> 00:19:40,960 Yeah, the last thing I had about that is, 388 00:19:40,960 --> 00:19:45,680 to implement EDR, you can typically spend 389 00:19:47,240 --> 00:19:52,240 10 years of EDR payments just to cover one security incident. 390 00:19:52,280 --> 00:19:56,160 It makes no sense why that would not be an investment 391 00:19:56,160 --> 00:20:00,160 because the risk is so high, the impact is so high, 392 00:20:00,160 --> 00:20:03,000 and the cost of EDR is so low. 393 00:20:03,000 --> 00:20:06,960 Multi-factor, most of them included for free these days. 394 00:20:06,960 --> 00:20:08,000 There are pain solutions. 395 00:20:08,000 --> 00:20:10,040 They have a couple extra bells and whistles, 396 00:20:10,040 --> 00:20:14,040 but 10 years of protection compared to one security incident. 397 00:20:14,040 --> 00:20:15,200 So, but... 398 00:20:15,200 --> 00:20:16,040 Sure. 399 00:20:19,000 --> 00:20:21,400 I'll touch on two, two more, 400 00:20:21,400 --> 00:20:24,040 and then I'll hand it over to Andrew for a little bit. 401 00:20:24,040 --> 00:20:27,640 This customer, they were not sitting in a great spot 402 00:20:27,640 --> 00:20:29,200 ahead of this security incident. 403 00:20:29,200 --> 00:20:31,920 They didn't have EDR, they didn't have a great antivirus, 404 00:20:31,920 --> 00:20:35,000 they didn't have multi-factor, anything like that. 405 00:20:35,000 --> 00:20:38,400 And so, but I wanted to talk a little bit more 406 00:20:38,400 --> 00:20:42,960 about the response capabilities and the response progress 407 00:20:42,960 --> 00:20:45,960 because that's what really impacted it the most. 408 00:20:45,960 --> 00:20:49,560 And so, this one was traditional phishing links, 409 00:20:49,560 --> 00:20:50,920 someone clicked on it. 410 00:20:50,920 --> 00:20:53,840 Their users did have local admin, which is terrible 411 00:20:53,840 --> 00:20:57,520 because it greatly expedites the attacker's ability 412 00:20:57,520 --> 00:21:00,400 to grab admin credentials in that environment. 413 00:21:02,200 --> 00:21:06,040 But one of the interesting things was we implemented EDR 414 00:21:06,040 --> 00:21:09,840 into this environment, we found 40 different computers 415 00:21:09,840 --> 00:21:13,480 that all had remote connections going out of the network, 416 00:21:13,480 --> 00:21:16,520 trying to maintain persistence to that threat actor. 417 00:21:16,520 --> 00:21:19,000 And so, what happened there was, 418 00:21:19,000 --> 00:21:22,280 even though you might have taken a machine offline 419 00:21:22,280 --> 00:21:27,000 and tried to clean it, you still had 39 or so computers 420 00:21:27,000 --> 00:21:32,000 that had access, so it was extremely dug into their network. 421 00:21:32,000 --> 00:21:34,320 Now, when we deploy that EDR, 422 00:21:34,320 --> 00:21:37,120 it found all those and shut them down really, really quickly. 423 00:21:38,120 --> 00:21:42,120 But where the issue lied with this environment 424 00:21:42,120 --> 00:21:45,080 was that the IT admins out there said, 425 00:21:45,080 --> 00:21:47,920 we are losing thousands of dollars every day 426 00:21:47,920 --> 00:21:50,320 with this system being shut down. 427 00:21:50,320 --> 00:21:53,520 And so, while we were going through 428 00:21:53,520 --> 00:21:55,800 the incident response process, 429 00:21:55,800 --> 00:22:00,000 which it has to be extremely meticulous and methodical 430 00:22:00,000 --> 00:22:04,160 of when do you bring up certain servers in particular order 431 00:22:05,160 --> 00:22:08,480 to help minimize reinfection of those 432 00:22:09,840 --> 00:22:13,840 brand new configured or restored systems, 433 00:22:14,840 --> 00:22:18,440 while it's still in a hostile environment, 434 00:22:18,440 --> 00:22:20,160 they kept bringing up these servers 435 00:22:20,160 --> 00:22:22,040 without the guidance of the forensics 436 00:22:22,040 --> 00:22:24,320 or CIT and center response team. 437 00:22:24,320 --> 00:22:27,200 And so, they actually ended up having to rebuild 438 00:22:27,200 --> 00:22:30,640 those systems three times because their internal IT 439 00:22:30,640 --> 00:22:32,800 kept spinning them up against guidance 440 00:22:32,800 --> 00:22:36,120 of the incident response and forensics professionals. 441 00:22:37,120 --> 00:22:39,080 And it led to additional downtime 442 00:22:39,080 --> 00:22:42,080 than if we had just gone through the proper process. 443 00:22:42,080 --> 00:22:45,840 And so, one of the big things here is 444 00:22:45,840 --> 00:22:48,280 when Andrew comes in and tells you 445 00:22:48,280 --> 00:22:50,920 that you're gonna be down for a potential two weeks 446 00:22:50,920 --> 00:22:55,920 on average and the customer wants to be up yesterday 447 00:22:55,920 --> 00:23:00,920 there's a delta there that you have to address 448 00:23:02,720 --> 00:23:05,720 on the emotional level almost and say, 449 00:23:05,720 --> 00:23:08,000 we know you want to move fast, 450 00:23:08,000 --> 00:23:11,240 but to move slow is to move fast 451 00:23:11,240 --> 00:23:12,800 because if you do it the proper way, 452 00:23:12,800 --> 00:23:15,160 you'll be up faster than having to do this 453 00:23:15,160 --> 00:23:16,920 two, three, four times again. 454 00:23:19,640 --> 00:23:21,640 And also in that slowness, 455 00:23:21,640 --> 00:23:23,920 if you do have cybersecurity insurance, 456 00:23:23,920 --> 00:23:27,360 there is requirements for them to pay 457 00:23:27,360 --> 00:23:29,120 that certain things are done. 458 00:23:29,120 --> 00:23:31,120 And if you do them out of order, you lose logs, 459 00:23:31,120 --> 00:23:35,080 they can deny your claim and then you're really up the creek. 460 00:23:35,080 --> 00:23:40,080 So it's important to move slowly, to move fast in the end. 461 00:23:41,480 --> 00:23:45,040 Yeah, and Andrew's talking about the OFAC check. 462 00:23:45,040 --> 00:23:48,240 This is something that the US Treasury put into place, 463 00:23:48,240 --> 00:23:50,320 I believe it was 2020 now, 464 00:23:50,320 --> 00:23:55,320 where they will prohibit payments to ransomware groups 465 00:23:55,320 --> 00:23:58,920 or nation states that are on a sanctioned list. 466 00:23:58,920 --> 00:24:00,600 And so in those cases, 467 00:24:02,280 --> 00:24:04,840 if you are compromised and it's known to be a 468 00:24:06,280 --> 00:24:07,520 sanctioned threat group, 469 00:24:07,520 --> 00:24:11,120 you're not authorized to make payment to that group, 470 00:24:11,120 --> 00:24:14,320 to restore data or retrieve files. 471 00:24:14,320 --> 00:24:16,280 Otherwise you could actually face a fine 472 00:24:16,280 --> 00:24:18,400 from the US government as well 473 00:24:18,400 --> 00:24:20,960 for making that payment on top of that. 474 00:24:20,960 --> 00:24:23,960 Therefore, and then also for a managed service provider 475 00:24:23,960 --> 00:24:26,520 like CIT, we cannot assist you with that 476 00:24:26,520 --> 00:24:29,600 because we could also face similar fines 477 00:24:29,600 --> 00:24:32,280 from the government for assisting you with that. 478 00:24:32,280 --> 00:24:36,320 So the government, CIT, we all say, 479 00:24:36,320 --> 00:24:39,960 do not pay the ransomware demand. 480 00:24:39,960 --> 00:24:43,680 The biggest thing is again, all this preventative time 481 00:24:43,680 --> 00:24:45,840 to prevent it from happening in the first place 482 00:24:45,840 --> 00:24:49,200 because it will easily dwarf whatever the cost 483 00:24:49,200 --> 00:24:51,240 of a major security incident is. 484 00:24:56,080 --> 00:24:58,920 Last one, this one holds a special place in my heart 485 00:24:58,920 --> 00:25:02,000 because it's the first one I ever had to respond to. 486 00:25:02,000 --> 00:25:04,280 Sorry to the customer if you're listening 487 00:25:04,280 --> 00:25:07,840 that this is someone that just called into CIT. 488 00:25:07,840 --> 00:25:11,160 I was brand new into CIT at the time 489 00:25:11,160 --> 00:25:14,040 and they just said, hey, our environment's down, 490 00:25:14,040 --> 00:25:15,480 can you come help us out? 491 00:25:15,480 --> 00:25:20,240 And I went out there and started helping out 492 00:25:20,240 --> 00:25:23,920 with ransomware and doing a little investigation, 493 00:25:23,920 --> 00:25:25,640 trying to figure out what happened, 494 00:25:25,640 --> 00:25:27,200 what does the environment even look like, 495 00:25:27,200 --> 00:25:30,280 everything like that, because again, brand new to us. 496 00:25:30,280 --> 00:25:33,720 And I found that there was RIAC ransomware, 497 00:25:33,720 --> 00:25:37,080 which used to be one of the most expensive ransomware groups 498 00:25:37,080 --> 00:25:39,000 out there on demand payments. 499 00:25:39,000 --> 00:25:42,000 I believe that's a group is no longer in place. 500 00:25:42,000 --> 00:25:43,920 I'd have to go double check that. 501 00:25:43,920 --> 00:25:46,640 But the way that it impacted their network 502 00:25:46,640 --> 00:25:50,440 was the systems out there were operational, 503 00:25:50,440 --> 00:25:52,600 but their server was down, 504 00:25:52,600 --> 00:25:56,480 which was driving their ordering software. 505 00:25:56,480 --> 00:25:58,480 They were still able to make phone calls. 506 00:25:58,480 --> 00:26:02,880 And so their order department was literally 507 00:26:02,880 --> 00:26:06,760 just writing down customers' names, credit card infos, 508 00:26:06,760 --> 00:26:09,120 whatever they wanted to order on pieces of paper 509 00:26:09,120 --> 00:26:10,920 while I was there, just trying to keep 510 00:26:10,920 --> 00:26:13,320 the business operational. 511 00:26:13,320 --> 00:26:15,160 And then once they could restore the systems, 512 00:26:15,160 --> 00:26:19,200 and they'd have to go manually enter all that data back in. 513 00:26:19,200 --> 00:26:22,040 What I found out there was that every single workstation 514 00:26:22,040 --> 00:26:23,720 in the environment was end of life, 515 00:26:23,720 --> 00:26:26,320 so it was no longer receiving security patches. 516 00:26:26,320 --> 00:26:31,000 They didn't have any type of EDR in place out there. 517 00:26:31,000 --> 00:26:34,200 Their server was a 2003 server, also end of life. 518 00:26:35,440 --> 00:26:39,320 And then as I was walking around, 519 00:26:39,320 --> 00:26:40,760 and I noticed that there was a hard drive 520 00:26:40,760 --> 00:26:42,760 plugged into the back of this, 521 00:26:42,760 --> 00:26:43,720 sorry, it wasn't even plugged in, 522 00:26:43,720 --> 00:26:46,360 it was sitting behind the server. 523 00:26:46,360 --> 00:26:49,920 And I asked them, hey, do you have backups for your server 524 00:26:49,920 --> 00:26:51,200 that we can maybe restore from this? 525 00:26:51,200 --> 00:26:52,320 And they said, yeah, we do. 526 00:26:52,320 --> 00:26:55,240 I said, is that the hard drive that was plugged 527 00:26:55,240 --> 00:26:56,600 or sitting right behind that server? 528 00:26:56,600 --> 00:26:59,240 They said, yeah, it was until we unplugged it. 529 00:26:59,240 --> 00:27:03,880 It's like, okay, let's take it onto a different system 530 00:27:03,880 --> 00:27:04,920 and see if we can get it up 531 00:27:04,920 --> 00:27:07,360 because the server was completely corrupted, 532 00:27:07,360 --> 00:27:09,160 brought it in, and I saw that ransomware 533 00:27:09,160 --> 00:27:13,480 started going down the files of their backups as well. 534 00:27:13,480 --> 00:27:16,480 And it stopped three files short 535 00:27:16,480 --> 00:27:19,400 of their entire customer database. 536 00:27:19,400 --> 00:27:22,440 And so we were able to actually restore that, 537 00:27:22,440 --> 00:27:25,520 but they said that if it had hit that file, 538 00:27:25,520 --> 00:27:27,760 they would have closed up their shop. 539 00:27:27,760 --> 00:27:32,160 But unfortunately, well, fortunately for them, 540 00:27:32,160 --> 00:27:35,080 it corrupted the server a little too quick 541 00:27:35,080 --> 00:27:36,520 and shut down the server 542 00:27:36,520 --> 00:27:39,720 before it encrypted their backup there. 543 00:27:39,720 --> 00:27:40,840 And so in that case, 544 00:27:40,840 --> 00:27:43,120 we were able to actually get the customer backup 545 00:27:43,120 --> 00:27:44,360 and operational. 546 00:27:45,280 --> 00:27:46,680 I believe that one was about two weeks 547 00:27:46,680 --> 00:27:49,800 to almost completely rebuild that environment. 548 00:27:49,800 --> 00:27:53,960 But the premise of that is you have to have 549 00:27:53,960 --> 00:27:55,280 what they call immutable backups. 550 00:27:55,280 --> 00:27:57,720 These are ones that cannot be tampered. 551 00:27:57,720 --> 00:28:01,680 And then also there's the three to one rule. 552 00:28:01,680 --> 00:28:03,560 When it comes to backups, 553 00:28:03,560 --> 00:28:06,440 and now I'm gonna totally blank on that. 554 00:28:06,440 --> 00:28:10,440 But essentially having multiple copies of your backups, 555 00:28:10,440 --> 00:28:13,760 having different locations where they're stored, 556 00:28:13,760 --> 00:28:17,560 and I should probably look that up for the details again 557 00:28:17,560 --> 00:28:19,760 on what the three to one specifics are, 558 00:28:19,760 --> 00:28:23,120 but essentially having good solid test backups, 559 00:28:23,120 --> 00:28:25,880 not a hard drive plugged into the back of your server. 560 00:28:30,640 --> 00:28:31,960 On to the next client here. 561 00:28:31,960 --> 00:28:36,960 This event was due to an unpatched exchange server. 562 00:28:38,120 --> 00:28:39,680 So publicly facing, 563 00:28:39,680 --> 00:28:41,520 running an older version of exchange. 564 00:28:43,120 --> 00:28:46,120 We had deployed EDR to their network 565 00:28:46,120 --> 00:28:47,960 because they did not have it previously. 566 00:28:47,960 --> 00:28:50,760 And we found that there were about 60 sessions 567 00:28:50,760 --> 00:28:53,560 connecting to the threat actors. 568 00:28:53,560 --> 00:28:57,120 I had the attacker slips in my mind here right now. 569 00:28:57,120 --> 00:29:02,040 In the event of this compromise, 570 00:29:02,040 --> 00:29:06,120 the attackers were able to encrypt their backup server 571 00:29:06,120 --> 00:29:07,200 as a Veeam server. 572 00:29:08,120 --> 00:29:10,360 I like that Nate brought up immutable backups here 573 00:29:10,360 --> 00:29:13,880 just a minute ago because these backups were not immutable. 574 00:29:13,880 --> 00:29:15,280 They were still locally stored. 575 00:29:15,280 --> 00:29:18,040 So the attacker on the network was able to 576 00:29:18,040 --> 00:29:20,120 encrypt all of those backups. 577 00:29:20,120 --> 00:29:24,160 It actually required some actual data recovery, 578 00:29:24,160 --> 00:29:27,440 taking the physical device to be 579 00:29:27,440 --> 00:29:30,280 manually restored down at the block level. 580 00:29:30,280 --> 00:29:33,000 It was very expensive and time consuming. 581 00:29:33,000 --> 00:29:37,680 That was truly two plus weeks before even business 582 00:29:37,680 --> 00:29:42,680 was functioning slightly close to what they were before. 583 00:29:42,840 --> 00:29:45,760 I did find what the three to one was again. 584 00:29:45,760 --> 00:29:47,520 Three copies of your backups, 585 00:29:47,520 --> 00:29:49,720 two different methods of storing it. 586 00:29:49,720 --> 00:29:53,120 So again, if it's on like a drive level 587 00:29:53,120 --> 00:29:56,080 versus cloud level, that would be two different versions 588 00:29:56,080 --> 00:29:59,040 of how it's stored and then one offsite. 589 00:29:59,040 --> 00:30:01,800 So the one that Andrew's talking about was, 590 00:30:01,800 --> 00:30:06,800 they had two copies, one form of storage, 591 00:30:07,400 --> 00:30:09,440 two locations, because there was an offsite one, 592 00:30:09,440 --> 00:30:11,400 but it was doing the same replication. 593 00:30:11,400 --> 00:30:15,080 So it wasn't following the full three to one rule. 594 00:30:15,080 --> 00:30:16,680 So sorry, Andrew. 595 00:30:16,680 --> 00:30:17,520 Yeah, good. 596 00:30:17,520 --> 00:30:20,480 That was essentially my story there for that one. 597 00:30:20,480 --> 00:30:22,320 Yeah, that was an interesting one 598 00:30:22,320 --> 00:30:24,160 because we actually had a call in forensics 599 00:30:24,160 --> 00:30:27,200 and data recovery on that one and take all their data. 600 00:30:27,200 --> 00:30:32,200 And that was the hill Mary play to try and get their data back 601 00:30:32,200 --> 00:30:36,200 and thankfully it was successful. 602 00:30:38,600 --> 00:30:39,600 Thanks for finding that. 603 00:30:39,600 --> 00:30:41,440 I was also googling on the side like, 604 00:30:41,440 --> 00:30:42,400 oh, you gotta find this. 605 00:30:42,400 --> 00:30:47,400 But yeah, Andrew, I had a couple more instant responses 606 00:30:47,400 --> 00:30:48,960 you wanted to share. 607 00:30:48,960 --> 00:30:51,880 Yeah, another instant here. 608 00:30:51,880 --> 00:30:53,880 This is by threat group called doppelpear. 609 00:30:56,040 --> 00:31:01,040 The incident started with a suspicious FedEx email. 610 00:31:01,320 --> 00:31:03,080 Now, obviously not from FedEx. 611 00:31:03,080 --> 00:31:05,680 The attackers were able to convince a user 612 00:31:05,680 --> 00:31:08,080 to open the document and from there, 613 00:31:08,080 --> 00:31:13,080 the attackers were able to compromise the entire network. 614 00:31:13,080 --> 00:31:16,960 I think it was $600,000 was the initial payment. 615 00:31:16,960 --> 00:31:19,560 And then after the time expired, 616 00:31:19,560 --> 00:31:22,600 the payment doubled up to 1.2 million. 617 00:31:24,200 --> 00:31:26,960 With this incident, there were a lot of end of life devices 618 00:31:26,960 --> 00:31:28,800 in the network and maybe more servers 619 00:31:28,800 --> 00:31:31,360 than they probably ever needed period. 620 00:31:31,360 --> 00:31:34,560 So it gave a very large attack threshold 621 00:31:34,560 --> 00:31:39,560 for the attackers to gain foothold Esquivel privilege. 622 00:31:39,760 --> 00:31:43,320 They did have RSA MFA in place, 623 00:31:43,320 --> 00:31:45,520 but that wasn't enough with the end of life servers 624 00:31:45,520 --> 00:31:47,200 that the attackers were able to compromise. 625 00:31:47,200 --> 00:31:49,600 Again, did not have EDR in place at the time. 626 00:31:52,400 --> 00:31:56,400 Now, I guess I have another story about a incident 627 00:31:56,400 --> 00:31:59,120 that was relatively new to me. 628 00:32:00,120 --> 00:32:02,320 It was still a network compromise. 629 00:32:02,320 --> 00:32:05,000 It was still a threat actor, 630 00:32:05,000 --> 00:32:07,520 but they did not encrypt the data. 631 00:32:07,520 --> 00:32:09,520 All they did was get into the network, 632 00:32:09,520 --> 00:32:11,920 ex fill data and leave a little note that says, 633 00:32:11,920 --> 00:32:14,000 hey, I got all your data, pay me. 634 00:32:14,000 --> 00:32:19,000 It's interesting, I guess. 635 00:32:19,600 --> 00:32:21,440 I get what the thought process is. 636 00:32:21,440 --> 00:32:24,000 They're not ending business because business ends, 637 00:32:24,000 --> 00:32:25,520 you don't get paid. 638 00:32:25,520 --> 00:32:26,640 They can still function, 639 00:32:26,640 --> 00:32:30,640 but it was definitely an interesting attack 640 00:32:33,200 --> 00:32:37,200 that was due to weak password policies, 641 00:32:37,200 --> 00:32:39,040 especially a legacy account 642 00:32:39,040 --> 00:32:43,200 that password was set for it many years ago. 643 00:32:43,200 --> 00:32:45,200 It was used for many services. 644 00:32:45,200 --> 00:32:49,200 The pain to change that password was an assumed risk 645 00:32:49,200 --> 00:32:50,400 because they didn't want to have to go through 646 00:32:50,400 --> 00:32:52,800 all this other stuff to update it. 647 00:32:52,800 --> 00:32:56,000 That was a difficult one to deal with. 648 00:32:56,000 --> 00:32:58,000 I didn't hear the end of that one, 649 00:32:58,000 --> 00:33:00,000 the internal IT handle, the rest of it, 650 00:33:00,000 --> 00:33:04,400 but it was definitely an interesting one. 651 00:33:04,400 --> 00:33:07,200 I think, Nate, I'll let you handle this very last one. 652 00:33:07,200 --> 00:33:09,600 Yeah, the one thing that I did want to add on 653 00:33:09,600 --> 00:33:15,600 about the threat groups that don't always encrypt data, 654 00:33:15,600 --> 00:33:17,600 even if they do encrypt data, 655 00:33:17,600 --> 00:33:20,400 the tactics of these threat actors change over time. 656 00:33:20,400 --> 00:33:22,400 It's really been interesting to see this 657 00:33:22,400 --> 00:33:25,200 where typically they would just encrypt the data, 658 00:33:25,200 --> 00:33:28,400 say give us money, and then that was the end of it. 659 00:33:28,400 --> 00:33:32,400 Then they switched to what they call double extortion, 660 00:33:32,400 --> 00:33:35,600 was they would take the data and then encrypt the data, 661 00:33:35,600 --> 00:33:37,200 and then if you didn't pay, they'd say, 662 00:33:37,200 --> 00:33:40,800 well, we still have your data that we can potentially leak. 663 00:33:40,800 --> 00:33:44,800 I think org-8 or org-3 that I mentioned up above 664 00:33:44,800 --> 00:33:48,800 was one that they had a lot of sense of data. 665 00:33:48,800 --> 00:33:54,000 They did pay to prevent it from being leaked further, 666 00:33:54,000 --> 00:33:57,600 but then they used those backups to become operational. 667 00:33:57,600 --> 00:34:00,400 Those are the ones where that double extortion 668 00:34:00,400 --> 00:34:02,800 does tend to sometimes work. 669 00:34:02,800 --> 00:34:04,400 One of the other things that we've seen, 670 00:34:04,400 --> 00:34:06,400 and so there's been threat groups like 671 00:34:06,400 --> 00:34:10,000 Tommy Leakes, Kara Kurt, anything like that, 672 00:34:10,000 --> 00:34:12,400 where they don't encrypt the data, but they take it. 673 00:34:12,400 --> 00:34:15,400 But then what is really interesting about these ones 674 00:34:15,400 --> 00:34:18,200 is they don't just tell you that you have the data. 675 00:34:18,200 --> 00:34:21,200 They'll start calling your employees and say, 676 00:34:21,200 --> 00:34:26,000 tell your IT team and the business leaders to make a payment 677 00:34:26,000 --> 00:34:29,200 because we already know that we have your data. 678 00:34:29,200 --> 00:34:31,600 They're ignoring it because that's the other thing. 679 00:34:31,600 --> 00:34:33,400 They monitor the chat logs, 680 00:34:33,400 --> 00:34:36,000 and so if you're hopping in and out of that portal, 681 00:34:36,000 --> 00:34:37,600 trying to see if there's any updates, 682 00:34:37,600 --> 00:34:40,400 they see every single time that you're in there, 683 00:34:40,400 --> 00:34:45,600 which then prevents better negotiation tactics. 684 00:34:45,600 --> 00:34:49,000 And so one of the things there is we've also seen it 685 00:34:49,000 --> 00:34:54,000 where they have obtained a customer list of that in business, 686 00:34:54,000 --> 00:34:56,100 then they started calling the customers 687 00:34:56,100 --> 00:34:58,600 or emailing the customers saying, 688 00:34:58,600 --> 00:35:03,400 go tell your vendor or partner to pay us 689 00:35:03,400 --> 00:35:05,100 because we have your data 690 00:35:05,100 --> 00:35:07,400 and they're not being responsible 691 00:35:07,400 --> 00:35:09,200 to prevent that from being leaked. 692 00:35:09,200 --> 00:35:13,900 So now you have your customers calling you saying, 693 00:35:13,900 --> 00:35:15,200 what happened? 694 00:35:15,200 --> 00:35:19,000 You know, the cat, is that the cat? 695 00:35:19,000 --> 00:35:19,800 Is that one of the bag? 696 00:35:19,800 --> 00:35:25,400 I forgot the saying there, but yeah, the cat's on the bag there. 697 00:35:25,400 --> 00:35:30,600 And so there is a lot of new techniques being done there. 698 00:35:30,600 --> 00:35:34,800 And then the last thing I'd say is that there was a bank 699 00:35:34,800 --> 00:35:38,400 that we ended up working with. 700 00:35:38,400 --> 00:35:40,600 I won't go into too much of the details, 701 00:35:40,600 --> 00:35:42,600 but the biggest thing was this customer, 702 00:35:42,600 --> 00:35:45,400 although they didn't have EDR in place, 703 00:35:45,400 --> 00:35:49,400 they had everything else done very, very well. 704 00:35:49,400 --> 00:35:54,200 And so we identified that there was ransomware 705 00:35:54,200 --> 00:35:58,200 in this environment at BLEEVIO's 5 AM, 706 00:35:58,200 --> 00:36:03,200 and then we had the entire environment operational 707 00:36:03,200 --> 00:36:05,700 by 7 AM. 708 00:36:05,700 --> 00:36:08,500 So when the first customer walked through the front doors 709 00:36:08,500 --> 00:36:10,600 of that bank, they could do business. 710 00:36:10,600 --> 00:36:13,000 Customers never suspected a thing, 711 00:36:13,000 --> 00:36:17,200 but that's just how I wanted to kind of wrap that up 712 00:36:17,200 --> 00:36:21,000 and say, when you don't have something like multi-factor 713 00:36:21,000 --> 00:36:23,800 or patch devices or EDR in place, 714 00:36:23,800 --> 00:36:29,800 as of fallback, making sure that you have the proper backups 715 00:36:29,800 --> 00:36:34,800 tested to ensure that you can do a full recovery in the timelines, 716 00:36:34,800 --> 00:36:38,100 restoring the same amount of data that is acceptable of loss 717 00:36:38,100 --> 00:36:43,500 to you within, again, everything is adequate to the business 718 00:36:43,500 --> 00:36:47,400 needs, because in that case, ransomware was a minor issue 719 00:36:47,400 --> 00:36:48,700 out there. 720 00:36:48,700 --> 00:36:50,800 It was just, yep, we have ransomware two hours later, 721 00:36:50,800 --> 00:36:53,500 we're back up and now we'll go deal with the residual effects 722 00:36:53,500 --> 00:36:54,600 of that. 723 00:36:54,600 --> 00:36:55,800 But no one knew about it. 724 00:36:55,800 --> 00:36:57,300 It didn't impact their business. 725 00:36:57,300 --> 00:36:59,500 They had no financial loss other than, you know, 726 00:36:59,500 --> 00:37:02,000 having to do their cybersecurity insurance claim, 727 00:37:02,000 --> 00:37:04,200 but that was the extent of it. 728 00:37:04,200 --> 00:37:06,400 So. 729 00:37:06,400 --> 00:37:08,100 I'd like to also add something to that, 730 00:37:08,100 --> 00:37:10,700 is that you can do everything perfectly. 731 00:37:10,700 --> 00:37:14,500 You can spend, you know, lots of money on all these tools 732 00:37:14,500 --> 00:37:15,300 and layer defenses. 733 00:37:15,300 --> 00:37:18,400 It's very important to do, but it does take one week link 734 00:37:18,400 --> 00:37:21,600 that the attackers can exploit. 735 00:37:21,600 --> 00:37:24,200 You know, you have to protect everything. 736 00:37:24,200 --> 00:37:27,500 The attacker just has to get one thing right to be able 737 00:37:27,500 --> 00:37:29,800 to compromise you. 738 00:37:29,800 --> 00:37:33,500 The goal is to make the attackers job difficult that they'll go 739 00:37:33,500 --> 00:37:34,400 to somebody else. 740 00:37:34,400 --> 00:37:37,800 They don't want to spend 45 hours trying to figure out how 741 00:37:37,800 --> 00:37:39,500 to get into your network where they could go to some guy 742 00:37:39,500 --> 00:37:41,900 down the street and spend 15 minutes and then get the same 743 00:37:41,900 --> 00:37:44,000 amount of money. 744 00:37:44,000 --> 00:37:45,700 So, you know, you can do a lot of things right, 745 00:37:45,700 --> 00:37:49,700 but it does take a lot of layering your defenses and making 746 00:37:49,700 --> 00:37:53,200 sure that the right tools and people and things are in place 747 00:37:53,200 --> 00:37:56,400 to help prevent it, because if a security person ever tells 748 00:37:56,400 --> 00:37:59,900 you you are 100% safe, they're lying to you because no one is 749 00:37:59,900 --> 00:38:04,800 100% safe at any point in time. 750 00:38:04,800 --> 00:38:08,500 So, on that vein to kind of wrap us up, 751 00:38:08,500 --> 00:38:10,600 so you've convinced me, I'm an IT admin. 752 00:38:10,600 --> 00:38:13,000 I need all of these things. 753 00:38:13,000 --> 00:38:18,100 How do I start that conversation with my C levels? 754 00:38:18,100 --> 00:38:23,800 I mean, going in and asking, free tools are great and maybe 755 00:38:23,800 --> 00:38:25,300 there are some out there, but you know, 756 00:38:25,300 --> 00:38:29,200 sometimes you need to pay for those tools. 757 00:38:29,200 --> 00:38:32,600 How do I start that conversation? 758 00:38:32,600 --> 00:38:35,600 Yeah, one of the big things is just talking risk to the 759 00:38:35,600 --> 00:38:39,500 business is security. 760 00:38:39,500 --> 00:38:43,000 You know, we're not here to say no to everything, right? 761 00:38:43,000 --> 00:38:47,600 We're trying to help just do risk reduction to organizations. 762 00:38:47,600 --> 00:38:50,800 That's everything that cybersecurity is about. 763 00:38:50,800 --> 00:38:56,100 And so, if there is some type of risk to the organization of 764 00:38:56,100 --> 00:39:00,100 saying, we don't have great backups today, 765 00:39:00,100 --> 00:39:03,700 so what is the impact if we lost data today? 766 00:39:03,700 --> 00:39:07,400 You know, maybe our backup server completely crashed and we 767 00:39:07,400 --> 00:39:09,600 weren't able to restore data. 768 00:39:09,600 --> 00:39:11,100 What does that cost the business? 769 00:39:11,100 --> 00:39:13,600 Is it $20,000? 770 00:39:13,600 --> 00:39:15,500 Is it $100,000? 771 00:39:15,500 --> 00:39:20,200 Whatever it is for all that data that you lost. 772 00:39:20,200 --> 00:39:21,400 That's one thing. 773 00:39:21,400 --> 00:39:26,300 Now, if we said we have solid backups, 774 00:39:26,300 --> 00:39:30,500 but it's going to take me two weeks to pull all this data 775 00:39:30,500 --> 00:39:33,400 from, you know, tape drives or something like that because 776 00:39:33,400 --> 00:39:36,200 we're still using a legacy version of it. 777 00:39:36,200 --> 00:39:39,700 What does your business make within two weeks? 778 00:39:39,700 --> 00:39:43,000 Calculate that and say, that's how long we would be down 779 00:39:43,000 --> 00:39:45,900 and not generating revenue. 780 00:39:45,900 --> 00:39:50,400 Now, that might be $50,000 a day. 781 00:39:50,400 --> 00:39:53,400 It could be, in some cases, we know customers that are 782 00:39:53,400 --> 00:39:56,600 generating $1 million a day of revenue, right? 783 00:39:56,600 --> 00:40:00,400 So, that very, very quickly will pay for your backup solution 784 00:40:00,400 --> 00:40:01,500 there. 785 00:40:01,500 --> 00:40:03,600 Same things with EDR, right? 786 00:40:03,600 --> 00:40:09,500 Is if we know that these are so successful at preventing 787 00:40:09,500 --> 00:40:12,900 things like malware and, you know, everything like that, 788 00:40:12,900 --> 00:40:15,700 taking what, again, whatever that downtime is 789 00:40:15,700 --> 00:40:18,200 for some type of security incident and whatever the cost 790 00:40:18,200 --> 00:40:21,400 is to go bring in a third-party forensics, third-party 791 00:40:21,400 --> 00:40:25,100 incident response firm, that'll very, very quickly start paying 792 00:40:25,100 --> 00:40:27,800 for those tools. 793 00:40:27,800 --> 00:40:31,700 But taking everything and correlating it back to the risk 794 00:40:31,700 --> 00:40:34,300 and operations of the business is critical. 795 00:40:34,300 --> 00:40:37,800 It's not just going to them and saying, please give me another 796 00:40:37,800 --> 00:40:43,800 $3,000 or whatever it is for the solution. 797 00:40:43,800 --> 00:40:46,400 Show them that you're trying to help keep the business 798 00:40:46,400 --> 00:40:51,200 operational and moving forward. 799 00:40:51,200 --> 00:40:53,000 Nate did hit on it quite a bit. 800 00:40:53,000 --> 00:40:55,100 I would say putting it into dollars and cents 801 00:40:55,100 --> 00:41:00,000 because to a C-level person, they don't care that, you know, 802 00:41:00,000 --> 00:41:02,100 all these bits and pieces of all these ways the attackers 803 00:41:02,100 --> 00:41:02,800 can get in. 804 00:41:02,800 --> 00:41:04,800 You know, if you put in that perspective of you're down 805 00:41:04,800 --> 00:41:07,400 for two weeks, how much money you're losing, 806 00:41:07,400 --> 00:41:10,800 this tool is one 1,000th that price. 807 00:41:10,800 --> 00:41:13,500 Let's talk about it. 808 00:41:13,500 --> 00:41:21,600 The thing our CISO likes to say is the boiling frog analogy, 809 00:41:21,600 --> 00:41:22,000 I guess. 810 00:41:22,000 --> 00:41:24,800 It's, you know, you don't throw a frog in boiling water. 811 00:41:24,800 --> 00:41:26,600 You put them in the pot and you turn the heat up. 812 00:41:26,600 --> 00:41:28,100 So let's get the MFA. 813 00:41:28,100 --> 00:41:28,900 Let's get the EDR. 814 00:41:28,900 --> 00:41:29,700 Let's put those in place. 815 00:41:29,700 --> 00:41:31,100 Make sure you're in and you're good. 816 00:41:31,100 --> 00:41:33,500 And then slowly work your tools upwards. 817 00:41:33,500 --> 00:41:36,700 I wouldn't say to buy all the tools, all the whiz bangs as 818 00:41:36,700 --> 00:41:40,400 fast as possible because you're not going to know everything 819 00:41:40,400 --> 00:41:41,100 about every tool. 820 00:41:41,100 --> 00:41:42,600 You're not going to make sure everything's all in place 821 00:41:42,600 --> 00:41:44,600 because you have all the things you're dealing with. 822 00:41:44,600 --> 00:41:50,100 Make sure that you have the best thing for your dollar 823 00:41:50,100 --> 00:41:53,400 and make sure that you have it well configured before you 824 00:41:53,400 --> 00:41:56,400 move on to something new. 825 00:41:56,400 --> 00:41:57,300 Yeah. 826 00:41:57,300 --> 00:42:01,600 This is, this will be as you're wanting to be a little bit 827 00:42:01,600 --> 00:42:04,000 more mature on the risk conversations, 828 00:42:04,000 --> 00:42:06,300 but there is something called FAIR. 829 00:42:06,300 --> 00:42:07,800 It's a FAIR model. 830 00:42:07,800 --> 00:42:12,100 And what it does is it takes it, takes risk and tries to 831 00:42:12,100 --> 00:42:17,100 quantify that whatever the total loss is. 832 00:42:17,100 --> 00:42:21,500 And then you can start helping take that model to talk about, 833 00:42:21,500 --> 00:42:22,600 you know, what's the cost? 834 00:42:22,600 --> 00:42:25,100 What's the impact? 835 00:42:25,100 --> 00:42:28,300 What's the likelihood of these different security threats 836 00:42:28,300 --> 00:42:31,900 and then help, again, quantify and better communicate that 837 00:42:31,900 --> 00:42:33,000 to the business? 838 00:42:33,000 --> 00:42:33,800 Well, thank you both. 839 00:42:33,800 --> 00:42:37,200 Thank you, Dean and Andrew, for joining us today. 840 00:42:37,200 --> 00:42:40,400 If you liked this podcast, please like, subscribe, 841 00:42:40,400 --> 00:42:41,200 reach out to us. 842 00:42:41,200 --> 00:42:44,900 If you have a question or topic you'd like us to discuss, 843 00:42:44,900 --> 00:42:49,800 you can email us at info at cit-net.com or head out to our 844 00:42:49,800 --> 00:42:54,100 website, cit-net.com slash podcast. 845 00:42:54,100 --> 00:43:11,800 And we'll be back next week with an all new episode.