Welcome to the Azure Security Podcast,
where we discuss topics relating to security, privacy,
reliability, and compliance on the Microsoft Cloud Platform.
Hey everybody, welcome to episode 56.
This week is another light week,
it's only myself, Gladys,
and Mark and Sarah are all on vacation.
But this week, we also have a guest,
Michael Malone, who's here to talk to us about
Microsoft Defender Advanced Hunting.
Before we get to Michael though,
I do have a few little news items.
Azure Advisor for MySQL has now just gone in preview,
and it allows for basically Azure Analysis,
giving you things like performance tips,
and giving you security ideas,
and just some advisory notes about
securing and improving the cost efficiency,
and the efficiency in general of
your products in this case, MySQL.
We've now also added the ability to
support custom certificate authorities
in Azure Kubernetes Service in AKS.
This is really cool because that way,
if you decide to create your own certificate authority,
you can now install your own root certificates in there,
and then use them just as you would do.
Say, I don't know, GoDaddy or
a various sign root certificate.
So that's another nice thing to see,
and I know a lot of customers have been asking for that.
Azure Application Gateway now supports
private link that's in preview.
I've talked about private link,
I think just about every single podcast.
More and more past services are
supporting private links so that way you can have
private IP addresses, private DNS names,
and essentially having traffic flow between
two past services without going over the public internet.
So that is Azure Application Gateway.
We now also have in public preview,
continuous backup enhancements in Azure Cosmos DB.
I don't pretend to be an expert in backing up Azure Cosmos DB,
but having some enhancement there is probably a good thing.
We've also now added to API management,
content security policy,
and core configuration support.
So core is cross origin resource sharing.
It is a critically important defense in browsers.
It allows you to share resources across
domains without violating the browser security policy.
You got to be very careful there though.
Don't go putting a star as that wild card to say,
I support absolutely everything from absolutely everywhere.
That defeats the whole purpose of course,
but it's great to see that because now you can actually
set it both in the actual portal
and you can also do it declaratively.
So that's the news that I've got this week.
Let's turn our attention to our guest.
This week, as I mentioned, we have Michael Malone,
who's here to talk to us about Microsoft Defender,
Advanced Hunting, and KQL.
Michael, welcome to the podcast.
Would you care to take a moment and give our listeners
a little bit of background on who you are and what you do?
Thanks for having me, Michael.
So yeah, I'm currently on the Defender 365
customer experience team.
So I work helping large customers on board
and operationalize the Microsoft 365 Defender XDR Suite.
So XDR is an expanded detection and response capability
for that's helps customers out finding suspicious
and malicious activities at their enterprise.
It's kind of like an upgrade, if you will,
from a traditional antivirus.
Prior to being on this team though,
I spent about seven and a half years with Microsoft Dart.
So essentially what we would do is we would go investigate,
determine human adversary or human operated type attacks.
So when you had a human at the other side of the keyboard,
it's not just purely a malware problem, if you will.
And actually, I heard you had a new book coming out
on Azure Security, Michael.
Yeah, I've been talking a little bit about that.
I'm not kidding, we're actually literally four or five pages
away from actually having the drafts completed.
So we're really excited for that.
It should be out in November.
And it's designing and developing secure Azure solutions.
Yeah, I actually just put out a book last year.
It's called Designing Secure Systems, which is kind of a
theoretical based approach on system security overall.
So it's like, essentially, if you can imagine
a unified model that lets you look at physical human process
as well as cyber systems and how a vulnerability in one area
can lead to an issue in another.
So the topic of this podcast is advanced hunting.
So the question is gonna be,
so what is advanced hunting and what does it sort of entail?
Absolutely, so advanced hunting is kind of like a roll your own
detection capability inside of Defender.
So Defender, as I mentioned, is an XDR.
It's essentially a capability to identify not just malicious
but suspicious activity.
And the reason why I make this clarification,
traditionally, when you looked at security products,
you had antivirus.
And antivirus is tuned to specifically find known bad things.
It's very difficult.
The way I like to describe it is,
from an antivirus perspective,
if you clean up a legitimate file,
you're likely to make news.
Whereas if you miss a bad file,
you're just a bad antivirus solution.
EDR and XDR kind of helps you cover that gap
because we are essentially tuning to avoid false positives.
If you have a false positive,
you can take down a customer's enterprise.
So EDR enables us to identify suspicious activities
that may not necessarily have inherently malicious files
involved.
So for example, if an attacker was to perform lateral traversal,
depending on the tooling they use,
they may or may not use malware.
They also may be performing
some other suspicious administrative activities
during the attack.
So EDR covers that gap.
EDR being endpoint detection response.
And then more broadly,
so about two years ago now,
we launched Microsoft Threat Protection, or MTP,
which eventually became Microsoft 365 Defender,
which is our XDR, as I mentioned,
it's the expanded detection response.
And the big difference between EDR and XDR is the breadth.
So in an XDR solution,
you're not just looking at endpoints,
you're typically looking also at things like identities.
So Microsoft Defender for Identity is our UEBA
and also Identity Detection Solution.
There's Defender for Office,
which is looking at suspicious
and malicious email activities going on.
And Defender for Cloud Apps,
which is looking for shadow IT
and suspicious use of cloud applications.
So Defender Advanced Hunting gives you the ability
to do raw access hunting to the data we use inside of Defender
to create detections.
So we create a whole bunch of detections out of the box
for things that we know are suspicious or malicious,
but sometimes you may have something
that's contextually unique to your organization.
For example, remote administration tools
are not inherently malicious,
but you may know inside your organization,
you only use one product or another,
you might only use remote desktop to manage these systems.
And as a result, you wanna identify things
that are not remote desktop
that enable you to control those systems.
With Advanced Hunting, you can create these queries
that'll identify when these processes have been created
or when files were written or activities occurred
on these devices
and also enable you to create custom detections.
So if your query is really good
at identifying suspicious activity,
you can have Defender automatically respond
by isolating the device, quarantining the file.
It basically gives you raw data access
to create your own customized detections for your enterprise.
So detections is one of the things
you can do with Advanced Hunting.
There's also a whole bunch of other capabilities
that we use as well.
So customers sometimes wanna have kind of a picture
of their entire enterprise,
i.e. find out where this particular software package
is installed, see where this individual user logs on.
So it gives you this kind of cool reporting capability
as well.
Since we have all these process creations and logon events
and network communication activities
and file creations, et cetera, emails and such,
you have the ability to kind of create your own report
of what goes on in your environment.
So you may be able to do your own baselining.
You can also identify anomaly-based activity as well.
Another thing is from on the baselining front,
if you get out of the box and what Defender really is,
you can do some neat things around, for example,
we have queries that'll help you design
your app blocker policy.
If you're not sure where, like what the impact
of a particular app blocker or a WDAC policy is gonna be,
you can very quickly report on where,
what processes would launch from that particular path
that you're allowing or declining,
or you can also use it as a mechanism to figure out
how many different executables
and how many different devices have things launching
from that.
So it helps you kind of tune app blocker, WDAC,
or like firewall policies is another great example.
You can see where inbound and outbound communication
happens inside the enterprise.
And when you wanna determine the impact of allowing
or blocking a specific IP address or application,
it'll help you get that picture of what's going on
inside your enterprise.
So you know it before you actually implement it.
Hey, what's WDAC?
WDAC or Windows Defender app control enables you
to build a policy that can define what applications are
or are not allowed to run on a given endpoint,
a given Windows endpoint.
For example, you can say files that are signed
by a specific certificate are allowed to run,
or things from a specific path are not allowed to run.
You can also use it for an auditing capability as well.
For example, you can say if a process launches
in this particular context,
I wanna log this and see it
from your Defender Advanced Hunting Console.
So this is kind of interesting, right?
So you could potentially use it for sort of alluding
to this for more than just detection, right?
Because you could actually just run a normal clean system,
a known clean system,
and get an idea for which applications are running,
where they're running from,
if they're digitally signed, which hopefully they are,
what network ports and protocols are being used.
And you could actually build that up
to be like a list of what good looks like.
Is that a reasonable way of putting it,
or is that just way too simplistic?
So that's one approach you can use.
You can definitely, you can also,
it's worth noting there's,
because we're using KQL or Kusto query language
under the hood,
there's also the ability to bring in external data.
So if you have a data set that has like, for example,
systems like maybe IP address ranges,
or list of file hashes or signers or whatnot,
you can actually dynamically import it into your queries.
So there's definitely one way of doing it,
of basically saying,
this is a known clean box,
and this is what its profile looks like.
But you can also, a lot of times,
enterprises are very complex.
There's gonna be some sort of drift
that's gonna happen across the board.
So what we can do with advanced hunting
is build a picture using summarizations and pivots
and joins and such,
that tells us dynamically
what your enterprise looks like today.
The interesting thing there is,
you can also dig into those outliers.
So that long tail that everybody talks about
in statistics is a very real thing in an enterprise.
But it's also where a lot of the really interesting
aspects of your enterprise are.
These are where your attacker tools might be, for example.
Or this might be somebody who's just violating policy
for one reason or another.
It gives you this really cool ability to dive into those
and rationalize and figure out what kind of policies
you'd like to create with tools like WDAC,
or ultimately, if you wanna create a custom detection
to respond to such a thing.
Can I get pre-built hunting queries,
or do I have to start from scratch?
I mean, let me put it another way.
So I'm not really an expert
in threat hunting by any stretch.
I can't imagine as a little API I can run that says,
show me how I got whacked.
Show me bad things that are happening.
I mean, because I don't know what I'm looking for.
I'm gonna be honest with you,
I don't know what I'm looking for.
So do we have built-in or sample queries
that people can use that might show
sort of indicators of either malicious behavior
or even indicators of compromise?
So there are, there's a couple
of really great resources out there.
So the first piece of advanced hunting
is understanding KQL itself for Kusto query language.
We've got the Kusto query language referenced.
So this is gonna be the same query language
using Sentinel if you've used it.
It's also what you're gonna use
on what's called an Azure Data Explorer cluster.
So if you've got experience with those, you're using KQL.
But there's also really great web-based doc
that references the entire language.
It's pretty easy to get used to.
I saw a shirt the other day that actually says,
KQL is a new PowerShell and invite me stop and think.
It really is across our security stack.
It's really the key language you need to understand.
And as well as, as I mentioned,
it's really great for if you have
an Azure Data Explorer cluster.
So the second thing is understanding the data landscape
that we've gotten advanced hunting.
Depending on which products you've got from Defender,
you may see information from devices, emails,
apps and identities, et cetera.
So understanding those is really important.
And for that, we've got our schema reference.
There's a public face,
there's some public facing docs that are out there on it.
But ultimately my favorite spot to go,
if you open up the advanced hunting page in Defender
in the upper right-hand corner,
there's a button for schema reference
that breaks down every single table
and every single column
and provides some really great detail.
One that I would like to highlight,
especially if you're a Defender for Endpoint customer,
is the device events table.
We have a lot of really good information in there.
For example, scheduled task creations,
plug and play device activity.
So if you wanna know if somebody's plugging in a USB drive,
you can see that, you can see volume mounts,
a whole bunch of really good information
for creating your own custom detections
or researching activity inside your environment.
And if you wanna get started with advanced hunting,
there's a really great series I put out about two years ago
with Taliash called tracking the adversary
with MTP advanced hunting.
We basically start off with very one-on-one approach.
So we cover the basic operators,
like take, for example, the where clause, et cetera.
And we cover and cover each of the individual datasets
that we've got advanced hunting at that time.
And ultimately build their ways up to things like joins.
So there's all different kinds of joins
inside of KQL they're important to understand.
And last, we start moving into summarizing,
pivoting and joining.
So essentially, if you wanna build these reports
or summarizations of what's going on inside your data.
And the very last episode we dig into a contrived hunt.
So we've gotten a targeted adversary situation
we replicated inside of a defender tenant.
And we go through and use the things that you learned
in those different episodes to hunt down the adversary
and figure out all of its attributes.
So one thing I mentioned at the beginning
that really sort of piqued my interest
is he talks about tracking down human adversaries.
Is there any sort of tips, tricks?
Is there a format or structure that you go through
to help with that?
So one of the tools I like to use
is I like to call it the ABCs of instant response
or the ABCs of security.
Essentially the ABCs represent the things
that you wanna look for in any target attack
or really any cyber situation you run into.
So to get started, so the ABCs are authentication,
backdoors, communication channels and data.
So authentication represents the identity aspect.
So what identity did the attacker use
when they connected to the service?
After they contacted this particular service,
what identity were you using on the device itself?
So for example, if it's a web server,
you might be anonymous coming in,
but if there's a vulnerability in that web page,
you might be able to run code as the web server.
In that case, the identity you'd be looking for
is the identity of the IS, Apple
or perhaps the Apache service.
The third piece of identity is what identities were compromised
as a result of that event.
Now you've got on the box,
do we have credentials that are exposed to memory
and are you running as admin?
Is there perhaps a password file or something on those lines?
What types of authentication does the attacker now have access to
as a result of that attack?
The second one is backdoors.
Now, backdoor is a malware term,
but ultimately it's the mechanism that the attacker uses
to control the endpoint itself.
So a backdoor can be a perfectly legitimate tool
and an intended capability.
And our web server, the backdoor,
the initial backdoor they connected to
is essentially the web server itself.
It could also be an exposed RDP port
or it could be a piece of malware.
If they've already infected the device,
you might have a remote access Trojan
or something along those lines on the device
that enables them to control the system.
So just like authentication, there's three time frames.
You've got the initial access,
so what backdoor did they use to control the endpoint itself?
Was there any backdoors installed as a result of the event?
So for example, did we see a downloader or a dropper
install a backdoor into the system?
And as a result, what backdoors do they have access to
as a result of the attack?
So now you've compromised this device.
Presumably you can use a device to pivot to other devices
that were protected by a firewall
or a security mechanism.
So it's essentially the mechanism for control.
The third one is communication channels.
And that essentially describes the way
that the attacker communicates with a device.
It's the path between the attacker and the backdoor.
So in our initial access scenario,
when the attacker contacted the web server,
it might be a user agent string or a source IP address.
It could also be more generic things such as countries
or ISPs or things along those lines.
These are all things you can use to profile
that attacker activity inside the enterprise.
You also have, of course, the post breach scenario
where what communication paths are open as a result.
And last, you've got data.
So data, it really represents the impact.
It's the confidentiality, integrity, and or availability
that you've lost of information as a result of the breach.
So if this is a ransomware or a wiper attack,
some of your data may have been encrypted or destroyed.
Or if it's a tamper attack,
you might have some additional rows in your database
you didn't intend.
Or last, if it's a data theft
or intellectual property theft type case,
it's what data essentially went out the front door
as a result.
Did we lose some intellectual property?
Some secrets or sensitive organizational information?
So at the start of this, you mentioned using KQL
to perform the hunting.
Is that the only way you can do hunting?
I mean, there are other APIs or anything that you can call
or you really restrict is just using KQL.
So there's two main ways you can really use
advanced hunting inside of Defender itself.
So the first way is to use the web interface itself.
That's probably the easiest way,
but there is some limitations.
You have a cap of about 10,000 rows.
So if you need to bring back a whole bunch of data,
you may want to use the API.
So the API will give you up to 100,000 rows
worth of results and enables you to essentially
programmatically call advanced hunting.
It's really great for hooking to things like, for example,
if you have some Azure automation out there,
you can pull your data directly into reports or whatnot.
The third way you can kind of use,
and this is sort of advanced hunting,
is really referring to the data itself.
So we've got a couple of different APIs out there
that enable you to pull the data into other systems.
So the first one is going to be your Sentinel connector,
which is a native connector between Defender
and Microsoft Sentinel.
So you can actually pull your data into Sentinel
and then join it with all the log data
that's in for instance at your SIM.
Or you can also use the streaming API,
which lets you pull it into either directly
into blob storage, or you can pull it into an event hub
and then do it with what you like.
I'm not gonna be right, I mean, I love KQL,
but I always like the ability to call an API,
just in case.
I mean, I want to build like some kind of custom tooling
or something, and it just gives me
that extra level of flexibility.
So that's really great to see.
I mean, is there a community of folks out there?
Cause I imagine this stuff's relatively complex,
especially if you're sort of learning this stuff
from the get go.
Is there other people I can talk to about this?
We also have a get hub.
So we're actually inside the Azure Sentinel repo.
If you look under the hunting queries,
you're gonna see Microsoft 365 Defender.
And inside there is a whole bunch of YAML formatted queries,
which is gonna be the same ones you see
in the advanced hunting portal under community.
So it'll help you get started if you want to see
how other people are using advanced hunting today.
All right, so one thing we ask all our guests
is if you had one sort of final thought
to leave our listeners with, what would it be?
So probably the biggest thing is
when you're looking through your EDR or XDR solution,
you see activity, make sure you stop
and determine if it's gonna be commodity stuff.
So the stuff you get just surfing around the web,
or if there's might be some targeted intent behind it,
i.e. if you're looking through seeing
some suspicious activity.
If you do see targeted activity, remember your ABCs,
authentication methods, backdoors,
communication channels and data.
And be ready to bring in help if you need it,
like from Microsoft Dart
or Microsoft Defender Experts for hunting.
And if you're looking for resources for hunting,
check out my book, Designing Secure Systems.
It's got a lot of good content in there
that'll help you think like a hunter
and help you track this adversary using the ABCs
and what I like to call authorization theory.
Well, thanks again for joining us this week, Michael.
I know it's an interesting topic.
I admit it's not an area that I'm particularly familiar with,
so it was always good to learn something new.
And to our listeners out there,
thank you to you also for listening in.
Stay safe and we'll see you next time.
Thanks for listening to the Azure Security Podcast.
You can find show notes and other resources at our website,
azsecuritypodcast.net.
If you have any questions,
please find us on Twitter at azuresecpod.
Background music is from ccmixter.com
and licensed under the Creative Commons license.