Today we're going to compare three different
models inside a Copilot agent built in Copilot
Studio. We're going to compare OpenAI's latest
model to Anthropix cloud model and to Grok which
is built by XAI. Now we're going to keep every
variable the same except just change the model
so it'll give us a good indication of how powerful
the model is. So right now you're looking at
my copilot studio environment and this is an
agent that i actually built in the past it's
called matchmaker and it's basically an autonomous
agent that receives emails and it looks at emails
that are related to job postings it matches the
candidate that fits that job posting the best
it ranks all the different candidates and then
it generates a document and emails it back to
the sender it's a super cool agent and if you're
interested in building this check out the previous
video that actually goes through this step by
step in today's video we're just interested in
this section here which is selecting the model
and when i open up all these different models
that i have access to from gpt41 to 5 chat and
5 auto the latest one that i actually have access
to is gpt 5 3 for chat that just came out and
on the anthropic side you can see that i have
cloud sonnet 4 5 cloud sonnet 4 6 and opus 4
6. opus seems to be for deep reasoning and i'm
going to use sonnet 4 6 which is for general
tasks because This is also for most tasks, the
5 -3. So I'm going to kind of do apples to apples
instead of using the deep reasoning models. For
example, the 5 -2 deep reasoning, I'm not going
to use that. And then I'm going to also compare
it to this XAI Grok 4 -1, which actually says
deep reasoning, but it's the only choice that
I have for XAI. So in summary, there's going
to be three models. But if you look at this agent,
this is the instructions. and i modified it a
little bit from the initial matchmaker build
that we had if we look at the instructions basically
a new email arrives and the agent determines
if it's related to a job posting or a hiring
type of a question and then it compares the job
description to the candidates that are on the
hiring bench in a sharepoint site that sharepoint
site is actually right here in the knowledge
sources and it's actually this is the sharepoint
site And when I go to the hiring bench, what
Arnold has done, we're looking at Arnold's SharePoint.
He's taken the resumes of a few people like Jessica,
Jordan, and he's put them on the bench because
he doesn't have an open rec to fill. But if you
look at Jessica, Arnold is hiring for product
manager types of roles. Back to the agent, you
got the knowledge section and the instructions
say, look at. the job description and compare
them to the hiring bench. Also use all available
resources on the web to research ideal qualities
that a candidate should possess to be successful
in this type of a role. Use this information
in your analysis. In this first example you can
see that I've highlighted GPT -53 chat experimental
from the list. And so let's go ahead and fire
off our first email. I'm going to open up my
Outlook and Albert is ready to fire off an email
to Arnold saying, hey, do you know anyone? Do
you know anyone that fits this job role? We just
got to drop the link in right over here. So I'm
going to go ahead and go to the Microsoft career
site. I'm just going to grab this link here and
go back to the email that Albert is creating
and just drop it in here. now we're going to
send this off okay so the email just came in
it's 318 so it didn't take that long and you
can see here this is the response and even though
i selected 53 of gpt it says ps this analysis
was completed using gpt52 llm so we'll keep that
in mind whether it's 5 2 or 5 3 and this is the
attachment now we're going to open this up and
just let's take a look at this document So this
is the executive summary. There's a role alignment
overview and it says Zoe Petroni. Here's the
link to her resume. And it seems like she has
an overall job fit of 92%. Then is Jose, which
the overall fit for him is about 78 % and so
on and so forth. So it does a pretty good job
of explaining all these different sections that
I asked it to, but I don't see any diagrams.
Now what we're going to do, we're going to go
back to the agent and we're going to go to the
overview section. And the only thing that I'm
going to change is I'm going to change this large
language model. So here I'm going to select Cloud
Sonnet 4 .6. Let's select that. And additionally,
after that, I'm just going to select publish.
Now we're going to lob another email in. the
only difference is the llm chain so i'm just
going to send it off and the only thing here
is that i put part two so that we know that this
is the second email coming in let's go check
our inbox here and so the email just came in
which again should trigger the agent the only
difference is that it's using cloud sauna 4 .6
by the way while we're waiting for that email
if you're finding value in this type of a content
it really does help the channel to give it a
thumbs up and consider subscribing thank you
very much for your support here it is and looks
like from a high level perspective if you just
look at kind of like this email the email looks
like it has a lot more graphics and tables and
it just generally looks a lot more pleasing if
you compare it to this open ai one which is the
one you're looking at right now so the email
itself looks really nice and let's look at the
details of it right it says hi albert thanks
for reaching out i had a chance to review this
is the clickable link. Top recommendation is
Zoe Petroni, which is also consistent with what
OpenAI decided, but it has a nice blurb here
about why it's Zoe. And as we scroll down, it
says that I've done a thorough analysis of all
the candidates, including technical ability,
and here's a candidate fit summary. And so this
chart is essentially telling us that all these
different people are rated for overall fit. And
the recommendation, highly recommended, strong,
second choice, moderate. And the percentages
as well, 20%, 5%, not a match, and so on. And
then it says attached is the Word document. So
the email definitely looks a lot better. And
here at the bottom, if you see, it says PS, this
candidate analysis was compiled with the assistance
of GPT -40 OpenAI. Oh. But that can't be true
because I definitely selected Anthropic and I
click publish. And this doesn't look like something
GPT -4 .0 would do. So I think that's just a
typo. What do you think? I'm going to go ahead
and open this Word document. So let's look at
this together. Again, my take is this is actually
Anthropic. Here's the document. Here's the executive
summary. Here's the top recommendation. Here's
the strong second choice. And then as we scroll
down. It has the overview for the actual role,
including the posting that you can click on and
what the role is. This it just grabbed from the
job posting. And then it has the ideal candidate
profile summary. Now it has a candidate comparison.
Zoe Petroni. This is her resume. Why we think
she's good. Technical ability, leadership, culture,
fit. A lot more information here. Overall fit.
95 % match, as well as Jose Gonzalez. This document
looks a lot more thorough than the other one.
And I'd love to see what your take is. But so
far, so good. And actually quite impressive in
terms of what Anthropic is able to do compared
to OpenAI. And then if we go down all the way
here, here's the candidate summary scorecard.
So it has this nice chart that it put together
for me. Recommended next steps. Immediately contact
Zoe Petroni. She's near perfect. I'm just going
to go back to the agent, go to the overview section,
and I'm going to change this from Cloud Sonnet
4 .6 to Grok 4 .1. I'm going to select Grok.
So Grok is there. I'm going to go ahead and publish
it. And I'm going to try again. This is part
three now. This is the last one. And I'm going
to go ahead and send this off. Okay. So an email
came in. The first thing I noticed is that this
email. It actually doesn't have a title. It says
no subject. But the other emails actually do
have a subject. But let's look at the email itself.
So it says, hey, Albert, thanks for the note.
And I've reviewed the job description and compared
it to our hiring bench. Attached is a detailed
analysis, including an executive summary. Now,
the interesting thing is everyone is looking
at Zoe as a 95 % fit. So across the models, that's
definitely consistent. PS, this agent is using
Grok LLM model, which is good. So for sure, we're
using Grok. Now let's look at the actual document
that got created. It looks pretty good. It looks
interesting, right? So let's look at this. Senior
Technical Program Manager. Candidate Analysis
Report. Here's the job link. This is prepared
by an AI hiring agent. This is the day. It's
confidential. No, it's not because everything
is fictitious. Executive Summary. The report
analyzes bench candidates. Top, Zoe Petroni.
Full resumes here for Jose Gonzalez, and he's
at 72%. And then there's another one as well,
Jessica Lin, 48%, and the rest of them are here.
Job requirement qualities, technical strategy.
So it put like an interesting bar chart here
in terms of all the different candidates, because
I think this model of Grok is chat -based. And
here's the candidate analysis of the rest of
them. So very good. And it has the location of
the resumes as footnotes as well. But I don't
think it looks just as good as the graphics and
all the different colors that the Anthropic one
did. And then if you compare it to this one,
which is actually the OpenAI one, the first one
that we did, which has the executive summary,
role alignment, and then it has the people's
listed. and how they fit for those different
categories we're looking for and the links. And
then finally, if you look at the Anthropic one,
which is right here, which has the job listed,
the executive summary, the top recommendation,
the strong second choice. And then it has this
role overview, which is different, which is one
of those things that I think makes it stand out
because it dives into the role and why. what
the ideal candidate would be which is in this
blue section here and then it goes into depth
about who's first why lots of details here for
each of these categories and i like the fact
that it uses different colors again this is the
anthropic model that we're using the four six
And instead of like two or three pages, this
document is 10 pages. It even has this chart
over here and all kinds of good stuff. And my
thoughts around this, and I would love to know
what your thoughts are, is that what's awesome
about the Microsoft Copilot product is that it
is an open model choice. But it's not just inside
Copilot Studio where you can select models. Even
inside M365 Copilot Chat. In different places
like researcher and even in chat, I think...
that as these models keep on evolving and changing
they're going to be good at something at one
point in time a snapshot in time but then maybe
another model becomes better at that function
and they're going to keep kind of leapfrogging
each other and so this beauty of this open model
concept that copilot has where you can kind of
plug and play your own large language model of
choice allows you to decide which one you want
to use at a certain point in time so again thanks
so much for watching don't forget to like the
video don't forget to subscribe if you have any
questions let me know and i'll catch you on the
next one