Welcome to Artificially Intelligent Marketing, a weekly podcast where we stay on top of the
latest trends, tips and tools in the world of marketing AI, helping you get the best
results from your marketing efforts. Now let's join our hosts, Paul Avery and Martin Broadhurst.
Hello everyone, welcome to Episode 40 of Artificially Intelligent Marketing. It's Paul Avery here
and I'm joined as always by Martin Broadhurst, the fantabulous AI guru. Martin, how are you, pal?
Overwhelmed with new AI tools to play with. You have been having a tinker this week and don't worry,
dear listener, we're going to get into that tinkering in some detail as we go through the
stories today. Why don't we just get straight into it, Martin? There's no messing about it.
I wanted to talk to Arby County briefly because apparently there's a team in Spain that's about
to have the worst league finish ever or at least they're on course for it. And of course,
your team, Derby County, currently hold that record so you must be pretty chuffed.
Currently the worst team in history with 11 points from an entire season, one victory.
It was bleak that year, I remember it well. So the idea that somebody's going to take that record
from us is welcome because we've had that since 2008. And to be honest, I didn't think it would
ever be beaten. I think there's still time for them to turn it around but I got my fingers crossed
for you. Anyway, that's digression. Let's get into the AI. So we've been on the air for a couple of
weeks. Some interesting stuff has happened. Probably the most exciting was the release of
Gemini Ultra from our good friends over at Google. If you are a casual listener to the podcast or a
casual person paying attention to AI, you could be forgiven for trying to figure out what are all
the different names of all the things that keep changing their name. So this was called Bard.
It's now called Gemini. Ultra is like Google's awesome most powerful model that's supposedly
on the level of GPT-4. And you can access it as a chat bot, but also it's going to be in start
baked into Google's products like Gmail and Docs as part of what was called Google Duet and may
still be called Google Duet at least for the next 15 minutes or so, who knows. But for the purposes
of this conversation, Google have released their new most powerful version of chat GPT-4. This is
the closest that we've got to a GPT-4 rival and that's why it's interesting and exciting.
A, because now we've got another GPT-4 level tool we can play with. And B, because as we've
talked about in the podcast before, different providers innovating in different ways with their
tools stimulates each other to keep finding new routes forward to give us even better tools. So
GPT-4.5 or 5 can now be much closer potentially because of this stimulation that Gemini Ultra
is going to bring to the market. But yes, it's available. We've been playing with it. I know
Martin, you got straight in, got yourself a paid subscription. We should all say many thanks to
Martin. He buys a lot of subscriptions to play with all these tools so that he can tell us if
they're any good or not. Tell us mine. Is Gemini Ultra any good? In short, yes. The slightly longer
answer requires a slightly different question, I feel. Is it as good as GPT-4? And I think we'll
get there is what I'll say. First and foremost, it is a very good model and it does feel very
different from Gemini Pro. For anyone that isn't aware of the pricing model of this, it's £18.99
British pounds per month. And it comes as part of an upgraded Google One plan. If you're somebody
who's been using Google One to get the increased storage space on Gmail and Google Photos and all
of that kind of thing, this is an upgrade where you can actually get two terabytes of storage,
as well as access to Gemini Advanced and Ultra. But that's about as much as the extra perks go.
So it's cheaper than chat GPT with GPT-4. And you get these additional bonuses of the increased
cloud storage and what have you. Compared to Gemini Pro, it's an immediate step up. You just
feel that. I think when Gemini Pro was launched pre-Christmas, we all tried it and said, yeah,
this feels like a GPT-3.5 capable model. And it's free, right? So Gemini Pro was free.
If you had a Google account, you could go play with it. Even Google Workspace, I think you could
go play with it. So to have a free model that was at GPT-3.5 or maybe even just above for some stuff
was nice. But yes, compared to a paid model, we're expecting a leap, I would say.
Yeah. And this definitely is a leap. One word of note is that it's only available on personal
Workspace accounts or Google One account. It's not available through business accounts,
which is slightly frustrating. And at the moment, that means that any input is not private, right?
So in the same way that with a Google Plus account, sorry, not Google Plus, Google Plus,
that's going back a few years. Bring it back. AI-driven Google Plus, bring it back.
The chat GPT accounts where you have to be careful what you're putting in because the data could be
viewed or going into the training model. This is the case with Gemini Ultra as it stands. In fact,
there is a warning that says prompts and inputs can be reviewed by human reviewers. So don't
say anything that you wouldn't want another human being to read. Crumps. Yeah. In terms of the
interface is very similar to what we're familiar with on the left-hand side. You've got your
history of your chat and the main central column is where you have your conversations.
Now the inputs are text-based and you can also input images. There is a microphone button
that when I clicked it, I was fully expecting it to record audio and upload that as an audio
input prompt because when they announced Gemini Ultra, this is a multimodal capability. And they
said that one of the modalities that you get with it is the idea that you can input audio
and it will be able to work with audio input. Unfortunately, if you hit record, all it does
is transcribe the text. So it's the same as the chat GPT microphone button. It takes a spoken word,
turns it into text and inputs it as text. That was slightly disappointing.
Okay. So quality of the outputs. I spent a good few hours playing with it, tried it with creative
responses, but also tried to use it as a programmer and a kind of coding assistant and came up with a
project and wanted to jump into it and see how far I got. And overall the responses are very good.
It feels like GPT-4 at that level. And if GPT-4 didn't exist, I would say this was the best on the
market by a margin. However, GPT-4 does exist and this doesn't quite get there. So a few things
that kind of let it down. When coding, I found that the quality of the code that it was giving me
did result in more errors than I was getting from chat GPT when I did the same project previously.
And the way to fix the errors or its ability to fix its own errors wasn't as good, wasn't close
to being as good. I got to a certain point with this one particular project where I asked it to
fix this code again and again and again, couldn't progress. So I took all of the code it had given
me, put it into chat GPT with GPT-4 and said, fix this code. It fixed it. And then I brought it back
into Gemini Ultra and carried on the project from there. I love it. Gemini Ultra, I love it.
I love it. Gemini Ultra is chat GPT's intern. I don't think that was Google's intention.
And that's kind of indicative of the experience with it. There's just things where it's
quite as good. It's very, very, very good. This is not as slight on it as such. I think Google
have done a tremendous job, but it's still not quite as good as GPT-4. That's the reality of it.
For in fairness, when all the launch videos came out, although given that some of them have been
debunked, how much we can pour credence in them. But they did mention having a specific
model for coding. And that wasn't clear to me at the time whether it was going to be baked into
Gemini Ultra. And in the launch blog, Sundar Pichai does speak about that there are other
models being trained and other product releases coming. So I wonder if there'll be like a dedicated
coding assistant that's maybe better than Gemini Ultra. But given the fact that it's supposed to
be multimodal from the ground up, unlike chat GPT, it's surprising that chat GPT would therefore
be better at coding than a multimodal from the ground up model. But I guess code images, video
text, you know, maybe code is a bit of a unique thing on its own and we'll have to wait and see
what this other model is if indeed it turns out to exist in the way that it sounds like it did.
There are other quirks with it as well. So I had an audio recording, it was just a voice memo
that I'd saved. And I put it into Gemini and asked it to turn that into a blog outline.
And I thought I'll use that blog outline to create a blog. And it was just from a 10 minute
rambling about Zapier and chat GPT. And it failed catastrophically because it said,
I'm sorry, I can't generate images at this moment. And it didn't make any sense to me,
right, because I wasn't asking it nowhere in this text. In fact, I put in a solid
block of text, 10 minutes worth of spoken word. At the end of the prompt, I said, use this spoken
word transcription to write a blog outline detailing this, this, this and this. And
repeatedly, it said it cannot generate images at this moment in time. And it apologized. And
if you ever used Bard, you'll see that it came up with a little drop down to show you alternative
drafts that it also created. Every one of them said the same thing. So somewhere, and I've,
I've reran this in a new chat. And there was one tiny section in this transcript where I mention
using Dali as part of a Zapier workflow to generate an image. But that is like one line
in the middle of this big chunk where at the end of the prompt, I'm very explicit,
I'm very explicit giving it the instruction that I want it to do. And repeatedly, it says,
I can't do that at the moment. And I just had this a couple of times where it was saying things
that it couldn't do or would tell me things that, that weren't quite right. So yeah, that was,
that was frustrating. That aside, there's a lot to like about it. It's super quick. I feel compared
to GPT-4, it generates responses quicker. Admittedly, we have GPT-4 turbo, which is,
you know, pretty quick now. This does still feel quicker. Yeah. But would you not say,
I don't feel like the speed of output of GPT-4 is like a major limiting thing where I'm like,
I'm not going to work with it. It takes, you know, half a second, a second longer to produce an
output. I think it is a little bit annoying. And I do like the idea at some point of just like
getting an instant output, but the moment outputs are about the right speed that I can read at.
So it works quite well for me. But yeah, so I appreciate the speed as being a little bit
of a benefit, but I definitely take quality of output over a slight improvement of speed, I think.
In terms of creative writing or editing content for like blogs and things like that,
I like it. Actually, it has less of those, I think of it as the, well, with chat GPT,
if you ask it to write a blog, it just seems to say we're going to delve, we're going to dive,
we're going to, it uses these phrases that are, just see them so much more with chat GPT than I
see them in real life. Well, certainly more than I would ever write them. And I think there's been a
proliferation of them across the web. This doesn't do that as much. And it gives nice reasons for why
it's phrased something in a particular way, if you ask it to kind of explain why did you describe
it in that way. So I do think that the content writing is pretty good. I thought the marketing
content that I got from it when I was testing it with email campaigns and subject lines and
things like that, all very good. I asked it about how to approach a particular project,
gave it an outline of what I wanted to achieve and the steps involved. And it did a really good job
with giving the step-by-steps as good or even better maybe than the chat GPT. So I thought that
was worth noting. It's context windows 32K, which is the same as GPT-4 and chat GPT. So that's no
real difference there. It can't do image generation at the moment, as it told me repeatedly, even
though I didn't ask it to create an image. I think that will, I would imagine that that will be
coming down the line fairly soon. Google has been making some noise in the image generation arena
over the past few weeks with some new models that it's created. So that's definitely something that
I think that might be being rolled out. I remember seeing a few posts online about Google's image
generation capability starting to make it into these models and that the images were good,
photorealistic, probably better than DORLY 3 if I remember reading. So I think that will come soon.
It's interesting, your take on creative writing. So Ethan Molluck, who does a lot of work in this
area, wrote a blog post about Gemini Ultra having had it for about a month before the rest of us
got it. And I think here's takeaways with similar to yours, right? Poor at coding, probably on par
with text generation for things like creative writing. You did mention that it had a bit of a
more friendly persona and a bit more inclined towards things like conversation and wordplay,
which I think is interesting. Yeah. One of the things I noticed about it very quickly was
its responses are super friendly, actually, and almost like a mentor. So the way that it's,
there was a particular phrase that is like, I'm really excited to get started on this project with
you. When I gave it a brief and asked it to tell me step by step how it would approach it,
it did that. And at the end it was really enthusiastic and said, I'm really excited to
get started on this project with you. But then as I reached a certain point, and it was the coding
one, it was the little app development piece that I wanted it to do, I got frustrated and I said,
this isn't working. I think this project's going to be a failure. And it responded with a piece of
like coaching and it was saying, I'm sorry you feel this way about the project. Let's break down
some of the reasons why you think you might be having these feelings about the project and
how we might be able to overcome it in the next steps. It felt like a real personal dialogue and
a bit of mentorship kind of jeeing me up and telling me that all isn't lost.
So I guess the million dollar question is, are you going to keep your subscription and run it at the
same time as having chatGPT teams or is GPT-4 does everything that Gemini can do and better? So
actually if you had to choose, you'd just choose GPT-4. Like where do you sit at the moment based
on what you've been playing with? If this wasn't my job and I was only going to pick one model,
I would pick GPT-4 or chatGPT above all others at the moment. However, there is some caveats to that.
And that's if you're someone who is personally in the Google ecosystem already, you've got Google
photos, you have got a Gmail account, you use Google Docs and Google Sheets to manage your kind
of personal, you know, maybe you've got a Chromebook and you're kind of all in on the Google
ecosystem. I think for 19 pounds a month, including all of those add-ons that you get, I think I'd be
tempted to go with that. Really? Because you get access to, if GPT-4 is really what, there's some
reason that you really need GPT-4, you can access that via Bing chat. You don't get advanced data
analysis and code interpreter. So if you need those specific elements, get chatGPT, you need that,
right? That's a specific function of Vita that you need. But if you're just looking at the quality
of the output and you're looking at it from a value for money lens, I think Gemini Ultra for 19,
so it's cheaper, 19 pounds a month, plus all of the add-ons. And if you think that you might already
be paying, I think I was paying seven pounds a month for my Google One account. So effectively,
it's 11 pounds or 10 pounds more per month. I think that's a good deal.
You said like a man who loves a subscription, Thierry Martin. I don't disagree with you. I'd
just like to tease you about all your subscriptions and your app sumo addiction. I think I'd be most
tempted to play with it when it comes to workspace. We run on Google Apps, a biostrata in terms of
that's what we use for email. And so the ability to summarise email threads, maybe suggest replies,
I think that's going to be interesting and a potential time saver. If you run on Microsoft,
you're going to use Copilot, probably going to get similar quality outputs by the sounds, I think.
So it might just depend on what ecosystem you're on. It makes me feel better to know that
Ultra is getting up there with GPT-4 because I haven't been on Google Apps this whole time.
There was a bit of a concern that if Google is slow to the party, do we have to migrate
to Microsoft just to get access to all these tools? Paul Roitzer was on the LinkedIn's doing
a breakdown of the value of these tools versus the time that they save you. And, you know,
you look at the time they save you, I would argue that there's also value in the effort of
switching platform if you need to, if the other platforms are not good enough. So that was
definitely on my mind. I think it's important for the listeners to recognise a lot of the things
that plague other models, plague this model. So things like hallucination are still an issue
when you're using Gemini Ultra. One thing I do like about the interface, though, is when it gives
you an output, there's like a little button that you can click. I think it's like a G for Google,
and it will actually go Google parts of its own answer to figure out if it can find information
that either backs up or goes against what it's just told you. So it's like a, it's like
hallucination checking light. For some of the outputs I get, I get there's maybe a thousand
words and 150 of them are highlighted in the green that says, hey, this we can corroborate this with
things we found online. So I don't know what that means for the other 850 words. Right? Is it made
up? Don't know. So it doesn't always work that well. But I've also had outputs where 70% of it
was I think it's green, it might be yellow, the colour that says A is okay, I found some of this
stuff on the web, and occasional red, which is a little bit of a red, which is a little bit of a
red, which means this actually goes against what I found on the web. And I found that button to
actually be quite helpful. Ultimately, that's not better for me than perplexity. Now you've got that
me onto that. So I just using perplexity anyway. So it's more of an oddity for me. I just like,
oh, that's interesting. An AI that checks its own work. That's what we need.
It does have another interesting little UI element as well, which is I've only seen this on the
mobile version though, not sure why. I haven't managed to find it elsewhere. But you can click
on a button and it will ask, you can get it to rewrite the response that it's giving you. So
make it more formal, make it longer, make it shorter, make it more professional. And that's
just right there in the interface rather than having to prompt it to say make that sound more
professional. Yeah, that's kind of similar to what you can get when you're using Bing. I think we're
going to see that in Gmail's AI add-on and Outlook's AI add-on. I think that's going to be. But when I
use those tools, I don't know about you, but it doesn't half love the edge of the continuum.
It's like, make this more chatty. Yo, what up dudes? I've been just chilling. It's like, oh,
crumbs. I didn't mean that chatty. Just like a little bit less stoic prison guard barking that
you had previously just maybe soften it up a bit. So those buttons have seemed to be a brilliant
way of like racing to the extreme without ever actually getting me what I was after in the first
place. So I actually think when these tools learn our styles, personal styles, and I'm sure they will,
that's when it will really help. They be more witty, be more conversational buttons,
and mostly just and fun oddity for me at the moment. They don't actually give me anything usable.
Will Barron Well, thank you for taking us through Gemini Ultra. I think we've given it 21 minutes
or so. That's more than enough for Google to dominate the start of this podcast. Let's move
on to our next story, Martin. You've been looking at some recently published AI readiness index from
Cisco. What's this all about? Martin McGrath
Yeah, they've published a report looking at how corporations around the world are preparing to
integrate and deploy AI within and across their various operations. And there's some stats in
there that won't surprise you. For instance, 84% of companies believe that AI is going to
significantly impact their business. That's not a great surprise at all. Some of the areas
and the recommendations from the report, the areas of insight and the recommendations, should I say,
are worth highlighting. So before we get into that, just a few of the key statistics. 97%
of respondents have seen increased urgency to deploy AI technologies, which is not a massive
surprise when every company that comes out on an earnings call, when they mention AI, their stock
seems to grow a few dollars and pumps the price a little bit. Only 14% feel fully prepared for
AI integration though. And 76% of organizations say that they will require more GPU resources
for AI workloads. So thinking about the actual infrastructure, they're going to have to be
investing. Good news for Nvidia, I guess there. 37% have identified skills gaps in understanding
AI tools and getting their workforce AI proficiency raised across, that is a big
priority for them. And unsurprisingly, generative AI is a focus. 40% of companies are actively
deploying or planning to deploy it. So they're the kind of headline figures for what came out
of the report in terms of takeaways for companies. Well, Cisco identified six pillars of the critical
success factors really for being AI ready. And they said, it's strategy, infrastructure, data,
governance, talent, and culture. So these components range from having a very clearly defined AI
strategy and roadmap, and then owning and having access to the necessary infrastructure to support
those AI workloads, hence those CPUs, ensuring that you've got the right quality of data,
because that's going to impact everything that you do with your workflows. Particularly if you're in
healthcare industry or if you're in the finance sector, having the right data and the right
quality of data, so important. And there's a huge part of any AI project, which is basically data
preparation. And that's something that companies need to get right. And then proper governance,
who has responsibility for what, what are the boundaries, making sure that you've got the right
technology stack. All of these things come into the mix. Finding the right talent, which is
increasingly proving difficult. There's so much demand for AI talent. And then fostering a culture
receptive to AI driven change was the big one. And that requires the right leadership and the
right messaging to communicate that through all of the teams. Surprisingly, the report suggests
that AI could be more of an opportunity for workplace growth than a job replacement threat.
With companies planning significant investments in upskilling their employees in AI technologies,
rather than talking about replacing employees with AI technologies. So despite the moderate and high
urgency to embrace AI, the willingness to adopt varies significantly depending on one's level
within an organization. So what it found was that people lower down organizations are a little bit
more reluctant to embrace the change when the pace from senior management is we need to be all AI,
all go, go, go, go, go. And I think that's reflected in a wider cultural piece outside of
the organization individually, but just a kind of, you know, the macro corporate environment.
There was a really interesting interview with the editor in chief of The Economist. And she sat down
at Davos with Sam Altman and Satya Nadella. And they had a conversation about AI, where it is,
where it's going and all of that. And what was very clear was that the theme of the year at
Davos this year was AI, AI, AI. So clearly corporate leaders are just swimming in this pool of AI hype,
but then cascading that down into the business where you've got, you know, the project managers
that are trying to just get on with the day to day and deliver what they've got to do,
whilst at the same time being told they've got to implement AI into their workflows. There's a
there's a mismatch in appetite for AI intervention.
Yeah, it's a funny one. When I reflect on this report and some of the conversations that we have
on the podcast, I actually think there's going to be not only is it not going to be one size fits all
in terms of how different organizations and different sectors can leverage AI to improve
productivity and efficiency. But just starkly, I'm thinking, if you're an enterprise level company,
the way you need to approach and think about AI as a differentiator and an accelerator for you is
going to be very different from an SME. And it's probably going to be different from the
mid market. And the first thing, and I think there's pros and cons, right? Like if you're
an enterprise level, you probably have lots of access to lots of data. And you do need to think
about format and cleanliness of that data to then drive the production of your own fine tuned models
or even your own models based on the data that you have. If you're an SME, you've probably
got some data, but it's probably so poorly structured as to make it maybe borderline useless.
I don't know. I think if the AI learning tools get good enough to be able to just create insight
from unstructured data, we've talked previously about having a company record all their sales
calls and all their customer service calls. Even if you're a company of 20 people, you could do
that right. But that's unstructured data. So as long as it can start to use that type of unstructured
data to then train models on, I think that's fine. On the smaller side, the change management's
easier because you've got fewer employees that you have to think about how you're going to bring
them together to adopt these tools. Training is probably easier than it is to roll this out across
a 10,000 person organization with offices across the world and different cultures and maybe some
of the tools are really powerful in English, but maybe in local language, maybe they don't work as
well and all the other problems that you're probably going to get. So you step back out of
this for a second, you can see that how you're going to deploy AI in your organization is quite
complicated depending on what type of business you are, what services and products you offer,
structure of your team, size of your team, access to data. We can keep mentioning these reports and
I think they're helpful because they give us all a lens to look through, but you're probably better
off really looking inside your own business against those different facets and thinking,
what does this mean for me? Well, now seems like a relevant time to say go and read my paper published
in October 2022, the month before ChatGPT was even launched, which was about how to deploy AI
in your business for any shape or size of organization. Because it touches exactly on
that. It's about how much resource and how much data you've got in the organization, right?
And if you're a small business with not a lot of resource and not a lot of data,
there's guides in there for ways that you can practically use AI within your organization.
There we go. Alley-oop. I'll put them up. You slam them down.
Cool. Good stuff. Thanks for sharing that, mine. Our next story this week is about Microsoft
Copilot and the recent upgrades to some of the design features and AI image generation features.
So basically, a bit of an improvement to Microsoft Copilot in terms of both its mobile and web
versions, which makes it easier for you to work with images that you've generated using the chat
interface. So it's kind of cool, but not as cool as I would like it to be. So what you can do is
when you're generating images powered by Dually 3, you can now instantly move those into an editing
environment where you can crop them, resize them, you can magic resize them, so you can change the
aspect ratio of them and the AI will fill in the gaps to make that landscape image work as a
portrait and what have you. And all of that's pretty cool. You can do some interesting things
like background blurring and some shifts in the style, which I think is interesting. But when I
first read the story, I thought you're going to have more fine control over AI-driven editing
of things in the image, like, oh, replace that hat, that bowler hat with a cowboy hat.
And we're not quite there yet. I have confidence that in 12 months we'll be able to do that. But
if those of you who saw this story out about on the LinkedIn's, the Twitter's or the press and
thought, oh, now it's finally here, I can be a designer or I'm briefing a robot designer with
text and it's doing cool things to images, not quite there. So you can do some interesting things
and it is a much streamlined workflow. But for those of you that thought it was going to be
like having a digital designer that you were brief and make very subtle amends to images were
not quite there yet. But I still think it's pretty cool. And it's definitely worth checking out if
you do this type of image generation. But yes, not magic. Thor's mine. They also announced
some updates to Designer, which is a kind of, well, it's a graphic design tool, elements of Canva
about it, lots of templates, drag and drop, very WYSIWYG, easy to use. You can export your
generated images into Designer and use those as part of creating a social media post or something.
That's all fine. The thing that I want to make people aware of is it's the first commercial
application that I've seen that users, or at least I think it uses, it certainly looks and
feels and behaves like it. The Segment Anything model from Meta. So you can just click on any item
and it will cut it out and it will select it. You can duplicate it. And it's got that same
feel where you hover over an item and it will kind of lasso it or most of it. And then it will
actually get all of it. And it just, it feels exactly like that implementation of it. So when
we saw that model, I think, was it April time when that was open sourced, whether or not it is that
model, I don't know, but it certainly feels like it behaves like that. So that's a nice little AI
feature added into Designer. Yeah. Adobe's got something like that in what was called Firefly,
but all these things are now having their names changed as they just get baked into products. But
a lot of the demo videos from two, three months ago had Adobe where you click on a suitcase,
automatic lasso the suitcase, delete the suitcase, and it fills in the background like it was never
there. And I think we're just starting to see those emerge. Absolutely going to be driven by
the type of Segment Anything models that you describe and the ability to contextually understand
what's in the image and select it for you. So I guess because of that capability, we're probably
not that far away from speaking to the computer or a text based prompt that says select the suitcase
and remove it. Right. Yeah. Because that's just going to be an action, isn't it? It's a series of
actions, which is click on the suitcase, select it, delete it, generative fill it. That will easily
be done through an AI agent before long. Yeah. Apple released something recently as well that
was similar. They were working with UC Santa Barbara researchers on new sort of image editing
AI that they've released as open source, which is interesting. And it's kind of very similar in that
you have the ability to interact with bits of an image in a conversational way to have it change
the brightness and tweak colors and stuff like that. I think what was interesting about this
model is because of the way it's trained, its ability to influence just the things that you
mentioned conversationally and leave the rest of the image alone, I think is a bit of a different
differentiator for it. So it's kind of drifting towards what I hoped Copilos News was. So if
segment anything is anything to go by, we can expect to see this type of capability in a paid
product within the next six to eight months. But just further evidence that we're drifting towards
that. As a marketer, it's interesting because the further democratization of tools to make it
easier for non-technical specialists to make these types of edits is obviously increasing.
You might write a written brief or call a designer and say, Hey, love the brochure cover,
but can we move this thing here? I want to change the color of this. Now, maybe you do that by
speaking to the computer instead. But the one thing I always fall back on when I'm thinking
about this is it doesn't teach the human user what good design looks like. So it's a bit of
a rubbish in, rubbish out potentially in terms of it's great to be able to ask the computer
and an AI to create an image and then edit specific parts of an image. But if you don't know
about font use, balance, image composition, all these other things that talented designers
are really good at, then probably you'll just have a more official workflow for creating crappy
looking images. And then the other thing is the creative brainstorming that comes behind
creative campaign concepting or brand concepting. Again, I do think you can use some of the tools
to get you started on some interesting ideas, but we're at least to me, it feels miles away from
coming up with really polished creative campaign concepts. So it's interesting and exciting.
And I think to be honest, we're probably going to see that these tools make professional designers
more efficient. I'm not so sure that they're going to make it that much easier for non-designers
to become designers. Do you know what I mean? Yeah. If you're still sticking word art on your
posters and putting things out in Comic Sans, this isn't going to help too much. Love a bit
of Comic Sans. It's just such a happy, such a happy font, but maybe not for the professional
branding arena. Right. Let's move on from image generation. Let's turn to some more AI regulation,
Martin. You've been looking at what's been going on in the European Union. Yeah, just a kind of
headline update really. So we've been following this story as the EU's AI Act has been moving
through the parliamentary process. Well, there was an update last week and the fight, so this is the
pre-final text of the EU AI Act has been endorsed by all 27 EU member states. So the next steps are
that the European Parliament, internal market and civil liberties committees will review it
with a plenary vote provisionally scheduled for April this year. It's hoping that it will be
enforceable, assuming it all goes ahead, in 2026. And I know that Germany's approved it,
which made big headlines. They were one of the ones that everyone was looking towards as one of
the big leader companies, companies, countries in the EU. So yeah, it's taking the next step
forward really through that process. I think if you want to have a look at what the text is going
to largely contain, the version that we've got is going to be it with very minor tweaks around the
edges between now and its final publication. I'm glad you mentioned this story because you reminded
me of something related to callback to Gemini Ultra. So bear with me, listener, we're going to
get there. We have to go around the houses to get there. But Gemini Ultra is going to be baked into
Google Assistant. And I won't say it because I'm surrounded by far too many Google
technologies. One of them is going to tear apart. But obviously you can
speak to your tool by mentioning its name and then you'll get the Assistant technology that
at this point hasn't really improved much in the last five to 10 years. But if you're in the US,
you can switch your Assistant to not being the standard Google Assistant. You can switch it to
being Gemini powered, which is interesting, right? We've been talking about where are all the better
LLM driven smart assistants? Where are they? Well, they're here, but they're not here for Martin
and I in the UK and they're not available in the EU either. And whilst we don't have a clear reason
for that, the rumor is that Google has been paying attention to what's going on with this act before
it takes too many big moves that it thinks could contravene aspects of the act in the EU. So whether
that means we have to wait quite a long time for Gemini powered Assistant or whether this is just a
rumor that isn't really real, you did remind me of it, Martin, when you talked about the story,
because I really, really wanted to try out the new Assistant and I just can't even with VPN,
I can't get it to switch. Which is a surprising move from Google's end, given that we are no
longer in the EU and the regulatory environment of the UK is more innovation friendly, is certainly
on the AI front at least what we're led to believe. Yeah. Well, as I said, it's just a rumor,
might not have any foundation whether or not there's belief in circles that the UK will lean
into following whatever the EU puts in place. I mean, it's tempting to think that the world will
lean into whatever the EU puts in place because the EU has done all the hard lifting of putting
this all together. And you could certainly imagine for a lot of countries, it would be easier to fall
in line with this than it would be to come up with your entirely own approach to doing these things.
But interesting. Well, we should keep an eye on that as it moves its way through. But it sounds
like it's going to pass a little changes. Right. Lungs couple of bits this week, they're less
stories and more just a few discussion topics that I wanted to get Martin's thoughts on.
So the first one in we're going to go back to our friend, Ethan Molloch again here. So probably six
months ago, Ethan ran an interesting experiment with maybe more than maybe it was a year ago,
but he ran an interesting experiment with AI tools as they were emerging to basically create
a variety of content. And I think it was come up with some sort of product launch,
maybe even a new website. And he was able to do all the work that he needed to do in about 30 minutes
using AI tools, which would probably have taken human people like a week or two of time to just
put everything together. So he recently decided to run a similar experiment again, and this time
to show what he could do using these tools, but in just a minute or as he says in 59 seconds. So
he undertook a range of tasks all at the same time. And if you can find this online, it's worth
going and watching the screen capture video of his desktop because he's split it into five parts.
So you can see all of these things running at the same time. So firstly, he uses Microsoft Copilot
to turn an AI written Tesla business case file into a PowerPoint presentation. So it's a word
to PowerPoint transition. He then uses Copilot and Microsoft to devise a detailed syllabus for a
six session intro to entrepreneurship class, complete with table summaries, assignments,
and the grading criteria. He used ChatGPT with a custom GPT made called trend analyzer
to identify and showcase some on trend designs for, I think it's for clothing.
He applied ChatGPT with his own product launch GPT to automate the launch process
of a Wharton interactive products at Wharton Business School. And then he used Bing to draft
a market research study on the VR and AI, VR and AR device market in the style of a top consulting
firm. And his biggest challenge in all of this process was that originally Bing said no,
and he managed to convince it to do it. Typical there of large language models.
So find the video online if you can, because he does all these. He clicks in all the different
windows and within a minute he's got outputs from all of them. And as he says in his blog post,
where he summarizes what he did, the quality of the outputs was surprisingly high,
which is even a big improvement on the last experiment he ran, because we've got these
better models now. So it's quite an interesting one. And as much as this is a big leap forward,
one thing he highlights in his blog, and this is what I want to get your take on Martin, is
up until now a lot of knowledge work was about words, writing reports, creating reports,
writing emails, the fact that that takes time and was considered high value work.
If you can now hit a series of buttons and get all of these tasks accomplished to a reasonable
level in 59 seconds, where your job is then to go in and edit and increase the quality of those
things. How does that change work? How do we understand what is good work and what is bad
work when AI is doing so much of the lifting? Where's the human bringing the magic to this?
For lots of jobs that are based on time investment, like agencies, honestly, a lot of agencies are
based on a time model, lawyers, lots of things are based on how much do I charge for this based
on how long I think it would take my team to do. We talk about this a lot, but it was just so
interesting to see all of that work done at a reasonable quality in 59 seconds.
What's your take on some of these questions, Martin?
Well, it makes me think of somebody I was speaking to at the weekend. They worked for a large
corporate, big engineering company and they are not allowed to use LLMs at all. They're all locked
down. And this person I was speaking to, an engineer, had this particular job which required
him to... It was basically, he knew it was going to be his entire afternoon copying and pasting data
from spreadsheet sources, like from here to here, here to here. And that was going to be
an entire afternoon spent doing this part of his task so that he could then run some models and
do some analysis. And he's not allowed to use ChatGPT, but he went on his phone and did it
on the side and said, ChatGPT, give me a VBA script that will enable me to execute this thing.
And he said within 30 seconds, he had the script and he implemented it and it did his four hours
work in about a minute. And the whole process, start to finish, was two or three minutes. At the
time it took him to write the prompt, to copy and paste it, email it to himself. He says, if I was
just... If we just had this as part of our workspace, think about how much time that would have... I
mean, it was already saving a huge amount of time, but if everybody in the organization had this,
rather than being in an organization where they are scared to open this up, for whatever reason,
I think there's data and compliance and IP related concerns, all of the stuff that we've spoken about
in the past. But that just made his job so much easier. And the output that he was going to get
to in his particular instance still required some human intuition, but that data prep piece,
literally copying and pasting, was going to be done for him in seconds. To the wider point about
knowledge and time, we've said this when GPT-4 came out and we saw lawyers, we could see that
GPT-4 could pass the bar exam with the score that is equivalent to top 10% of humans that sit it.
The thing that makes solicitors and lawyers so highly respected is the time, effort, energy
they've put into learning their subject matter. And now everybody has an expert in their pocket.
That can do it. So it's going to diminish some of the value of that work. It has to. There's an app
called Robin.ai. I don't know if you've seen this one. And it's been built using Claude from
Anthropic. And it does contract analysis. And the whole purpose of it is to analyze all of the
clauses within a contract and then help you write the clauses that you want adding into the contract
and help you basically negotiate better using AI. That's something that previously a lawyer would
have charged you, I mean, what do lawyers charge? Hundreds of pounds, thousands of pounds per hour
to sit and go through line by line, checking over every clause. Now we've got AI that can do that,
explain every clause to you and it'll take... I don't have the answer to the question of what does
this mean for work. I just know that it is going to, it's going to change what we perceive as
the value of that kind of work, but it's also going to make as much more productive as possible.
Much more productive. It's going to empower individuals in so many ways that we... It's kind
of hard to imagine really. If everybody has access in their pocket to a legal advisor that is
pretty good, right? Really high level or a doctor or a tutor or a mentor, it's going to change the
way that we think about how we approach and how we value outputs and tasks within the workplace.
Yeah, it's a bit of an Ethan Molyk episode, isn't it? Ethan Molyk and his team released another
paper where they studied consultants at Boston Consulting Group. It's a pretty robust study by
looks of things with large numbers of consultants split into two groups. One group had access to AI
and the other group didn't. One of the sort of findings, it's more of a hypothesis that came
from the analysis, so it was not robustly tested yet, was does access to AI make the smartest,
most capable people better or does it take people who are new in their careers or maybe lower skill
levels and bring them up higher? So does it compress the average? Fundamentally, there's no
reason why both those things can't be true, right? But with AI still prone to hallucination,
my feeling is AI can help do a lot of initial drafting type work. If you're in knowledge work
and a lot of what you're doing is producing insights that end up in written form,
somebody who understands the topic deeply has to review those pieces to look for hallucinations,
but also just to make sure, are we talking about the things that are important for this report?
Right? Because even when you try and give AI a lot of context, you can't give it all the things you
know when you've done something for 20 years, like, and it doesn't have experience like humans have
experience, right? So I'm drifting towards, and this is a deep fear of mine, really, but I'm drifting
towards, could it gobble up intern and graduate roles where people coming into the workforce
from university or by other mechanisms would usually get their training, but now they don't
get it because AI can do some of that entry-level work and at $20 a month, it's cheaper than hiring
a grad, which sounds amazing, like commercially, like, let's say you're the FD, you're like,
well, yeah, okay, this is an obvious choice. But sooner or later, we're going to run out of the
experts to actually validate the outputs from the AI because we won't have trained any juniors
because we would have let AI be the juniors. So we won't have people to validate the outputs anymore,
whilst also creating a bit of a socioeconomic meltdown by not giving kids jobs in graduate
positions. So it's kind of interesting, if I look at the way capitalism is incentivized or
incentivized businesses, it's almost like, don't choose the easy route today because you may pay
for it tomorrow. I guess the only bet would be if AI can accelerate fast enough as to not
make that a problem. But yeah, I think when I look at it, my worry is more for those entry-level
graduates and, you know, I'm in my early 40s, I don't have any kids of myself, but I know a
number of people who've got kids of that 15 to 20 years old and they're asking me what should they
do? And it's, you know, I'd be worried if I had kids of that age, to be honest, because I think
they've had to deal with the pandemic first during their schooling.
Deal with the pandemic first during their schooling. And now potentially, AI could come
and gobble up some of those entry-level jobs that they need to kickstart their careers.
Yes. Equally, because of how good LLMs can be at mentoring and teaching,
I think there's also a case that it can help people become experts quicker as well.
So it could almost help them to become the experts that they need to review the documentation. Like
where I was talking about the Gemini Advanced had this kind of mentorship role where it was telling
me how to approach a project in a slightly different way or how to think about things in a
slightly different way. If we embrace and lean into the idea and the opportunity of
AIs being personal mentors and assistants, I think we could actually start to get to a place
to a place where taking on junior members of staff that are well-versed in how to engage
and interact and use an LLM could be less risky or more, what's the word, deliver a better
return on that hire quicker than it would do otherwise. Because if I was, at the moment,
I don't hire any staff. If I was taking someone on, that would be a big consideration for me.
It's like how comfortable are you using ChatGPT? Because actually, I don't really, as long as you
can figure that out and you're a bit of a tinkerer and you can ask the right questions and you've
got the right mindset, you can GSD, right? Get shit done. And that, I would be more prepared
to take on somebody that was in a grad role with that mindset and with that toolkit than taking on
someone that maybe had a little bit more experience but was a bit reticent and not prepared to embrace
ChatGPT. I think that's the critical piece. Ironically, I think that's always been the
critical piece, honestly. And when I answer that question, I do answer something similar to what
you've said, which is people who are curious problem solvers who make the best use of the
tools around them will always succeed. Especially against the old adage of,
I don't have to outrun the bear, I just have to outrun you. So I do think those people will
still be successful. When I was in the lab, I hate my lab colleagues, don't listen to this podcast.
When I was in the lab, I used to wear a t-shirt that said, just f-ing Google it on it. Because
I used to get so many questions where literally people would watch me Google it and then I would
answer their question and I'm like, dude, just Google it, right? That's what I'm going to do.
You can do it and you don't have to bother me. I didn't have it that often, to be honest. I just
thought the t-shirt was funny. But I think that just ChatGPT is the equivalent, right? So I would
argue we've had the tools for those types of curious self-starting problem solvers to excel.
We've had them for best part 15, 20 years and this is just the latest version of them.
I think they're powerful, but I think they're prone to surfacing the wrong information like
someone using Google who's not sure how to validate the outputs would be.
To add a little extra part to what you said, for me, one of the biggest challenges post-pandemic
is training junior people because they're not learning... I don't know how to describe this
adequately, so I'm just going to do my best. The juniors are learning more slowly and they're not
adopting skills and they don't have the insights that they might have had pre-pandemic
two, three, four, five years into their career journey. If I had to guess why, I would say it's
all those little conversations that you see happening in an office environment that you're
learning from by our osmosis where people with more experience than you are having a conversation
about something that's somewhat related to what you're doing that you're learning from.
I think they're massively missing out on those and I still think LLMs are great at producing
information but I'm not sure they produce insight. They don't have the hard-earned experience of,
yeah, but if you do it in that way in this particular context it's going to go wrong because
of this and how do I know that? Because six years ago it got burned pretty badly when I messed that
up. LLMs don't have any of that so they can't transmit that and that would be my fear. So
I completely agree that a savvy, curious, problem-solving driven grad with an LLM is going
to be very powerful than someone who doesn't have it but I still just worry that that lack of insight
is where the value is going to be lost and how to get that baked back into the system.
Well, yeah, that's for GPT-5 to figure out.
Yeah, I was joking with a mate of mine about how awesome it would be if everyone could
take a year off work, like literally everyone, find all the best people in the world in their
domains and have them do the human reinforcement feedback on a model based on all of the deep
experience that they have. Take the best surgeons, take the best artists, take the best politicians,
take the best lawyers, marketers, business people, entrepreneurs, you name it, and have them do the
reinforcement because then it's almost like humanity spends a year together training AI by
having our very best and brightest be the trainers and the mentors for the AI because I think you'd
end up with something quite cool and interesting after that.
I have heard some AI scientists discussing this exact problem and the problem with reinforcement
learning from human feedback being that you've got to have the right humans giving the feedback
because there's no point having me or you giving feedback on responses about brain surgery.
We are not the right people to be doing that, but equally trying to get brain surgeons sitting
there rating AI responses doesn't feel like a great use of their time or at least feels like it
would be quite an expensive process to go through to get hundreds of people to do the
work to get hundreds of brain surgeons that just short term responses.
So that's why we all need a year off. Who do I speak to about this, Martin?
You must know someone. How do we get a year off for everyone?
That's Sam A on Twitter.
Maybe that's why he's getting the seven million, seven trillion I should say.
Seven trillion, yeah. Currently trying to secure funding for chip manufacture, fusion based energy
systems and all the things that he feels he needs to create Skynet. Maybe we just put all of that
into getting all the experts that we need and paying for them to have six months off work so
that we can train all these AIs to be ninjas. If your investment request doesn't start with a
two, then what's even the point? Let's get out. I have got an idea for a bubble app that I do
think would do really well with a trillion dollars of funding. I'm going to spend a hundred dollars
less than a trillion. All of that's going to be spent on paid advertising and a hundred dollars
that we spent on trying to create the tool. Anyway, let's respect our audience's time,
Mike. We're going to move on. Last segment here is going to be, we're bringing back tool of the week.
Are we bringing it back forever? Probably not. Are we bringing it back today? Yeah,
because Martin's been playing with Claude for Sheets and he loves it and he wants to take us
through it. Well, that about sums it up to be honest, but to put more flesh on the bones,
I have been playing with Claude for Sheets and this is a login or should I say add-on,
it's a Google Sheets add-on developed by Anthropic themselves. It's a first party
plugin which unlike ChatGPT and GPT for Sheets, well, there isn't one for that. There is a GPT
for Sheets, but it's third party plugin and you have to pay for the privilege now of using that.
It was free for a year and that's actually what made me check this one out because
it's been available for a while. It's well documented on the Anthropic website and I thought
I'd give it a go. It's dead easy to set up. All you need is your API key and you can then connect
it to all of your Sheets. When you use it, you just input equals Claude, open brackets,
and then you write your prompt and use your various parameters. But as it's a spreadsheet,
you can nest the prompt and the function inside other formulas. So you can have if this, then that.
You can have if statements and run a prompt according to different criteria or different
conditions of your spreadsheet. There's actually three types of input. So you can have Claude,
which is basically the user sends a message and gets a response. You've got one called Claude Free
and with Claude Free, what you're doing is you're having it more as a conversation starter. So you
can actually have more control over the response that you get back from Claude. So where Claude
might have a bit of a habit of starting its response with a bit of preamble, something like
here is the summarization of the text that you have requested. You don't want that. So you can
put that at the start of the response from it so it doesn't give that. Or you can make it say
anything that it wants at the start and then the cell populates with the rest of the response from
the assistant. And then there's a third setting, which I can't remember what it's called, seems
to do a similar thing. It's more for message based back and forth. You have full control over
the API settings. If you've used chat GPT, you'll know that you only have the input of the text.
You can't control settings like the temperature, which controls how creative or constrained chat
GPT is with its response. You can't control the token count except by saying, do it in less than
400 words or something like that. You can put it into the prompt, but with Claude for Sheets,
you can control the token length. You can control the temperature to make it more creative or more
precise. And you can also control, well, we'll see the other setting. There's another setting as well,
but it escapes me now. But it's super easy to use. If you're familiar with spreadsheets
and you're quite comfortable using Excel or Google Sheets, check it out. Within 15 minutes
of playing around with it, you'll see huge value, particularly if you work in SEO,
if you work in e-commerce. In fact, I had been playing with this and then did a workshop last
week getting started with chat GPT. At the end of the session, I spoke about integrations and tools,
and I mentioned this particular plugin. Yes, I know it's not chat GPT as Claude, but
people were interested. And one of the people that was on this session,
he's got an e-commerce store, hundreds of products, and he's
moving from one platform to another. So he's changing his content management system,
e-commerce platform. As such, he needs to reformat all of his product descriptions
to match the new template that he's using. And he said it's hundreds of them and he's
kind of dreading it. I said, well, have you got them all on a spreadsheet? He said, yeah,
I've exported all of that, but now I've got to sit down and do the rest of the work. And he said,
all of that, but now I've got to sit down and just do it all. And I showed him the plugin and how it
could work. And we estimate that with, well, maybe like 10, 15 minutes finding the right prompt and
playing around with the right prompt. But once he's got that, completing the task less than a
minute, and then he just needs to upload the file to his products. A massive game changer. Yeah.
And we've talked about some of these plugins before. I mean, one of the pains of the GPT-4
based plugin is it hangs all the time, right? Because I think it's a 30 second timeout. So
we always got fed up of using it because it hangs. What's the clawed timeout like?
So the limitation, that 30 second timeout that you're referring to is a Google Sheets
restriction. So anytime it makes an external call to an external API,
if it doesn't get the response back in 30 seconds, it just errors.
So clawed has a setting within it, or clawed for Sheets has a setting within it, to rerun any errors.
And it will just rerun those cells. It also has a state called deferred. So sometimes,
let's say you've got 100 cells that want to be updated, because of the API limit,
it will restrict that and it will put some of them in a queue or it will put some of them as deferred.
So you can rerun it there. I've, having played extensively with GPT-4 through the GPT for Sheets
plugin and the clawed for Sheets plugin, I can say that the response time from clawed is far and away
quicker than GPT-4. And that's the other parameter that you can change in the formula. You can change
the model, of course. You can choose clawed 2.1, which is their most powerful and capable model,
or you can choose the quicker clawed 1 turbo, which is very quick, but obviously not as powerful
and as capable as clawed 2.1. Even clawed 2.1, where I've had some quite long inputs with quite
detailed outputs, I've still found that the error messages are not an issue. Whereas I've got a
project that I'm actively working on at the moment and I've had to switch off GPT-4 with GPT for
Sheets because it errors on at least 20% of them and it's just not, you know, not, you can't do it,
right? Yeah, it's not usable. Yeah, that was the reason for the question, because that was a pain
and we've talked about use cases of this in terms of setting up customized prospecting emails at
scale for hundreds, if not thousands of prospects where you're able to find a way to get ChatGPT
to customize writing the outbound email and a bunch of other stuff. And I think as clawed
2.1's writing output is pretty high quality, maybe not quite GPT-4, but not far off, I can see why
that would be attractive. And if it doesn't hang all the time, which basically was making
the GPT plugin for Sheets unusable, let's be honest, that sounds like a big step. Are you
paying for the API to access this and how much does it cost to do these types of things?
So at the moment, you only start paying for API usage when you upgrade to a production ready.
So anybody can sign up to Clawed API with Anthropic and get a free account and you get 2000,
was it? No, was it? Two million or two and a half million tokens per day or something before you
have to start even looking at a free version. So if you're using it for yourself, you're paying
for it. And if you're using it for yourself in these types of experiments or even for internal
processes, you're probably just going to be able to operate for free. Correct. Yeah, certainly for
the time being. Interesting. It's time for me to go installing this again and see if I can get some
of the cool stuff I've been trying to get done over the last 12 months for some of these plugins
working because I gave up because they were hanging all the time. But if I was, I would say
there's actually a guide from Clawed and Anthropic about prompt engineering specifically for Sheets
and they've got a whole Google Sheets workbook with exercises that you can work through and do it.
But just on the quality output piece, if you are looking for those GPT-4 level outputs, assuming
that the output you're after isn't a thousand words long, if it's, you know, 100 words or so,
maybe it's a summary of text, take a GPT-4 summary or take a few summaries and give that as a,
give that as part of the prompt. And Clawed too is then getting few shot prompt trained. Right.
On GPT-4 quality outputs. And it does a very good job of matching that thereafter. So if you add it
in your prompt, you're basically saying this is what good looks like. Apply this to your outputs.
Interesting. Well, there we have it. Tool of the week made a comeback. If you are trying to figure
out some next level automated uses of chat GPT and Clawed and other large language models until
agents make their appearance, hey, next week, next month, who knows, then these things are good to
play with. Martin, it's always a pleasure hanging out with you. Thanks for your time today. Good
conversation. Yeah. Looking forward to the next. Take it easy, buddy. Speak soon. Thank you for
listening to Artificially Intelligent Marketing. To stay on top of the latest trends, tips and tools
in the world of marketing AI. Be sure to subscribe. We look forward to seeing you again next week.