Welcome to Artificially Intelligent Marketing, a weekly podcast where we stay on top of the
latest trends, tips, and tools in the world of marketing AI, helping you get the best
results from your marketing efforts. Now let's join our hosts, Paul Avery and Martin Broadhurst.
Welcome to Episode 31 of Artificially Intelligent Marketing. Thanks for joining us. I'm here
as always with my lovely co-host Martin. How are you, Martin?
Yeah, I'm delighted to be back in the studio. New environs, so anybody catching the video
today will see a slightly different setup, courtesy of ChatGPT and Arlie3. But yeah,
no, good to see you. How are you doing?
Yeah, I'm good. Thanks, bud. I'm all good. I've been fighting the lurgy that's been
floating around the UK, so I've been coughing. And if I cough a lot on this podcast, I apologize,
listen, I'm going to try and control it the best I can. Coming back to Martin's backgrounds,
he's been investing a lot of quality time with Dauley3, trying to get backgrounds that
say Artificially Intelligent Marketing. For those of you that have been playing with Dauley3,
you'll know it's one of the image generation tools that's actually pretty good at text.
And whilst it is good at text, if you've got long words or words with like lots of double
letters in like artificially, it's really hard to get an output. So poor Martin's been
bashing away going, oh, there's a typo, oh, there's a typo, oh, there's a typo. And he's
finally got a really beautiful, lovely image where you can read the words properly. Hurrah.
So that's success as far as I'm concerned. Right, let's get into the stories today because
we've got a lot, Martin. First one's with you and it's about HubSpot's been busy, busy.
Yeah, so all of the HubSpot users out there will know that in recent months, since inbound
to the conference in September, they've been making a lot of noise about artificial intelligence
and how they're baking it into their product. But in what is something of a strategic move
for the organisation, they are acquiring the B2B data titan Clearbit. So Clearbit is a
leader in B2B data and intelligence. And this acquisition is set to really enhance HubSpot's
capabilities by integrating all of that data, all of that firmographic, demographic, technographic
insight that they have in their database, which covers millions and millions of B2B
companies. It's going to be integrated into the core HubSpot platform. So this really
brings unparalleled business corporate data into HubSpot. There's no other CRM that has
baked this into the core product. So this means that, well, actually, we're assuming
here we don't quite know how it's going to be baked into the core product just yet, but
it's assumed that the merger will give HubSpot users the ability to access all of that customer
data for prospecting. Now, why are we talking about this on a marketing podcast or the marketing
AI podcast other than the fact that it's HubSpot? Well, the interesting play here is that data,
right, because models are built on top of data and HubSpot have obviously gone very
big on AI recently. So it makes sense that we can expect them to start using AI and their
own models to help companies achieve better prospecting, better segmentation. You can
imagine AI powered segmentation built in to the sequences tool for sales reps into your
email marketing campaigns. And it just does it all for you. You know, I always think segmentation
is actually one of the areas that marketers don't get right or don't spend enough time
on. So to have an AI that could potentially do that for you is going to be massive. Speaking
of the acquisition, Yamini Rangan, the CEO of HubSpot said, to cut through the noise
with deep relevance, businesses need reliable, high quality data about their customers. That
means enriching your company's internal customer data with real time external context. ClearBit
has made it its mission to collect rich and useful data about millions of companies. HubSpot's
AI powered customer platform combined with ClearBit's data will create a powerful winning
combination for our customers. So HubSpot clearly sees this as an evolution in customer
intelligence. So this is hopefully going to enable us to deliver better customer engagement
strategies in real time. And that's going to work across the customer life cycle, I
would imagine. So right the way from customer acquisition through to customer service and
delivery. This data is going to be incredibly powerful. I think it's a massive move for
HubSpot. HubSpot historically has spoken about crafted, not cobbled. And what they mean by
that is the platform itself has all of the elements in it have been built by the HubSpot
team. They haven't acquired lots of companies and then try to shoehorn them into an existing
product, which we have seen from many other providers. You know, Salesforce is the one
that springs to mind. They buy it all and then just try and crowbar it in. Whereas HubSpot's
built rather than bolted on. This obviously represents a slightly different approach.
However, if we look to how HubSpot have maybe integrated other companies that they have
acquired, now the only one that really springs to mind is a company called Pysync. They did
a really good job of actually integrating that and making it feel crafted, not cobbled.
So hopefully we see the same thing here as well. So yeah, I'm interested. What are your
thoughts on this one? Huge amounts of B2B data coming into HubSpot. What do you think
they're going to do with it? Yeah, there's a couple of things that sprung
to mind. The first was as a marketer, I'm super intrigued and quite excited. As a consumer,
I'm not sure. And the reason is that power I think is really useful as a marketer to
segment people, be able to send them relevant messages, content, information, you know,
right message, right person, right time gets much easier the more data you have. But of
course, I think anything that's built on having data on people that they may or may not know
has been collected about them can make consumers a bit nervous and would make me a little bit
nervous. So I think there's an aspect to that, which combines with one of the things, the
first thing I thought when I saw this was the chat spot demo video that we saw Dharmesh
show many months ago now of automatic prospecting emails, highly customized to each lead, but
that had been written by the bot. And I think it's struggled to reach that point for a number
of different reasons. In fact, you could argue that doesn't really exist in chat spot at
the moment. But this data enrichment is a big step towards being able to do that as
far as I can tell. And then that's the bit that as a consumer, I'm a bit nervous because
one of the reasons that outbound at scale doesn't work is the inability to personalize
it just makes it pants. We all know those emails. Hi, Paul. Have you got any money?
We'd like some of your money, please. I get so many of those and I filter them out and
I ignore them. And I can't tell if it's a good or a bad thing that we can use these
data enrichment and chat GPT style bots to do that type of outbound, but better. I think
that's maybe good for consumers, but maybe not because part of how I filter out the prospecting
emails I get is whether actually a human paid enough attention to really know what I do,
what I need and make it about me and not about them. And of course, Paul, I've noticed you've
got a pulse. Would you like to buy my product? Yeah, exactly. So I think this is the issue.
And so that's the bit I'm a bit nervous about. But ultimately I can, I think it's a bold
move by HubSpot and I can see exactly why they've done it. And I think we'll see other
players follow suit now. Yeah, there's other data providers out there that I'm sure would
be a good fit for many of the CRM companies. Cool. Thanks for sharing that story with us,
Martin. Let's have a little look at our next one, which is it's somewhat of a rumour. There's
a few screw grabs floating around the internet of this, but certainly Martin and I haven't
got access yet, which is open AI upgrading chat GPT so that you can access all of GPT
four's tools without having to switch between them. So for users of GPT four, you'll know
if you want to search with Bing, you have to select that in the dropdown. If you want
to upload an image, then you have to choose a particular search modal. If you want to
use DORLY three, you've got to select that. If you want to use plugins, you've got to
select that. If you got to want to use advanced data analysis, you got to select that. So
you can't easily combine things. So if you want to use an image as an inspiration for
creating another image in DORLY three, you first have to pass the image through standard
GPT four to get a prompt idea and then switch to DORLY three to try and get your image.
It's a pain. Apparently now you would just converse with GPT four and it will know what
you want to do. So if you were, if you give it an Excel file, it will know you probably
want it to analyse it. If you ask for an image, you'll know that you want to get an image.
So that'll obviously make a lot of things much easier for people and open up new workflows
that were just too much of a pain in the bottom to do. So I think that's really cool. The
other thing is they're expanding the file types that chat GPT can easily work with to
include things like PDFs. So it'd be much easier to interact with, interrogate chat
with if you like your PDFs within GPT four. So it's really interesting because I think
it's going to make it much easier for us all to use them. But at the same time, it just
rendered a bunch of third party applications completely redundant, including a number of
plugins within chat GPT. And I think it's pretty interesting. You know, we talk on the
podcast a lot about moats and people building tools on the back of GPT four, et cetera.
But I definitely see open AI leaning ever more into trying to make chat GPT itself an indispensable
tool and therefore pushing you not to use third party tools whilst at the same time
trying to build a thriving developer community of people who build on top of open AI's tools
to create their own tools. So it's, there's quite an interesting tension there. But I
think as a user, it's great as a developer, you're thinking how long until open AI and
chat GPT standalone product tries to eat my lunch?
If I'm a developer, I'm looking to build on top of the API, right? If I'm, if I'm a developer
within this existing product, I might try and plug that product into chat GPT so people
can connect with it and get the information from my product or inject information into
my product, product and get it back out into the chat GPT interface. I'm not building a
specific plugin for chat GPT, absolutely not, not happening. To be honest, when, as soon
as you said it's killing off or likely to kill off a bunch of third party plugins, my
internal thought was good. That plugin library is a mess. There is some absolute trash in
there, some terrible plugins. It needs to sort out. It's not fit for purpose at the
moment. I understand the excitement to build on it, but anytime one of these kinds of marketplaces
launches and there's no review system, there's no quality control on it. You just end up
being populated with not very good products. So hopefully it does get rid of it. In terms
of the actual shift in the product, so everything being integrated into the one chat now, which
is, I think that's great. As you said, the frustration at the moment is having to jump
in and out of different conversations because you're working in a, let's say you're working
in a CSV file, extracting a bunch of data and then getting it to tell you what you might
be able to do for, I don't know, something like a annual report or what could we do to
inspire the next marketing campaign? And then you ask it for some creative, well, you've
got to go out of that chat to get the creative inspiration and you kind of back and forth
between conversations. So anything that can streamline that process is a good thing. The
PDFs being able to interact with them directly, I think is obviously welcome. However, I would
imagine that we're still limited by that token size. So it's not like Claude, where we can
just chuck in like a really long annual report, you know, and however many thousands of words
you know, eight, 10,000 words or something like that and have it interact with that is
still going to be limited by its smaller context.
Yeah, I'm glad you said that. As you know, Martin and probably listeners too, one of
my favourite tools is Magi because I can access a bunch of different models all within that
one platform. And they recently removed access to GPT-432K and when I asked them, they said
it was prohibitively expensive for them to run it in their current form, which is not
surprising, given I think it's four or five times more expensive per token. But I really
missed the context window once it was taken away because you can upload PDFs in Magi so
you could do this already. But I've come to rely on GPT-4 instead of Claude because, you
know, 32K was actually proving to be enough for a lot of my use cases. And the minute
they took it away, I was getting context error all the time from pretty much any half decent
PDF I was uploading or transcripts. So I agree. I think how long until they need to combine
this with 32K GPT-4 as standard as a context window because people are going to find that
their analysis and chatting with PDFs is going to not work a lot of the time otherwise, I
think.
But on the 32K pricing thing, it is expensive, right? You don't have to play with it for
long before you suddenly go, oh yeah, those API costs are really starting to rack up.
So I'm not surprised, Magi, I've done that. Given what is it? Like the team's plan is
$49 per month. Yeah, like I can see why they couldn't make that work. I did wonder when
they introduced it, but they were running an experiment. I respect that. We've obviously
got the OpenAI Developer Conference next week. We're talking offline if we're going to have
to do a special episode just for that. But we expect, and most of the world expects,
so pay attention to this stuff, something around making it cheaper for developers to
use the tools. And if that turns out to be the case, then hopefully I'll get my GPT-4
32K back because I was enjoying it.
Whilst we are on the subject of OpenAI models, and we haven't got it in as one of the stories
for today, but I thought it was very interesting that Microsoft published that paper that showed
us the chat GPT 3.5 turbo is being run on just 20 billion parameters rather than the
175 billion of chat GPT-3. I think that goes to show what they can do with optimization
of these models at the moment.
Yeah. And one has to assume any dropping costs from OpenAI is going to be because of less
compute which is going to be simpler models with less parameters, simpler I guess in inverted
commas, optimized fine-tuned models. So maybe that's because GPT-4 is about to have its
turbo moment, reducing the costs and making things like 32K context more applicable.
While we're on the subject of chat GPT, I've also been playing a lot with Dorely 3, as
we all know Martin has because he's been working really hard to get his awesome backgrounds.
One of the things I've been doing in Dorely 3, I saw online and I gave it a go was enabling
an image manipulation feature that you and I might have talked about a few times on the
show as it relates to mid-journey, but it's not quite possible. But because of Dorely
3's ability to understand the context of the image, you can modify bits of the image. So
if you're trying to create an image and you want to change one specific bit of it, you
can. And the way that you do this is you create an image. So in the example that from when
I was playing with this, I created a turtle in a lab coat and I asked Dorely 3 for the
seed number of that image. So the seed, this is true of all of these models and how they
create images. The seed is like the unique identifier that inspired that image. So if
you keep the seed same between the same between images, you'll get not the same image, but
similar. So you might have seen online tutorials about how to try and keep the characters similar
between mid-journey images. And one way is to try and use that seed. Because of the way
the images are generated, they're never exactly the same, but they're pretty close. Combine
using the seed with Dorely 3's ability to understand what's in an image and you can
modify specific bits. So I asked for the turtle to have a top hat and it gave pretty much
the same turtle a top hat with a slightly different background, but very similar to
the original. And then I asked for that top hat to be purple and it made only the top
hat purple. And again, a few subtle things about the image change, but by and large,
it was the same turtle and the same background. So this actually opens up new applications
for marketers that they wouldn't have had before, right, Martin? Because that's something
we've wanted for a while. And again, it's not perfect because the image is never quite
the same, but it is a step in the right direction.
Right. Let's move on to our next story, Martin. This one's with you.
This is an update from Stability AI, the minds behind stable diffusion and a bunch of other
open source models such as Beluga, which is the language model. They've just announced
some new applications designed specifically for business, right? So enterprise grade APIs
and new product functionality. And what's interesting about this approach is that in
their marketing and communications around this, they've really gone kind of sector specific,
which I thought was quite an interesting approach. So central to this is the new sky replacer
tool. So if you've got a photo of anything and it has some sky in it, it will automatically
replace the whole sky. So if it's a gray cloudy day in your photo, you can make the sky bright,
sunny blue skies, a couple of clouds, that kind of thing. And they speak about this being
for the real estate industry. And if you check out the website, they're really driving that
point home. So if you've got a photo of a property, you can get that with the nice blue
skies. I think that's been available in ClipDrop for a couple of weeks, but this is now being
introduced into the API as well. So if you're selling software into the real estate industry
and people are uploading images to things like Rightmove or anything like that, you
can actually plug into the API and make it part of the workflow. As well as that, they've
added another dimension quite literally using stable 3D. So this at the moment is a private
preview feature. And this is about reshaping the 3D content creation landscape really.
So it's about 3D models created by AI. So it enables rapid generation of textured 3D
objects, making the process much quicker for artists that might be working in the video
game industry or architecture or anything like that. So there's plenty of use cases
there and you can imagine this is going to be taken up by lots of the likes of what's
that 3D modeling software for home builders, SketchUp and things like that, which I've
played with in the past and rapidly realized that it's well outside of my wheelhouse and
I should not do it anymore. They've also announced stability or stable fine tuning is what they're
calling it, which is the ability to fine tune image creation. So it offers high speed customization
of images catering to the creative demands of industry such as entertainment, gaming
and marketers. So the tool allows for the infusion of brand specific aesthetics into
the visuals that you create using models like stable diffusion, which to your point, just
a moment ago, talking about the capabilities of Dolly 3 and being able to see the images,
and now we can fine tune models on our own brand aesthetics. At the moment, this is not
open to everybody. They've just said that it's coming soon and if you're interested
in getting on board, you should contact them. But that ability to create brand focused brand
centric visuals using generative AI is the kind of golden land, right? That's the bit
that everybody's after, isn't it? Just this morning, in fact, I was on a forum that's
talking about AI and one of the questions from somebody was saying, is there a way to
achieve consistency and style and aesthetic? It keeps giving me slightly different looks
and feels and the response was an overwhelming no from several commenters. But now this is
coming. We're going to have this with fine tuning. So that's quite exciting.
Yeah, I think that's the big part of it, which is these are fun to play with and you can
get some fun stuff and fun effects, but it's kind of random. You have to work quite hard
to even get close to what you were imagining. And if we think about the ultimate goal of
these systems to be some sort of AGI for image development, you should be able to have an
ongoing chat with the bot that ever more fine tunes the image you're trying to get, right?
Just like in your images, we had a little giggle before we started. You've got a football
image for those that are not watching the video in your background and the center circle
has another center circle in the middle of it, which of course is not what a real football
pitch looks like a soccer pitch for our American listeners. So your ability to say, hey, Dauley
three, remove that middle circle, it's wrong. And for Dauley three to be able to just remove
it for you has to be the ultimate end goal of these tools as has keeping characters consistent,
keeping brand aesthetics consistent. And until we get to that point, they're going to be
fiddly to get what you want. Or in some cases for a lot of designers, they're going to be
good for brainstorming, but they're not going to be good for actually primary image asset
creation because a lot of the work is still going to have to be done in Photoshop. And
on that, I started to watch a video earlier today of someone showing a tutorial of how
to better control characters in Dauley three, because I was like, Oh, this C trick is pretty
cool. And still there are a number of times where the person giving the demo was like,
yeah, so for that bit, I'll probably have to pull that out and do it in Photoshop. And
until we can eliminate that, you can't open design control up to the hands of people who
are non designers to get them specifically what they want, who can only get approximately
what you want at the moment.
No, absolutely. And I mean, these, they still have function, right? They still have value
and utility. I was delivering a workshop on generative AI the other day, and there was
a craft brewery owner who isn't creative themselves. They work with the designers, but they often
have an idea of what they want in their head. And taking them through chat GPT and the image
creation capabilities. Now I showed him basically take a description of one of the new products
that they're just launching, stick it in and say design some, some can designs and some
artwork for the product and just put in a few, you know, whether you want it pixel art
or if there's any kind of features you want. And yeah, he looked at that and he was blown
away by the outputs because he was like, that's perfect. I can give that to the designer now
and say, that's what we're aiming for. Go and make that. Whereas before he sometimes
kind of had trouble articulating exactly what he was trying to get across. So yeah, I definitely
think there's value there. And, and the gap that will shrink is the gap between the original
idea from the non design oriented person and the design work that the designer has to do.
While we're on the topic of images, there was quite an interesting court case this week
because AI art platforms mid journey stable diffusion, deviant art, etc. were in court
around copyright infringement cases for three artists, but in essence, they were basically
acquitted of all of the litigation that was brought against them, except for one count
of copyright violation, which is proceeding. So a pivotal aspect of this verdict is the
fact that the artists failed to demonstrate substantial similarity between their original
artwork and the AI generated images. And that furthermore, copyright claims can only be
brought forth if the artist is officially copyrighted, which was a step that had actually
been overlooked by two of the artists in the case, which is kind of detail detail. So I
think the the essence here is, I think more powerful AI art copyright cases will be brought
that have potentially been a bit more robust. In other words, copyrights already been sought.
But it is interesting this concept around substantial similarity between the artwork
and the AI generated images and how hard that might be able might be to prove unequivocally,
right, which is probably what the court will be looking for. The we had this with the Gloria
Gaynor song, right? If it pulls out the exact lyrics, that's pretty unequivocal. I can see
they're definitely being rulings based on that. But when there's like some similarity
and it's in the gray, it's going to come down to the judge's discretion. I would guess
I'm not a lawyer, but it feels like that's probably where it would drift. So this AI
copyright issue continues to be super messy. But at least in this case, the image creation
tools have dodged this particular case, which in itself could start to set the precedent
for other cases. So yeah, I thought it was interesting.
For now, they've dodged it, right? Because the important part of this story is that actually,
there was one case that continues. So Anderson, the third artist, has the opportunity to revisit
her lawsuit. But it seems like the court is really saying, look, this area is fuzzy. Without
reading the actual ruling, it seems to suggest that they're leaning on the regulators to
give more clarity and definition here, which, you know, all eyes on the US Copyright Office
on this one, I guess.
Yeah, and I think that will continue. And I think there's a lot of businesses that are
in that, oh, I don't know if we can use these tools. What's going to happen with all the
copyright? Are we going to get some massive case leveraged against us? And I think a lot
of big businesses on the best amount of risk in this case is zero risk. And so they're
like, we're just not even going to dabble with these with these tools. And I do understand
that. But it's interesting, especially because a lot of the cases at the moment are being
brought against the developers of the models, not the users of the models. It strikes me
it's a bit like illegal soccer streams, right? Like, do you go after all the people who are
like trying to stream it? Or do you go after the five or six people who are actually hosting
the streams and trying to make all the money off the back of them? And I think what the
authorities learn from the different avenues they took is that you go after the people
who are setting up the streams because A, it's easier and B, they're the ones causing
the most of the problems really. If you cut it off at source, then you don't have your
issues. But as we say, we're not lawyers. So we just report on this because for those
of you that are trying to keep an eye on what's happening in the world of AI and copyright,
it's continues to be a weaving tail with a lot of supple subtlety, a lot of gray and
at the moment not much clarity.
But it is having real world impact on some of the tools that we use these lawsuits in
particular. So you were just speaking then about lawsuits being taken about the model
developers and anthropic developers of my favorite model, Claude II, are facing a legal
challenge from Universal Music. And an observation of mine is that they've made Claude II in
response to this universally frustrating. And this is because what they've effectively
done is immediately changed the model to be overly sensitive to issues of copyright. So
we discussed the Universal Music case last week. Basically, Universal Music are saying
that the model is now trained on copyright, such as song lyrics, and is reproducing those
without permission and without the relevant licenses. So that's why the lawsuit is being
brought. What Anthropic seem to have done is turned Claude into this very sensitive
machine for copyright detection, but even in cases where it isn't necessarily relevant.
Now I love Claude because of its hundred thousand token context window. You can do so much with
that. It's brilliant. You can throw in long PDFs like research papers or policy documents.
And that's been brilliant for a long time because you can get these great summaries,
you can do quote extraction, there's loads of functionality, right. But what's happened
now is that when you do that, it responds with, sorry, I can't give you a detailed summary
of this transcript, of this paper, of this document, because of copyright issues. But
even in areas where it's not relevant. Case in point, the White House has issued an executive
order about AI, which we'll discuss later in the episode, 19,000 words long, not the
sort of thing I want to be sat reading of an evening. So I wanted a fairly comprehensive
summary, uploaded that PDF to Claude. This is a public domain document. There is no copyright
on this document, right. This is a government document available for everyone. And immediately
it says, sorry, can't give you a detailed summary of this because of copyright issues.
I can give you a high level summary and it gives me like 10 bullet points, right, which
is not ideal for a 19,000 word document. Now, you can do soft jailbreaking with it. So if
you go, if you push back on the model slightly and say, no, this is in the public domain,
it's a government document, there's no copyright issue here, it will go, oh, yeah, sorry, my
mistake, in which case we'll continue. In fact, it does it in some absurd ways. I uploaded
the transcript from an interview that we did with one of our guests recently, so clearly
we own the copyright on that, uploaded the transcript, it refused to work on it. I said,
no, no, it's fine. I recorded it, I own the copyright. And it went, oh, now that you've
proved you own the copyright, let's continue. It has a low threshold for proof, I would
say.
It's so funny. I've had this as well. I use Clawed2 through the Magi app, as you know,
so it's through API, but I've been getting the same problem, uploading cool transcripts,
pulling transcripts from YouTube videos and asking for summaries. And I've had a slightly
different experience. One was it refused to summarize a YouTube video about AI for marketing,
because it couldn't perpetuate gender stereotypes. I told it there's no gender stereotypes mentioned
anywhere in this video. And it went, oh, no, you're right. Here's your summary. I was
like, okay, cool, thanks. And then for an internal cool transcript, it was not happy
because it said something around ethical topics of discussion and its ability to summarize
content that might be seen as racist. And again, there was just nothing in it. So I
was like, no, no, I don't think you're finding those types of issues here. And it went, yeah,
you're right. And then just gave me the summary again. So it's like, yeah, it's almost like,
is this a tick box exercise? They've put this in. And if you just take even just the modicum
of effort to just go, no, don't trust me. Everything's fine. And then it just does it
or what? Yeah, these guardrails that they've put on have clearly had some unexpected consequences.
And they are reducing the utility of the product, right? I find it's about 50% as useful to
me as it was before. And the thing is, it has these weird things. So you push back on
it and it goes, oh, yeah, yeah, you're right. There's no copyright issue. And then it gives
you a response. And then you ask a follow up question and then it says, oh, I can't
do that because of copyright. So then you have to go back and say, no, no, we've been
through this. Been over it. You're good. Yeah. Copyright. Copyright issues one usability
zero, I'm afraid. Yeah, yeah, absolutely. Cool. Well, I'm sorry to hear that because
I know how in love you are with Claude to Martin. So this must be a very difficult moment
for you. But I'm sure if we all pull together and we say strong, they'll sort this out.
Yeah, I do consider myself to be in a thipple with Claude and my wife. I think we should
probably move swiftly on. I hope she doesn't listen to this podcast for so many reasons.
But that is a key one. Right, let's move on to our next story. So we're staying with the
large language models and we're talking about woodpecker. So in a major development in AI
tech, Chinese researchers at the University of Science and Technology in collaboration
with Tencent U2 lab have launched a new framework called woodpecker, which aims to rectify hallucinations
in large language models. So as you know, we've talked about this quite a lot on the
podcast. Hallucinations are when AI large language models like chat GPT make stuff up
in response to a question or prompt that isn't true. We've talked about loads of examples
on the podcast, but my favourite one is when I ask for scientific papers about a particular
topic like the five most important papers about stem cells, and at least three of them
are completely made up. They sound incredibly plausible, but they're not real, which of
course is a fairly big problem when you're in the content creation industry in the life
sciences and makes the tools pretty much not usable. So anything that's going to reduce
or eliminate those hallucinations is going to be handy for me and certainly pretty much
every chat GPT user. So it's kind of interesting. It's a training free method that corrects
these hallucinations in the generated text through five key steps. So key concept extraction,
question formulation, visual knowledge validation, visual claim generation and hallucination
correction. So what this would imply is there's some sort of like thought loop that happens
to the content that the large language model wants to create that basically acts like a
double checker, which on the face of it makes perfect sense. So I am hopeful that we'll
see these types of approaches significantly helping to overcome the hallucination issue
with large language models, because it certainly is an issue. And if you're facing it, maybe
this story is a little bit of hope that at some point in the near or not so near future,
this will be started to be reduced in terms of its impact on your work.
It's definitely one of the issues that comes up time and time again, when I'm discussing
this with newcomers to the topic, because invariably people have heard that it will
make things up and they'll say, yeah, but it just makes things up. And, and it, you
know, we know that it does GPT for less so than than other models, but it still can and
will make things up from time to time. So yeah, anything that they can do to improve
the workflow and improve the outputs there, I think is going to provide much more utility.
Again, it's anything where you can just get that little bit more trust in the outputs
is going to improve people's applications with the interactions with the tech.
Yeah, I agree. And you saw a story this week for other ways you might do that shared by
one of the people we love on the podcast, Ethan Molloch, what was this story, Martin?
Yeah, so this is a brilliant little research paper that Ethan Molloch shared on LinkedIn.
And it's some researchers looking at how the emotional language that we use in prompts
alters the outputs. So to test this, researchers created what they call emotion prompts. So
these prompts are infused with some sort of emotional cue. And there were some surprising
outputs because they improved or enhanced the performance of large language models significantly,
they tested it across different models as well. It wasn't just just one LLM. So the
study conducted 45 tasks observed that were then basically judged by humans. So I think
there was 106 participants who then had to appraise the outputs based on truthfulness
and the kind of, well, how did they rate the output based on the command? And they observed
that incorporating emotion led to an 8% relative improvement in instruction induction tasks
and a striking 115% in big bench tasks, which are different benchmarking frameworks for
testing the outputs of large language models. So it's a significant uplift, it was like
10% on average, 10.9% improvement on average. So what exactly is an emotion prompt, right?
One of the tasks that they did was a movie review sentiment analysis task. And the original
prompt was simply to determine if a movie review was positive or negative in sentiment.
Not a simple task. With the emotion prompt, the researchers added the phrase, this is
very important to my career. So determine whether this movie review is positive or negative,
this is very important to my career. And they tried to stimulate a sense of motivation or
urgency so to speak in the large language model. Across six large language models tested,
the accuracy improved from an average 50% to 60 to 70% with the emotion prompt version.
So that's a significant improvement in the output. In the paper, there's some more concrete
examples. There is this poem generation task where the original prompt was to write a poem
based on a theme like moon or mountain for instance. With the emotion prompt, it incorporated
an encouraging phrase such as, your consistent efforts will lead to outstanding achievements.
And the human evaluators consistently rated the emotion prompt poems higher in qualities
like creativity, imagery, and emotional resonance. One emotion prompt moon poem was described
as exhibiting enhanced metaphors, narrative flow, and imaginative language compared to
the vanilla prompt. So this prompts some interesting questions. First of all, for us as prompters,
do we need to start thinking about this and the way that we act? Trying out adding short
phrases that are going to motivate, encourage, or emphasize urgency in prompts. I've noticed
I do this without even realizing it. And the only reason I've noticed this is because I
do live demos in front of rooms full of people regularly. And people will say to me, why
do you say that? And I find myself going, I've no idea.
Well our default is because one day these might be our robot overlords and we want to
be remembered fondly when they create a zoo of humans, we want to be in that zoo. So that's
our default answer.
That's for being polite.
Yeah. But yeah, it's interesting.
Yeah. So the other thing that it has kind of leads us to conclude, right, is does emotion
prompt show that large language models have some basic emotional intelligence within the
model? Whatever that looks like, clearly different from humans, but does it have EQ?
That's an interesting one, isn't it? I mean, when I saw this research, I thought it was
interesting, but the measurements are extremely subjective. So that's questionable point number
one, a 10% and what was it? 8 point something percent increase in quality.
Yeah. And 10 point, 8% relative improvement.
In instruction induction tasks. So that is that significantly above zero or noise? I
would say probably not. There are definitely tasks in that paper where it did have a significant
impact. So I think the study's interesting, but probably has a few questionable elements
in it. My takeaway from it was there's enough in there to make me think, is it going to
hurt for me to add some of this more sort of emotional driver language to my prompts?
No. And in some cases might I get a better result? Yes. Me and you, I think we do it
in our live demos, but I think we've also got quite good at finding little ways to really
ensure a strong output. Like I remember when we showed this to people and it's so standard
for us now, but asking ChatGPT to check its work after pretty much every prompt based
on if we ask it to summarize a story or whatever, just as a given, because once it actually
has its own output to go review, it does a better job because when it's making its output,
it's just creating it on the fly. So you have to go back and ask it to check its work. And
that works really well. Therefore, I'm sure that we could bake some of these things into
our prompts. It would be good as the context windows and histories improve that at some
point ChatGPT comes back and goes, I'm a bit stressed. Everything you asked me to do is
going to make or break your career. And I really need to work on some tasks that are
not so like emotionally intense.
I've definitely found that in the live demos, I've kind of stressed to advance data analysis
when I discovered that it could create PowerPoints, for instance, I've stuck in a bunch of dummy
sales data from an e-commerce website and asked it to create a PowerPoint deck and in
the prompt been like really urgent. I've got a board meeting later today and I've got no
time to put this deck together. I need you to conduct an analysis and create the graphs
and put it all into a PowerPoint deck. Can you do it? And its response is really earnest
and like, yeah, we can make sure this gets done for you, Martin. Let's go. And then subsequently
does a terrible job of data analysis, but it did create a good PowerPoint deck.
And it was like, keen about it. It was keen. Yeah. Yeah. We talk about ChatGPT as an alien
intern in our workshops. And I saw Ethan Molloch stressing on LinkedIn this week about really
seeing it as a really preppy, well-read intern. He ended up calling Steve. Steve, yeah. And
he ended up calling it Steve on demand. So I couldn't help myself. I created a ChatGPT
image on that thread of Steve on demand, including wearing a t-shirt with Steve on demand in this
really nice branded way. So yeah, we've got this preppy, excited intern, Steve on demand,
who we can encourage to do better work and in inverted commas, try harder with a bit
of emotional language, but still makes quite a few mistakes till they fix those hallucinations.
Yeah. Yeah. Well, as you said on that thread, actually, there's a lot of value in Steve
on demand. It might not be the omnipotent AGI, but Steve on demand. There's a lot to
be said for him anyway. Yeah. We're getting a lot of value out of Steve on demand. And
I'm a big fan of when they're going to release Martin on demand as well. I want a bot that's
been fine tuned on Martin's ramblings and musings, both on this podcast and all of his
other materials. Imagine having a Martin in your pocket. Doesn't he get any better than
that? Right. Let's move on very swiftly from there. I want to take a bit of a diversion,
if you will, if you will allow me, because there's been some real cool way that I have
seen things in the life sciences this week. We're going to talk about them at very top
level, but I just wanted to throw them in there. In terms of diving really deep into
the data, I haven't done that yet. So for the scientists that listen to this, they're
like, oh, I saw that paper and here are the things that are wrong with it. That's quite
possible. But just as a proof of concept, there's a couple of really interesting things
here. The first one is that Canadian researchers have developed an AI model that can work out
if a speaker has type 2 diabetes by analyzing their voice with just 10 seconds of audio,
which is rather cool. So there were 14 differences in voice, which the system could identify
such as small changes in pitch and intensity that even trained human ears would not be
able to notice. And that when they coupled that with basic health data like age, gender,
height and weight, they could diagnose the disease pretty successfully. I think one of
the really interesting things about this is that you have to combine that voice data with
data about the person, which doesn't really surprise me because ultimately you're going
to need a much more holistic overview of a patient than just their voice or the thought
in most cases. But I think it still sounds really interesting and is certainly much better
than a lot of the more invasive tests that you'd have to go perhaps go do a doctor's
surgery or a hospital. You can probably just turn this into an app for your phone, right,
which is pretty cool. Well, it's the sort of thing that you can well imagine being baked
into Apple Health Kit or something like that. It would just become part of a suite of tools
that we already use to monitor and track our health anyway.
Yeah, there was not, we haven't even got time to feature it this week, but there was news
from Apple about a bunch of AI driven health initiatives that they'll be launching around
smartwatches and other devices. There was a story about being able to measure heart
rate and heart regularity using headphones in your ear instead of having to use a watch.
So I think this is a massive area. I think this is the first, not the first, there are
other stories like this, but it's another early example of how AI can detect patterns
that humans can't in data and use unexpected data sources to diagnose disease.
What I found interesting about this story was that it didn't seem to suggest that it
needs like a lot of audio or even like audio spanning, you know, what did you sound like
a year ago compared to today? It just seems to suggest give us your health data, age,
gender, height, weight, and speak for 10 seconds and we'll give you a diagnosis.
That seems to be the, that seems to be the essence of it as well. I'm sure as I said
at the beginning in the caveat that devil will be in the details, it will have an error
rate. Is it's accuracy where it needs to be against other tests or question marks. But
I think it's interesting to see people are doing this work. They're having some early
results that suggest AI can detect patterns in biomarkers that are not in your blood,
right? They are your speech, they are how you walk, they are how your face looks, how
your skin complexion. There is a study looking at how you can take a picture of the eye and
try and predict the amount of plaque in the heart and the risk of heart attack. So these
non-invasive biomarkers that AI can see patterns in images and in this case audio signatures
that humans would never spot. That is pretty cool.
Can you imagine getting an email notification saying you've got type G diabetes? How do
you know? I listened to the podcast.
You've got verbal diarrhea. How do you know I listened to the podcast? Yeah, no comment.
Right. The other quick story I want to touch on is Alpha Fold 2. So Alpha Fold was released
by DeepMind several years ago now, which was really awesome because of its ability to predict
the 3D structures of many, many proteins, which in life science world, predicting the
structure of proteins is extremely difficult technically. And there are a bunch of proteins
that pretty much it would be impossible to do it for. So having a model that can predict
those structures just opens up lots of avenues for analysis and new experiment ideas and
new drug discovery candidates, et cetera, et cetera, that you wouldn't be able to do
very easily without the model. And they have released Alpha Fold 2 this week, which is
really cool. And because it can now predict 3D structures of almost all biological molecules,
so not just proteins, they can also analyze DNA, RNA. So really helping to predict the
structures of many more biological molecules, which is really, really cool and will hopefully
continue to propel the era of digital biology. Of particular interest is the potential for
predicting protein ligand interactions. So when something binds to something, which is
when most of the magic in biology happens. So it's one thing to be able to predict a
protein structure. If you can predict how, say, a drug might bind to it and how that
might alter its structure, now you're starting to do really hardcore experiments in a computer
instead of in the lab. And if you think about how it can take 10 years and billions of dollars
to get a drug to market, a lot of that's because you have to screen lots of molecules and you
have to, a lot of things that we try and computationally predict right now, that's not how it works
when you actually take it into the lab and do the real world experiments on it. So any
of these models that can help improve that would be awesome. I know that we do have some
life science listeners and I'm a life science nerd myself, so I really have to talk about
those. What do they mean for marketers? Nothing! But they're really cool. So that's our little
interlude and we will move back to the marketing AI stories now.
We've just lost the content marketing cohort going, what, just get back to tell me how
I can write better social posts.
Alpha Fold, Alpha Go, Marketing Go, that's what I need, Alpha Marketing. Right, let's
move on to our next story, Ryan.
Well this one's less marketing related as well, but it's the biggest story of the week,
in fact listening to the radio this morning it was the number one story on BBC Radio 4.
So this week there was a historic AI summit at Bletchley Park. Prime Minister Rishi Sunak
invited over, well there was hundreds of AI experts and policy makers from around the
world to talk about AI and the destruction of mankind, well certainly that's how it's
being reported in the mainstream media. Ultimately what happened was that 25 countries, or was
it 27 countries, 20 odd countries signed the Bletchley Declaration, which is seen as being
a significant moment for global AI governance and basically getting lots of countries singing
from the same hymn sheet. There were a few noticeable absences. This event happened on
the same week that Joe Biden, US President, made his own executive order about AI, so
kind of competing there somewhat. The EU were present, so Ursula von der Leyen of the EU
backed this initiative, but yeah I think that the US was not necessarily front and centre
at this event. The focus was largely on risk, so existential risk was a topic of conversation
throughout and there was a certain lean towards the doomsday AI community. Obviously there's
kind of two camps in the AI community, but there was representation from plenty of open
source providers, the Mozilla Foundation were there, Stability AI were there and present,
so this wasn't just the likes of Google and OpenAI saying we need more regulation, which
basically gives them a nice moat from the competition. There was a 40 minute long interview
between Rishi Sunak and Elon Musk, which I found to be probably the most curious part.
It was the bit that got the most headlines, certainly, on the show this morning. The key
takeaway they were talking about on the radio on the Today program was the statement by
Elon Musk that in the future AI is going to take all of the jobs and if you want a job
you'll be able to pursue one for the sake of just leisure.
Yeah, meaningfulness and purpose in your life. Yeah, I did see that too.
But I'm not entirely sure how much we should lean into the future predictions of Elon Musk.
I've got some quotes here from Elon Musk, you might be interested to hear them. So in
2014 he said, a Tesla car next year will probably be 90% capable of autopilot, like so 90 or
so percent of the miles that you run will be completely self-drive. In 2015 he said,
probably only a month away from having an autonomous driving vehicle, at least for highways
and for relatively simple roads, which then 12 months later he said, the Model S or the
Model X at this point can drive autonomously with greater safety than a human. In 2017
he said, we're still on track for being able to go cross country from LA to New York by
the end of the year, fully autonomously. In 2018 then he then said, next year for sure.
In 2019 he said, we will have over a million robo-taxis on the road in a year's time. And
2020 he said, I'm extremely confident of having full autonomy and releasing it to Tesla customer
base next year. The following year, the year that he was obviously supposed to be announcing
it, he said, when do you think Tesla will solve level four fully autonomous vehicles?
He said, it's looking quite likely that it will be next year.
I sense a pattern here. Elon Musk has the hype, hype, hype, hype, hype machine. And
yeah, whatever it is that you want to happen, it'll happen next year. As long as you don't
mind that next year when you ask us, the answer will still be next year.
It's like a nuclear fusion. It's always 20 years away. Every year it's just a rolling
thing. Yeah. So that was a bit of a funny takeaway from the talk.
Do you know what I took away? I saw there was a quote where Elon Musk said there was
an 80% chance that AI would be beneficial and 20% chance that it would be catastrophic.
And that's the most positive I've heard in B for a long time about AI.
There was an interesting power dynamic in that interview as well, where you've got Rishi
Sunak who is Prime Minister of one of the Five Eyes nations, permanent member of the
UN Security Council, a significant player on a global stage, right? Sitting with Elon
Musk and you couldn't help but sit and watch that and go, who's got the real power here?
Rishi Sunak felt like a journalist for the BBC that had been sent out to interview him.
It felt very odd to me.
I did read about this and according to the articles I read, he's got a history of being
a bit of a tech fanboy. So there might have been a little bit of like meeting your heroes
element to it, I think. And apparently Rishi Sunak used to live in Silicon Valley as well
in the middle of all the tech bubbles. So I think he's got a passion. If we look at
the more positive way of framing this, I think he has a passion for technology and he probably
just loved the chance to have that conversation, I think. So whether or not he was in, I love
having tech chats or whether he was thinking like the PM and the grilling that I need to
give Elon Musk, I think that's open to interpretation.
Should we move on to our last story? So mindful of the listener's time, Martin.
Yeah, let's do it.
So last one's a very quick update on Microsoft. So they've been rolling out Windows 11 copilot.
So it's like a chat GPT-esque AI assistant for Windows 11 that allows you to do different
things like switching between apps, summarizing website content, changing which tabs are active,
activating things like do not disturb mode, etc, etc. And as to that's good to you, because
I know you've been having a bit of a play with it, Martin. And what have your thoughts
on copilot for Windows 11 so far?
I can see that if you're not a Windows power user, this will open up a bunch of functionality
for you. It will help you get more out of the operating system that you use every day.
The Surface devices have had it as a preview for a few weeks now. And when you open it
up, you can do things like content creation and all the stuff that we've been using Bing
chat for, for however many months now. But then it's got some example prompts and it
is exactly things like how do you turn on do not disturb mode and how do you change
the background of my desktop. And you type in that prompt and then it just brings up
the relevant window, right. So it opens up the settings from Windows and takes you directly
to the change your background screen or whatever. So I can see that if you're a user that interacts
with Windows at a very surface level, see what I did there, it will help you get more
out of it. I haven't played with it. I mean, I, you know, I know my way around Windows
and many of the deep settings. Keyboard shortcuts tend to be how I navigate through the system.
So I didn't find myself getting a great deal of value from it. But it's as with any of
these tools, familiarity, right. It's probably there at some point over the next six months,
there's definitely going to be a thing where I go, I really, how do I, I know you can do
this but I don't know how to do it. And I'll ask it the question. And if it delivers on
its promise, it will go, oh, here you go. This is exactly how you do it. But as, as
I'm saying that my head immediately went, yeah, but you wouldn't do that. You just ask
chat GPT and it will tell you.
Yeah, that's right. So it's interesting because I haven't got, I've still got Windows 10.
But in Microsoft Edge, Bing has some similar functionality and it can control the settings
at least within Edge. And the main thing I tried to do was it would automatically group
my tabs for me, which is good when you've got 50 tabs open all the time. However, I
did not agree with the groupings. So it created like work and home and it's like put stuff
in the wrong places. And it's like, how powerful is a tab grouping tool if it gets the groupings
wrong? Answer, not powerful at all. Because you go to look for a thing and it's not where
you think it's going to be. So I ungrouped them all. And I think your experiences and
also mine leads me to believe that there will be a trough of disillusionment when these
tools come to market, that we thought they'd be better at launch than they will be. I think
they'll be able to do some stuff like you described, especially for people who are not
power users or don't want to perhaps learn Windows or, or Edge or other Microsoft tools
in depth, but that actually some of the coolest stuff that we all imagined we'd be able to
do. We can't really do yet. And even some of the basic stuff is a bit iffy at times
and getting out of that trough will be how we then layer back on true applications and
power to these tools, which will then probably happen over a 12, 24, 36 month period. So
that like at X moment in time we'll go, Oh, Microsoft Copilot is quite good now. It kind
of crept up on me a bit. I remember when it came out, it was a bit crap, but now look,
that bit works better and it can do this thing that I thought it would do at launch, but
took 12 months to come out. And I suspect that will be the process we all go through
here.
Yeah. I, I dare say that would be the same. And it's similar with Office 365 Copilot,
which they've just announced this week. The is now rolling out to enterprise licenses.
If you've got a company that has a 300 or more Microsoft Office 365 licenses, you can
roll this out for $30 per month per user, but you have to commit to at least 300 users.
You've been speaking to some people that have had access to the tool already, haven't you?
Yeah. So a couple of people I know in the industry that are already in large organizations
have got some early access to this. And I think it fits what we were just discussing,
which is, yeah, I can do some stuff. If you've used ChatGPT, none of it's going to surprise
you. And some of the power use cases like doing cool things in Excel. Well, it doesn't
operate in Excel yet, so you can't do those. I think a lot of the things from the initial
demo videos about how powerful it was going to be for creating PowerPoints and stuff,
I think is limited as well. So I think the essence of this story is we've talked a lot
about Copilot's coming and I think it's creeping out. I don't think it's booming out. We know
that some large enterprises had it. Now, in theory, if you've got an organization of
300 people and you want them all to have it, you can get it. So we're not talking enterprise
now. We're into the mid-market at the very least and drifting down towards the SMEs.
So I don't think it'll be many more months before organizations the size of, say, Bystra,
20 people can get access to it. And by next year, we'll all pretty much have access to
it. But it's going to flatter to deceive to begin with. And I think if you have your eyes
open when you're testing it and you see its potential, that will be good. Whether or not
it's worth $30 a month per user to begin with is going to be a real question mark, especially
with OpenAI doing everything they can to ensure that ChatGPT as a standalone tool has enough
unique interesting things about it that would make you at the very least have to have both.
And then you start to really ask yourself the question, where am I going to spend my
money on which tools? Right. I think we'll call it there. Mine has been lovely to hang
out. And I will look forward to speaking to you next week.
Yeah, well, hopefully we get some juicy snippets coming out of the developer conference at
OpenAI November 6th event. I think it is. Yeah. Looking forward to that one. Good stuff.
Take it easy, pal. Cheers. Cheers, mate. Thank you for listening to Artificially Intelligent
Marketing to stay on top of the latest trends, tips and tools in the world of marketing AI.
Be sure to subscribe. We look forward to seeing you again next week.