If you would see the process, what Bing's does, this goes beyond anything you could imagine when it comes to image generation.
It's like every possible technology, different models, different weights, aspects, prompts, like, it's just fascinating.
Ladies and gentlemen, boys and girls, children of all ages, dogs, cats, robots, and everybody in between, especially you, poorly performing GPT-4, we knew you were up to no good.
This is HTTTA, how to talk to AI. I am your host, Wes the SynthMine, SynthMine Wes.
And as always, in a galaxy where gigabytes govern and gliffle algorithms generate our glowing futures, one gallant voice emerges, galloping through the grand delinquent,
gracing us with unprecedented gaiety in the world of artificial intelligence. Whom might you ask? Well, it's G herself, the insatiable go-to-go.
G, how are you this week?
Hi. Well, that was a lot of big words, I can give you that. Or, TaiGPT, right?
Yeah, I'm great. We have so much to cover, so little time, but let's just jump right into that.
I want a follow-up on the leaked document on GPT getting dumber, especially with Claude 2 coming into picture.
This is like, ooh, politics and drama, let's jump in. So you got a follow-up, right, for us?
I do have a bit of a follow-up, and its timing could not be more interesting.
The leaked document that we helped to share with you last week is still circulating out there, but there's plenty of takedown requests that whoever puts it up has been fielding.
But kind of validating some of the assertions in that, that GPT-4 might be getting a little dumber, there was a research paper that was just released that completely validates this.
It's kind of incredible. They've been evaluating GPT-3.5 and GPT-4 performance over time, and this is not just some guys.
This is Stanford University and Berkeley researchers conducting this study. If you want to just look at some of their accuracy figures here, it's just...
No.
Yeah, so I mean like, hey, is this a prime number? Think step by step and answer. Just in the period of a couple months, just like, tanking.
94 to 2% accuracy. GPT-3.5, though, however, in some instances seems to be getting a little better.
So, it's interesting. I think OpenAI, they got to watch out. Everybody is coming hard on the scene with some of these ulterior models that are open source, that reason better.
So, yeah, I mean, what do you think?
Well, looking at this prompt, right, the question what we also talked before that, if they have this system where depending on your prompt, which direction to go to, like a validation system,
I kind of don't get surprised by seeing just this simple prompt or question getting less of a good results.
So, that kind of also proves what we talked last week, I think. I would think, you know, I don't know for sure.
What I can't totally understand is it seems like it was tuned to take a simpler zero-shot prompt with some instructions before and handle it.
If you look at the top right of this chart for our viewers, our listeners will post this study in the newsletter and in the show notes,
but we're looking at a graph of accuracy solving math problems, which admittedly large language models are not set up to do inherently.
But the question is, via the prompt, is 17,077 a prime number? Think about it step by step and then answer yes or no.
In March, GPT-4 is nailing it, 97.6% of the tests, and then is accurate. And then now 2.4.
You know, a couple of things come to mind. There is the whole thing about privacy.
Then there is about reinforcement learning with real human feedback. Then there is just simple capacity and training GPT-5.
So, if you have to allocate resources just from simple compute, maybe that is if it is something like, OK, click a button, GPT-4.
We noticed that people do use with simple prompts because it was marketed as a chatbot. So, you should be able in a way to just go with a simple prompt and get something good.
But these AI models or large language models to really perform them at the high level, there is techniques, there is fundamental, you know, practices of prompt engineering.
And this is why the whole thing emerged. So, maybe this kind of just proves, you know, that maybe prompt engineering is not getting completely, completely simplified.
Yeah, I think another thing from them removing web browsing, code interpreter also added on top.
So, again, managing the resources. Code interpreters performing really good. Some people kind of speculate that code interpreter is 4.5, GPT-4.5.
What do you think about that?
I mean, this paper would potentially assert otherwise, granted that the executable that they're asking for, there is another part of this chart that talks about code generation accuracy.
So, they give it a question, be a prompt, be given an integer of n is greater than zero, find the sum of all integers in the range of 1 to n inclusive that are divisible by 3, 5 or 7.
Python should be able to knock this out of the park every single time. You have to write it obviously in Python code, but it's a, that should be a very simplistic question.
And in March, 52% of the time that GPT-4 is asked to write that in some various code language, they could take the directory, the code that's produced and execute it and run it and it would be correct.
And now it's down to 10% for that kind of question. So, maybe it is, we need to get better, we need to adjust some of our prompts, but I definitely don't appreciate the gaslighting that OpenAI has done as a result of some of this,
where they're like, oh no, you're just using it so much that you're just getting used to the quality of the output.
Yeah, so, you know, we have Gemini model on the horizon, but yesterday, two days ago, whenever we are recording Lama 2 came out.
And this is Open Source model released by MetaAI. And it's so funny how the whole dynamic and Meta coming into place.
And there's so many interesting aspects about this model. So, it's split into three parts. You have completely small ones, 7 billion parameters, 13 and 70.
And actually 70, like all these you can download on your computer and it actually doesn't require that powerful machine. You can run it on laptop.
But the 13 one, you can go online and test it. Lama2.ai and you can start playing with that.
And the interesting part, I found that you can change the overarching prompt.
Like, you know, when you say, oh, you are very helpful AI model, you can play with that.
And it's not bad. But besides it being good or bad, like clearly the size is smaller than GITPT or GPT-4.
But this is just a glimpse into Open Source. That's one aspect.
But another aspect that I think the real magic happens today, I mean, everybody can run it on their computers and then eventually on their phones.
And this is kind of just a glimpse to that. You have so many like insights.
So, just a comparison chat window is 4096, the same as chat GPT. So, you can actually provide a lot of context.
No, it's I think the future is definitely kind of somewhere on the upward slope of, you know, a logarithmic curve
because you have very, very diminishing returns with the current architecture of the transformer model, the current amount of compute.
You have to throw at it to get higher parameter models.
You know, you could probably in the next few months train your own 7 billion parameter model.
That's maybe just for a really specific use case. But then it can run on a phone or laptop however you want,
as opposed to something needing to leverage this huge trillion, multiple trillion parameter model that costs a ton of money somewhere
just to even use because of how much compute you need to throw at even answering simple questions.
There's going to be space for both of them. But I think it's pretty big to go to come from like, you know,
one of these global corporations like Meta to open source something like this.
I was trying to think about it and I think of some of the big tech players.
You know, I think Amazon has Bedrock, we have OpenAI, Microsoft with the Bing, Bard from Google.
Like no one's open sourced it yet, one of their models. But Meta seems to be, you know, really kind of that's their path forward.
Like here it is, make some cool stuff. We're all going to benefit from it.
And this was basically as a reaction to what OpenAI did with their, you know, release of GPT-4 and their research paper behind it
or marketing pitch deck, you can call it, telling nothing. Basically the research community was just in a way shocked, I would say.
But also naturally other companies started taking the lead into the direction to be more closed because of the competition,
because it's very competitive market. And another aspect to that is Meta's partnership with Microsoft.
Yeah.
That is something I did not thought I would see, Zach, with Sasha together in one picture, just, you know, holding and cuddling
and Microsoft positioning into AI. Again, as, you know, we covered last week when I got the chance to talk with a person just below Sasha.
And they are going full on AI. And I don't want to speculate or, you know, say something I would get in trouble.
But OpenAI is just one partnership, just one investment.
They made the splash. How many examples are there of this throughout history, especially in technological innovations?
I think one of my favorite ones is I think it was HP. They invented the computer mouse like back in the late 70s,
but it wasn't until Steve Jobs and took it and put it with, I think, the Apple II. Did that become just the standard?
You know, so while there is plenty of first mover advantage in the AI space and, you know, these are well established companies.
I don't think they need this. They need that. Kind of doing it right, I think, might be the enduring and lasting way to kind of succeed
if you're a bigger company like this.
Yeah. So on my side, I'm actually like, you know, I learned APIs. I'm learning to write code, playing with Python.
So the next thing what I want to do, especially with, you know, upgrade with Mac, with M2 chip,
I want to download one of the models and just get into that.
And that being said, I wanted to kind of bounce something with you because how you say your chief operating officer,
can we call Valhari Vats that we spent at the task manager?
Yes. What Gota is referring to is our one of my colleagues at Synth Vines who tries to keep the madness organized over there.
He's brilliant. Basically, he whips everyone's ass to get things in time, update project manager.
That's I work with so many project managers and project manage myself.
And what he's doing as like, you know, side hustle, side hobby is fascinating.
So that being said, I looked at myself and I was like, OK, I complained to every possible person I meet how much I work,
how exhausted I am, how many new projects there are, what is the YouTube game?
And you know, I like sat down and was like, either I burn out or I need to make changes because I'm talking about AI.
Like, it should make life easier. Of course, keeping up with it, just with news and things coming out is the whole other, you know, full time job.
But it should be how I was asking myself, how can I make my life way easier with the knowledge I have?
And I don't think that I'm fully, fully leveraging, you know, our network or the tools possible that I come across.
Like, it's good to talk about them. It's good to play. But how can I transform my business?
Because being, you know, content creator is new for me.
Like, I know how to be in different roles, but this is completely new business model.
So, yes, I basically went into that and, you know, plays kind of my marketing hat.
And you need to talk with audience. You need to hear feedback.
And as much as it's time consuming, it's really valuable. So, something what I did, I created a Google Doc form.
And I started sending out that to people who wrote to me on LinkedIn as a start.
And I hope we, let's include that in the show notes or newsletter.
Because also this could be something to do for podcasts.
And questions are basically, I'm asking, you know, what is your industry?
What problems do you face when it comes to AI? I ask some questions for my channel feedback.
But also people share their ideas. What is interesting to them?
And I can tell you so far, it's a lot of people already submitted. So, that's exciting.
And some very interesting ideas, but also the common theme.
And I can tell you a common theme of that is how do I practically implement that into my life?
Yeah.
So, that was like another one was keeping up with, you know, what's available, what's out there.
Which is I think is important part. But then you can keep up every day.
Like, you know, there's a bunch of amazing YouTube channels which keep you updated on a weekly basis, posting a bunch of videos.
But the other aspect is how do I understand it and how can I practically implement that?
So, my personal goal right now is completely to revamp the whole process how I do things.
From email management, content creation, to social media, which I'm not really active on social media in a way.
So, how can I leverage all these tools? And then once I achieve that, the share, you know, the whole process that anyone can replicate and go ahead with that.
I mean, it's undoubtedly, you know, a game changer. But one of the challenges with all these new tools is one, which one do you pick?
But then also like getting them to work together in a way that's logical is also difficult.
We're building something that hopefully does this called an orchestrator that, you know, we think is going to be able to tie a lot of these AI technologies together.
So, as new ones come out, we can just add it in and then it can be useful for building apps for, you know, interfacing with.
But there's a total need for it. We built some Discord bots just to try to keep us up with, you know, headlines that drop, YouTube videos that drop, new papers that come out.
You know, it's a full-time job.
It is next to all the other projects. I think what you just said, there's kind of three questions, you know, what is available out there?
What is good out of what is available? And the third one is like how things work together for my specific use case.
And my specific use case is, you know, specific to me, but there is probably another million people who have exactly the same problems as I do.
So, yeah, so I think also we could do something for this podcast.
I think you're totally right about a ton of that.
You know, one of the ways though that you can overcome it is, hey, find someone or, you know, invest in your own learning and knowledge in the space and benefit from people who are frankly just obsessed with this stuff.
It's been in our newsletter. I'll give it another plug here.
Our course with Co-Rise, AI and Chat GPT for Everyone, presale is live.
We're kicking off in August. This is to help people kind of come and use some practical applications with Chat GPT to do exactly what you're saying.
Make a business chat bot, a business analyst that can help you in your day to day, get information organized, validate ideas, write different plans, emails, letters.
But I think an important arrow in someone's AI quiver also then, especially in a lot of different fields, is some design tools with AI.
And I know you have a project to share that you've been working on hard with a course that you're going to be teaching as well.
Yeah, right. So I'm really excited to talk about this because of multiple aspects.
First of all, I come from design background, so everything is native to me and all this technology is just fascinating how it's going to completely transform design, how we know it.
So I was approached by Politecnico di Milano, the professor there.
And back like months back, the true story is that I almost end up studying there.
So Politecnico di Milano is one of the top, if not the top schools for postgraduate design in Europe and eighth in the world, I think.
So anybody from Asia, from African countries, and I think even the US, if they think to study design in Europe, they think Politecnico di Milano.
And we have this poly design segment. We basically put together a two-part course.
The whole theme of that is designing with emerging technologies.
But here's the twist. It's not just AI. Because in design practices, it's not just one spectrum.
And the real creativity shines when you combine technologies and you combine expertise.
We're covering AI, XR, which is virtual reality and augmented reality, and also physical things, and WebTree and DAO.
But more broadly, how WebTree, blockchain, technology, and also affect design practices.
So now what I want you to imagine is mixing pieces of these different domains together.
And this is our goal. The first part is level up. So this is why we are bringing experts and some crazy names, experts, startups, and tools.
So we combine everything and it's going to be 16 hours, so in October.
In English, fully online. So this is first time Politecnico di Milano does it on a global scale.
So everyone is invited. If you're a designer, if you run agency of any sort, anybody who really touches design aspect would be benefiting from this.
And the experts we are bringing are going to share their expertise and teach what's out there, what they've been using.
So the second part is game on. So this is where people who graduated level up.
They are going to move and this is very intimate. This is only 24 selected people who are going to be in this kind of study, academia group.
This is going to be 36 hours, twice a week in November.
Also both level up and game on, as I said, there is an aspect that seats are limited just because of faculty.
And we want to really dive into quality.
So another aspect is when you are in a classroom, you are learning from experts equally as much as the fellow next to you or someone in your group.
So there's going to be workshops, collaborative process, and game on is hands on that you will have a project, you will launch a product, you'll have full branding covered.
I'm very excited. This is huge projects and we are going to give on July 27.
We're going to have live stream on LinkedIn and also in Discord and on my own YouTube channel where we are bringing one of the OGs in the AI space, basically Binks.
He goes all the way back to 2018. I met him on Discord, Discord, Discord.
This was before stability, before mid-journey.
And they've been nerding out and playing with these things like years before everything came out.
That's like the messiolithic era in the AI.
Yeah, it's really crazy.
So if you would see the process what Binks does, this goes beyond anything you could imagine when it comes to image generation.
It's like every possible technology, different models, different ways, aspects, prompts.
It's just fascinating and it's really a huge privilege to gather all these people together.
I'm so proud of you.
Yeah. So I hope that this is going to be interesting for more people to join.
Definitely. This looks fantastic. Where can people go if they want to find out a little bit more or see the webinar?
So the easiest is polydesign.net and you can go to executive courses and under that you will see on a landing page level up and game on.
We will also include the landing page where both courses and you can learn more about that.
Another thing is that we are going to gradually introduce each of the experts coming in.
So this is going to be a campaign running throughout summer.
I'm very excited to share the names which are coming in for the live webinar.
So we will share in the newsletter my own YouTube channel and we are finalizing to host it on lernardo.ai.
And if you don't know what lernardo.ai is, you need to look it up.
It's basically one of the top AI companies in prompt text to 3D.
What they're doing is incredible. Their models are fantastic.
And they have just alone on Discord 1.6 million people as a community.
Yeah. I think we may have mentioned them on one of the first podcasts.
I know that they've pretty much blown up, but they have an entire marketplace, an app to basically generate actual 3D assets that can be used in games, videos, virtual environments just from prompting alone.
And an illustration of how important this can be and how useful it can be, we saw this week because I don't know if folks are aware, but right now there's a Hollywood writer strike.
That means they can't put out new shows, new content if your writers are in the Writers Guild Union.
So a company called The Simulation, which in and of themselves I think is an offshoot of the company Fable, they put out a completely tunable, a completely AI generated world essentially.
South Park!
Generate South Park episodes in.
My childhood. Oh my God.
So you pick the characters, pick the set, who's the hero character, and then prompt it and it simulates the story and generates the cartoon for you.
We're watching a little clip of it right now.
There is a basically kind of like a tree where I asked some questions, has pictures of the characters, and then you just give it a prompt.
What's the story? Who's the hero? And then it's generating.
Right now we're watching Cartman and Stan banter back and forth exactly like they would be doing on the show.
Like yours, no way you could tell.
So my brain, especially when we just talked about designing with emerging technologies.
Now, if you put a hat on next two years.
So prompt 3D, we already talked.
Leonardo is doing that. But now imagine you're prompting Game of Thrones series on VR interface where it's actually also interactive and you can, you know, touch, you can buy things, you follow this AI personas and every single person maybe has different experiences.
Or maybe we as people collectively watch this AI's just live their life and we tune in and maybe we are part of their journey or the whole gamification of everything.
So in all of this design is going to play a huge role and designers can take the lead.
So what we are watching is South Park, which is very simple animation.
This is in a way simple to do, but that's not going to stop here. This is just the starting point.
And it's already pretty damn good. Like you can tell it's a South Park.
You would never be able to tell. And mind you, it generates an 11 minute episode very quickly.
Yeah, that's, I mean, that's, that's where we're at now.
This is just exciting.
I do have one more announcement I'd like to finish on.
I'll be attending the AI for conference in Las Vegas.
It's one of the premier AI for business community conferences.
There's over 250 speakers. There'll be a ton of different events and I'll be going representing how to talk to AI.
So we're going to be interviewing some folks. I encourage people to go to AI for.
Let me know who of the speakers you want us to try to interview, who you want us to talk to.
We'll be doing some live streams, some content for the podcast and the video podcast from the floor.
That's, that's coming up. It's the seventh through ninth in Las Vegas in August.
Stand by for some, uh, hit of content from the floor of AI for.
I'm sorry that I'm not going. Like that would have been iconic, but I'm very happy that you're going to represent how to talk to AI.
Yes, indeed.
Yeah, because I also got invitation to be as got to go. So it's a little bit disappointing, but there's going to be more opportunities for us to do the live event.
But I'm excited that people you're going to meet and watch you.
Yeah, rock the scene.
Exactly. We're very sporting aloha shirts that are AI themed all through Vegas.
Probably pretty needed because it's going to be hot with that.
We hope to do that a bunch more in the coming weeks.
But with that, I think this is a good enough place as any to end.
So for go to go, I am West, the synth mind saying happy prompting everybody.
Happy prompting everybody.
Thanks for listening to how to talk to AI with your hosts go to go and West, the synth mind.
As always, you can check out the show notes and links at how to talk to dot AI.
That's all for this week's episode.
Happy prompting everyone.