All right, so today we're taking a deep dive into
Tencent's latest creation in the AI world.
Yeah, their new model, Hanyuan Large.
Hanyuan Large.
That's right.
And what's really interesting about this,
and I know we've talked about some other
large language models on the show before,
but this one takes kind of a different approach.
It does, yeah.
And I think what's really interesting is the efficiency
and specialization that this model brings to the table.
It's built on this structure called a mixture
of experts or MOI.
And what that means is that the model itself
has different parts that are really good
at very specific things.
So instead of just one AI brain that tries to be good
at everything, it's more like a team.
Yeah, like a team of specialists, you got it.
Okay, I like that.
And so you have an expert for math,
you have an expert for coding,
you have an expert for understanding
long chunks of text.
So it knows when to call on the right expert for the job.
Exactly, yeah.
It's smart enough to know which expert to use
for any given task and that's what makes it so efficient.
So is that part of the reason why,
even though it's a massive model
with 389 billion parameters,
it only actually uses 52 billion of those parameters?
That's exactly it.
And that's I think one of the most interesting things
about this research is that it's showing
that you can get incredible performance
with a smaller number of activated parameters.
So Hanyu in large is going head to head
with models like Lama 3.1, which uses 405 billion parameters.
Wow.
And it's outperforming it on many tasks.
It's just kind of like a David and Goliath situation
in the AI world.
It really is, yeah, it's this smaller,
more specialized model that's outperforming a giant.
Yeah, and so how are they pulling this off?
What's the secret sauce here?
Well, there are three key things
that they highlighted in the paper.
The first is the data, they trained this model
on an absolutely massive amount of data,
seven trillion tokens, which is like feeding it a library
the size of a small country.
Wow, it's a lot of data.
Yeah, it's a ton of data,
but it's not just the size of the data that's important,
it's also the quality.
Oh, interesting, okay.
They actually generated 1.5 trillion tokens
of synthetic data.
Now, hold on, I'm not an AI expert,
so what is synthetic data?
So think about it this way.
Imagine you're teaching a kid about animals.
You could take them to a zoo
and show them all the different animals,
or you could show them carefully curated pictures
and videos of those animals.
So synthetic data is kind of like those pictures and videos.
It's data that's generated artificially
to mimic real world data.
So it's more controlled and efficient.
Exactly, it's like giving the model
a super concentrated learning smoothie.
I love that analogy.
Packed with all the essential information.
That's great, so massive amounts of data,
including this synthetic data.
What else is in the secret sauce?
So the second thing is a system
that they call recycle routing.
Recycle routing.
Now, remember how we talked about
the different experts in the model?
Yes.
Well, this routing system basically makes sure
that none of that information gets lost
as it's processed by those experts.
So it's like, if one expert's already overloaded,
it'll send it to another one that has the capacity.
Exactly, it's like having a super efficient
air traffic control system for the AI model.
I like that, okay, so that's two ingredients.
What's the third?
The third one is what they call
expert specific learning rates.
Okay, break that down for me.
Essentially, they realize the different parts
of the Moe structure need to learn at different paces
to be most effective.
Interesting.
So it's kind of like having a personalized learning plan
for each AI expert.
Fascinating.
So they've got this really efficient model.
They're training it on a ton of high quality data
and then they're even optimizing the learning process
for each expert.
What kind of results are they seeing with all of this?
Yeah, well, that's where things get really exciting.
Hanyu and Larch has really blown some benchmarks out of the water.
Oh, really?
Especially when it comes to things like language
understanding and even coding.
Okay, so give me some specifics.
Like what benchmarks are we talking about?
Well, one of the big ones they use is called MMLU.
Okay.
And that basically measures how well a model
understands language across a huge range of topics.
So it's like a test of like AI smarts.
Exactly, it's a pretty comprehensive test.
And Hanyu and Larch scored 88.4% on this benchmark,
which is significantly higher than models
like Lama 3.1, which only got like 85.2%.
Wow, and remind me how many parameters
each of those models is using.
Right, so Hanyu and Larch is only using 52 billion
activated parameters, and Lama 3.1 is using 405 billion.
So it's not just winning, it's winning by a landslide.
It really is, and it's not just the raw scores.
Hanyu and Larch is also showing that it can do things
like common sense reasoning, question answering,
and even generate code.
Oh, wow.
Okay, so it's not just talk,
this model can actually like walk the walk.
Exactly, it's the real deal.
And they really wanted to push it to the limits
and see how well it could handle long chunks of information.
So they actually created a special benchmark
just for this called Penguin Scrolls.
Penguin Scrolls, that sounds adorable.
It's a very catchy name.
But the benchmark itself is very rigorous.
They use things like financial reports, academic papers,
some of these documents were over 100,000 words long.
Wow, that's like feeding in an entire encyclopedia.
Yeah, it's a ton of information.
And they tested how well Hanyu and Larch
could extract key information,
answer really complex questions,
and even engage in multi-turned dialogues
based on these massive documents.
So it's not just about understanding language,
it's like understanding and reasoning
about complex information.
You got it.
And the results were outstanding.
It even outperformed models that are specifically designed
for these long context tasks.
That's seriously impressive.
Yeah.
So zooming out for a second,
what are the broader implications of this research?
What does it all mean?
I think it's really challenging our assumptions
about how we design AI in the future.
We've always thought that bigger is better,
but this research shows that it's not just about size,
it's about being smarter, more focused.
It's about being more efficient.
Exactly, and that has huge implications
for efficiency and accessibility.
Imagine AI models that are not only more powerful,
but also more energy efficient
and less computationally demanding.
Yeah, that would be a game changer,
especially with all the concerns that we have these days
about the environmental impact of AI.
Absolutely.
So what's next for Hanyu and Larch?
What are they planning to do with it?
Well, one of the coolest things about this
is that they're committed to open sourcing it.
Oh, so you mean making the model
and code available for anybody to use?
Exactly, they're setting a great example
by making this technology accessible
to the wider AI community.
And that fosters collaboration and innovation.
Absolutely, and it also helps address concerns
about transparency and ethical considerations.
Yeah, okay, so what does this mean for the average person?
How is this gonna impact our lives in the coming years?
That's the million dollar question.
It's hard to say for certain,
but this research is paving the way
for more powerful, more versatile AI systems.
So AI that can help us do more, understand more,
even create more?
Absolutely, and that has the potential
to really revolutionize everything from healthcare
and education to scientific research
and artistic expression.
Yeah, it really sounds like
we're entering a new era of AI.
I think so too.
And it's exciting to see where this journey takes us.
Is there anything else you wanna add
before we wrap up this deep dive into, honey, you enlarge?
Just one final thought.
It's not just about building bigger models,
it's about building smarter models.
And that, I think, is the big takeaway here.
So it's about being more strategic
with how we approach AI development.
Exactly, and that opens up a whole world of possibilities.
Imagine AI systems that are not only more capable,
but also more accessible to researchers
and developers all over the world.
So if I'm like a developer out there,
and I'm like, man, I wanna play around
with this, honey, you enlarge,
what can I actually do with this open source model?
Well, you can actually download the model weights
and the code, right from 10Cent's GitHub repository.
I can get my hands on the same tech
that powered all those benchmark results
we were talking about.
You got it.
Wow, okay, what could I do with that?
So the possibilities are endless.
You could fine tune the model for specific tasks,
you could experiment with different architectures.
You can even contribute to the development
of the core technology itself.
So it sounds like they're really creating
a sort of like a community around this.
Yeah, definitely a very collaborative
and open approach to AI development.
I like it.
So to wrap this all up,
what are your final thoughts on Honey When Large
and this whole idea of like the future of AI?
I think we're seeing a real paradigm shift here.
We're moving away from these giant monolithic AI systems
and towards these more specialized,
adaptable collaborative models.
And this is a prime example of that.
Honey When Large is a fantastic example
of this new wave of AI.
And I think it's just the tip of the iceberg, honestly.
I think we're entering a golden age of AI innovation
and it's gonna be really interesting to see where it goes.
I'm excited for it.
Well, I wanna thank you for joining us
on this deep dive into the world of Honey When Large.
It's been my pleasure.
It's been a really thought provoking conversation
and to our listeners out there,
if you wanna learn more about this research,
we'll be sure to include links to the paper
and 10 cents GitHub repository in the show notes.
And until next time, stay curious.