Okay, so we're back and ready to go deep on this
one. It's NVIDIA's GTC March 2025 keynote. And,
well, Jensen Wong did not disappoint, to say
the least. Not even close. We've got the full
keynote transcript, the slides, the demos. I
mean, it was a firehose of information and some
really, really big announcements in there. Absolutely.
So the mission today is to cut through all that
and really distill, you know, the key takeaways.
What are the big moves NVIDIA is making? What
does it mean for the industry as a whole? And
most importantly, how does it connect back to
what our listeners need to know? I think the
biggest theme right from the start was just how
intertwined everything's becoming. You know,
AI obviously is at the center, but the way it's
impacting graphics, data centers, cars, robotics,
even basic scientific research. Yeah. It's all
connected now. Right, right. And the speed at
which it's all happening is, well, it's kind
of mind blowing. So let's jump in. One of the
first things that really caught my attention
was this concept of AI as a new kind of factory,
a token factory. Can you break that down a bit?
Yeah, it's a really powerful analogy. I mean,
think about it. Traditional factories, they take
raw materials and produce physical goods. But
these AI factories, they're producing something
different, right? They're churning out tokens
as their output. Tokens. So in simple terms,
what exactly are these tokens? Essentially, think
of them as the fundamental building blocks of
information that AI models use. Like a token
could represent a word. a piece of code, a pixel,
and an image. There would allow AI to process
and generate all these different kinds of data.
And what's really amazing is just how versatile
they are. Okay, that's starting to click. So
the more complex the AI, the more tokens it's
processing and generating. Exactly. And that
connects to this whole evolution of AI that Jensen
Huang talked about. We went from AI that was
primarily about perception, like recognizing
objects and images. Then we had generative AI,
creating new content. Now we're moving into what
he calls agentic AI. Agentic AI. What does that
mean? Like AI with a sense of agency? Yeah, essentially.
I mean, it goes beyond just generating outputs.
It's AI that can reason, plan, make decisions,
use tools that can actually take actions to achieve
a goal. Wow. That's a big leap. And the next
stage, he said, is physical AI. I mean, that
sounds like something out of science fiction.
Right. It's about bringing AI into the real world,
you know, enabling it to truly understand and
interact with the physical environment. Robotics
is a huge part of that. OK, so we have this clear
progression, but each of these steps comes with
its own challenges, right? Like accuracy for
perception, coherence for generative AI. What
are the big hurdles for agentic and physical
AI? For agentic AI, it's all about trust. I mean,
if these AI agents are going to be making decisions,
we need to be confident that they're making the
right ones, that they're aligned with our values
and that they won't go rogue. Right, right. That's
a bit unsettling to think about. And for physical
AI? Well, the real world is messy. I mean, it's
unpredictable. So the challenge there is building
AI that can handle all that complexity, that
can adapt to changing conditions and still operate
safely and effectively. And all of this, of course,
requires massive amounts of computational power.
I mean, Jensen Huang really hammered home the
point that the need for compute is just exploding.
Yeah, it's almost like a wake -up call. He said
the computational demands for these new AI models,
especially the agentic AI, it's something like
100 times greater than what we were anticipating
even a year ago. Wow. And what's driving that?
Is it just the sheer size of these models? Well,
that's part of it, but it's also the techniques
they're using. Like he talked about chain of
thought reasoning. Basically, it means the AI
breaks down a problem into smaller steps, generating
a ton more tokens in the process. So the smarter
and more complex the AI gets, the more compute
it needs. And that brings us to, well, NVIDIA's
bread and butter, right? Accelerated computing,
GPUs, the whole CUDA ecosystem. Absolutely. It's
all about harnessing the power of parallel processing
to tackle these massively complex computational
tasks. And CUDA, that's NVIDIA's platform for
making that happen. Okay. For those who might
not be familiar with C2, can you give us the
quick rundown? Sure. Think of CUDA as the language
that lets developers write software that runs
on NVIDIA GPUs. Unlike CPUs, which are good at
doing a few things really fast, GPUs have thousands
of smaller cores designed for massive parallel
processing. So for tasks like AI training and
inference, that's where GPUs really shine. And
then there's all these CUDAx libraries. It felt
like they were announcing a new one every few
minutes during the keynote. Can you give us a
sense of what they are and why they're so important?
Yeah. Think of them as specialized toolkits,
right? Like they're built on top of CUA and they're
specifically optimized for accelerating different
areas of science and industry. So instead of
starting from scratch, developers can just plug
in these libraries and boom. Instant performance
boost. And there's a library for almost everything,
right? I mean, we saw QNumeric for NumPy, Keyletho
for chip manufacturing, Arial for 5G. Right.
And each one is addressing a very specific need.
Like QNumeric, that's a huge deal for data scientists.
NumPy is like the foundation of scientific computing
in Python. And QNumeric makes it run seamlessly
on GPUs with massive speedups. In Kulitho, that's
the one for computational lithography, right?
The process of making chips. That seemed to tie
in really closely with this idea of AI factories
that Jensen Huang kept talking about. Totally.
His point was that every company that has a physical
factory, they're also going to need an AI factory
to design and optimize the processes in that
physical factory. And Caletho, it's a perfect
example. It's accelerating the creation of those
incredibly complex masks used to etch circuits
onto silicon wafers. So it's almost like a factory
within a factory, right? The AI factory powering
the physical factory. Exactly. And he said it's
not just chip manufacturing. It's going to be
true for every industry. Aerospace. automotive,
pharmaceuticals, AI is becoming fundamental to
the design and creation of physical products.
That's a big statement. And then there was Arial,
which is enabling GPUs to basically function
as 5G radios. How does that even work? It's pretty
mind -blowing. Arial is a library that allows
GPUs to handle all the complex signal processing
that's needed for 5G communication. And then
they layer AI on top of that with AI -RAN, which
stands for AI and Radio Access Networks. AI,
Ran. So AI is being used to optimize the network
in real time. Yeah. It can dynamically adjust
things like power allocation, beamforming all
the things that affect the performance of the
network. The goal is to make it more efficient,
more responsive, and ultimately deliver a better
user experience. That's fascinating. And it's
another example of how AI is blurring the lines
between hardware and software. Absolutely. And
that trend continues with Qopt, the library for
numerical and mathematical optimization. Right.
Optimization problems are everywhere, right?
From logistics and supply chains to financial
modeling. Exactly. It's all about finding the
best solution out of a huge number of possibilities.
And traditionally, those calculations can be
extremely computationally intensive. Qopt brings
the power of GPUs to bear on those problems,
significantly speeding up the process. And then
there was the whole rapid fire section on all
the other CDX libraries, Parabricks for gene
sequencing, MONAI for medical imaging, Earth2
for weather prediction. It was almost overwhelming,
but in a good way. Yeah, it really showcased
just how broad the impact of accelerated computing
is. I mean, these libraries were enabling breakthroughs
in critical areas like health care, climate science,
drug discovery. It's incredible to see the potential.
And they're even dipping their toes into quantum
computing with KuQuantum and CuDay Quantum. They
even had their first quantum day at GTC, which
is. Pretty forward -looking. Definitely. Quantum
computing is still early days, but NVIDIA is
clearly taking it seriously. These libraries
are providing tools for researchers to explore
and develop quantum algorithms and to start integrating
quantum processors with classical GPUs and hybrid
systems. So NVIDIA is laying the groundwork for
a future where quantum and classical computing
work together. Exactly. And then there was CUT
ESS, the big announcement for sparse solvers
in CAE, which is computer -aided engineering.
Right. That one seemed to generate a lot of buzz.
What's so significant about it? Well, SCAR solvers,
they're used in all sorts of engineering simulations,
like how a bridge will withstand stress or how
a new airplane design will behave in flight.
And until recently, NVIDIA themselves were using
general purpose computers to design their own
accelerated computing hardware. Wow. So they
were essentially using slower tools to design
faster tools. Exactly. But with Cut ESS, they've
optimized those solvers for CD. which means they
can run on GPUs. So now the entire process of
engineering design, it's getting supercharged.
Faster simulations, more complex designs. It's
a big leap forward. And they also touched on
CutEF for data frames and Warp for physics simulations
and Python. I mean, the list just goes on and
on. And that's the key takeaway, right? They're
building this incredibly rich ecosystem of software
tools, all powered by CBA and GPUs. And because
CDA is so widely adopted, these tools are immediately
valuable to millions of developers across all
these different fields. It's why NVIDIA believes
we've reached this tipping point of accelerated
computing. Before we move on from CDA, I wanted
to touch on Halos, which is their framework for
automotive safety. They didn't spend a ton of
time on it, but given the growing importance
of self -driving cars, it seems pretty crucial.
Oh, absolutely. Halos, it's their way of making
sure that self -driving cars are, well... safe.
They're doing rigorous testing on every line
of code, which millions of lines, by the way.
And they're building in principles of diversity,
transparency and explainability. They even dedicated
a whole workshop at GTC to it. So they're not
just focused on the performance of self -driving
systems, but also on their safety and trustworthiness.
Exactly. It's not enough for these systems to
just work. They have to work safely and predictably.
OK, so we've covered the foundational layer,
the software tools. Now let's get into the hardware.
NVIDIA announced their new Blackwell architecture,
and the big news was that they're essentially
putting two GPUs in a single package. What's
the thinking behind that? Well, it's all about
increasing compute density, right? Putting more
processing power into a smaller space. But the
really interesting part is the disaggregated
NVLink switch, which they're calling MVLink Now.
MVLink. So what does that mean and why is it
important? So NVLink, that's NVIDIA's high -speed
interconnect, that allows GPUs to communicate
with each other really fast before it was integrated
onto the motherboard. But now with NVLink, they've
made it a separate switch. And that gives them
a lot more flexibility. Flexibility in what way?
Well, they can connect more GPUs together with
higher bandwidth and with more simultaneous communication
pathways. And that's crucial for scaling up these
massive AI models. So MVLink is all about allowing
the GPUs to talk to each other more efficiently.
Exactly. And that, combined with the move towards
liquid -cooled data centers, means they can pack
a lot more compute into a single rack. I mean,
they talked about achieving one exaflops in a
single rack. One exaflops? That's mind -boggling.
I mean, what does that even translate to in practical
terms? It's an astronomical amount of computing
power. And it's all made possible by liquid cooling.
which is much more efficient at removing heat
than traditional air cooling. So they're pushing
the limits of both chip design and thermal management
to achieve these massive scales. Absolutely.
And it's not just about training these AI models.
They also talked a lot about the importance of
inference, which is actually using the trained
models to generate outputs. Right. That's where
the rubber meets the road, right? It's what actually
delivers value to users. Exactly. And inference,
especially for these new reasoning -based AI
models. it can be incredibly computationally
demanding. I mean, they gave this example of
a wedding seating problem where a reasoning model
used over 8 ,000 tokens compared to just under
500 for a basic language model. Wow. So reasoning
requires a lot more token generation, which translates
to more compute. Exactly. And users expect those
outputs quickly, right? They don't want to be
waiting around. So you need a really powerful
infrastructure to handle all that. And of course,
these models are so big that they have to be
split across multiple GPUs. They mention techniques
like tensor parallelism, pipeline parallelism,
expert parallelism. Can you give us a quick overview
of what those are? Yeah, they're basically different
ways of breaking down the model and its computations
so that it can be distributed across multiple
GPUs working together. Each technique has its
own advantages and challenges, and the best approach
often depends on the specific model and the hardware
you're using. So it's a complex orchestration
problem trying to manage all these different
pieces. Totally. And that's where NVIDIA Dynavo
comes in. Yeah. They're describing it as the
operating system for these AI factories. An operating
system for AI. What does that mean? Basically,
it handles all the low level stuff like scheduling
workloads, routing data between GPUs, optimizing
the distribution of compute resources. It's like
the conductor of the orchestra, making sure everything
runs smoothly. And they're making it open source,
right? So other companies can build on top of
it. Exactly. And they're collaborating with companies
like Perplexity to really build out this ecosystem
around Dynamo. OK, so we've got the hardware,
we've got the software infrastructure. Now, what
about the actual performance games? I mean, they
were throwing around some impressive numbers.
Yeah, they showed Blackwell with MVLink 72 and
Dynamo delivering up to 40 times faster token
generation compared to Hopper, especially for
those reasoning models. And the ISO power efficiency
gains were around 25x. ISO power. Can you explain
what that means? Sure. Basically, it means they're
getting 25 times the performance for the same
amount of power consumption. So it's not just
about raw speed. It's about efficiency, too.
Right. Which is really important for data centers
where power is a major cost factor. Absolutely.
And then there was this whole discussion about
digital twins and the role of NVIDIA Omniverse
in designing and managing these AI factories.
Can you unpack that a bit? What are digital twins
and why are they important in this context? A
digital twin, it's basically a virtual replica
of a physical system. So in this case, it would
be a virtual representation of the entire AI
infrastructure, the servers, the GPUs, the networking,
the cooling systems. And the idea is that you
can use this digital twin to simulate different
scenarios, optimize performance, and even identify
potential problems before they happen in the
real world. So it's like having a virtual test
bed for your entire AI factory. Exactly. And
NVIDIA Omniverse, that's their platform for creating
and running these digital twins. They're partnering
with companies like Vertiv and Schneider Electric,
who are experts in data center infrastructure,
to make these digital twins as realistic and
useful as possible. OK, so they're really thinking
holistically about the entire AI infrastructure
from the chips to the software to the physical
data center itself. Totally. And then there's
the networking piece, which is critical for scaling
these systems. They talked about their Spectrum
X Ethernet solution and how Cisco is adopting
it for enterprise AI. Cisco is a big name in
networking. What's significant about this partnership?
Well, it means that high performance networking
for AI. It's becoming more mainstream. Cisco's
putting Spectrum X into their enterprise products,
making it more accessible for companies that
want to deploy AI at scale. And they also talked
about the limitations of traditional copper interconnects
at these massive scales. What's the alternative?
They're betting on silicon photonics, which is
essentially using light to transmit data instead
of electrical signals. It offers much higher
bandwidth, lower latency and lower power consumption.
So it's like fiber optic cables on a chip. Yeah,
sort of. They're co -packaging the optical transceivers
directly with the processing chips, which is
a pretty big engineering feat. And they highlighted
this new type of modulator called a micro ring
resonator or MRM. What's the advantage of that?
MRMs are much smaller. and more power efficient
than traditional modulators so they can pack
more optical connections into the same space.
It's a big step towards making silicon photonics
practical for large scale data centers. OK, so
they're laying the groundwork for the next generation
of high performance networking. Now let's get
to the roadmap. They talked about Blackwell Ultra
coming in the second half of 2025, followed by
Vera Rubin in the second half of 2026. What's
the big deal about Rubin? Rubin is a complete
architectural overhaul. They're introducing a
new CPU, a new GPU, a new networking fabric called
MVLink 144, and the latest HBM4 memory. It's
like a whole new generation of their high -performance
computing platform. And then there's RubinUltra
in 2027, which promises even more extreme scale
-up. They were talking about 15 exaflops of compute
and 4 ,600 terabytes per second of bandwidth.
Those are just insane number. Yeah, it's clear
they're not slowing down. And they showed how
the total cost of ownership is actually going
down with each generation, even as the performance
is skyrocketing. So these incredibly powerful
capabilities are becoming more and more accessible.
That's an important point. It's not just about
pushing the limits of what's possible. It's about
making those capabilities affordable and practical
for a wider range of users. Exactly. And then
they shifted gears to talk about AI for enterprise
computing. They announced these new DGX systems,
DGX Spark, which is like a personal AI development
platform, and DGX Station, which is a super -powered
workstation. What's the target market for those?
I mean, who needs that much power at their desk?
Well, DGX Stark. That's aimed at individual researchers
and developers who want a dedicated AI development
environment. It's powerful enough for serious
work, but still relatively compact and affordable.
That's for the real power users, people doing
cutting -edge AI research or developing really
demanding applications. It's like having a mini
supercomputer on your desk. And they talked about
reinventing storage for AI. What does that mean?
It's all about moving away from traditional file
-based storage to something that's more intelligent
and aware of the meaning of the data. They're
calling it semantics -based retrieval, and they're
partnering with all the major storage vendors
to make it happen. So it's about making the storage
itself more AI -aware. Exactly. And then there's
the open sourcing of their reasoning model as
part of their NIMS initiative. What's the significance
of that? It's about democratizing access. to
these advanced AI capabilities. By making the
model open source, they're allowing anyone to
use it and build on top of it. So they're not
just building the tools, they're also making
the core technologies more accessible. Exactly.
And finally, there was the big push in for robotics.
Jensen Wong declared that the time has come for
robots, and they laid out their vision for how
AI is going to revolutionize this field. They
talked about this continuous loop of simulation,
training, testing and real world experience.
Can you walk us through that? It's about using
simulation to generate massive amounts of data
for. training robot AI, then testing those AI
policies in virtual environments before deploying
them to real world robots. Right. And then the
data from the real world feeds back into the
simulation, creating this continuous cycle of
improvement. So it's like a virtual proving ground
for robot AI. Exactly. And they announced these
new tools to make that happen. Cosmos for generating
synthetic data. Newton for physics simulation.
And then there was a big announcement of Isaac
Groot N1, their generalist foundation model for
humanoid robots. And they're open sourcing that
as well, right? Yes, which is huge. It's like
giving the entire robotics community access to
this incredibly powerful starting point for building
their own robots. So it's clear that NVIDIA is...
taking a very comprehensive and forward -looking
approach to robotics. Totally. And that's really
the takeaway from the entire keynote, right?
They're not just focused on one piece of the
puzzle. They're thinking about the whole stack,
from the hardware to the software to the applications,
and they're pushing the boundaries in every direction.
It's an incredibly exciting time to be following
this field. I mean, the pace of innovation is
just relentless. It is. And it's not just about
the technology itself. It's about the impact
it's going to have on our lives. I mean, AI is
going to transform everything from the way we
work to the way we interact with the world around
us. So for our listeners, the key takeaway is
that this stuff matters. Even if you're not a
developer or a data scientist, AI and accelerated
computing are going to shape the future. And
understanding the trends and the key players.
is going to be increasingly important. Absolutely.
This keynote, it was a glimpse into that future,
and it's clear that NVIDIA is playing a central
role in shaping it. Well, on that note, we'll
wrap things up here. As always, thanks for joining
us for another deep dive. We'll be back soon
to dissect another fascinating topic. Until then,
keep exploring, keep learning, and stay curious.
See you next time.