Jensen Huang – Will Nvidia’s moat persist?

Then it sends it to an ODM in Taiwan where they assemble the racks.

And so NVIDIA is fundamentally making software that other people are manufacturing.

And if software gets commoditized, Does Nvidia get commoditized?

Well, in the end, something has to transform electrons to tokens.

That transformation, um, there's no— the transformation of electrons to tokens, uh, and making those tokens more valuable over time, uh, I, I don't— I think that, that, that's hard

to hard to completely commoditize.

The transformation from electrons to tokens is such an incredible journey.

And making that token, you know, it's like making one molecule more valuable than another molecule, making one token more valuable than another.

The amount of artistry, engineering, science, invention that goes into making that token valuable,

Obviously we're watching it happening in real time.

And so the transformation, the manufacturing, all of the science that goes in there is far from deeply understood and it's far from, the journey is far from over.

And so I doubt that it will happen.

We're gonna make it more efficient, of course.

I mean, the whole thing about NVIDIA, in fact, the way that you framed the question is my mental model of our company.

The input is electron, the output is tokens.

That is in the middle NVIDIA.

And our job is to, to do as much as necessary, as little as possible to enable that transformation to be done at incredible capabilities.

And, and what I mean by as low as possible, whatever I don't need to do,

I partner with somebody and I make it part of my ecosystem to do.

And if you look at NVIDIA today, we probably have the largest ecosystem of partners, both in supply chain upstream, supply chain downstream, all of the computers, the computer companies,

and all the application developers and all the model makers and all the, you know, AI is a 5-layer cake, if you will.

And we have ecosystems across the entire 5 layers.

And so we try to do as little as possible, but the part that we have to do, as it turns out, is insanely hard.

And I don't think that that gets commoditized.

In fact,

I also don't think that the enterprise software companies, the tools makers, you know, most of the software companies today are tools makers.

Some of them are not, but some of them are workflow codification, you know, systems.

But for a lot of companies, they're tool makers.

For example, Excel's a tool, PowerPoint's a tool, Cadence makes tools, Synopsys makes tools.

I actually see the opposite of what people see.

I think the number of agents are going to grow exponentially.

The number of tool users are gonna grow exponentially.

And it's very likely that the number of instances of

all these tools are gonna skyrocket.

It is very likely the number of instances of Synopsys Design Compiler is gonna skyrocket.

And the number of agents that are gonna be using the floor planners and all of our layout tools and our design rule checkers, the number of agents that are— today we're limited by the

number of engineers.

Tomorrow, those engineers are gonna be supported by a bunch of agents and we're gonna be exploring out the design space like you've never seen explored before.

And we're gonna use the tools that we use today.

And so, so I think, I think tool use is gonna cause, cause these software companies to skyrocket.

The reason why it hasn't happened yet is because the agents aren't good enough at using their tools yet.

And so either these companies are gonna build the agents themselves or agents are gonna get good enough to be able to use those tools.

And I think it's gonna be a combination of both.

I think in your latest filings it was, you had almost $100 billion in purchase commitments with foundries, memory, packaging.

And then Semi Analysis has reported that you will have $250 billion of these kinds of purchase commitments.

And so one interpretation is Nvidia's moat is really that you've locked up many years of these scarce components that are, you know, somebody else might have an accelerator, but can

they actually get the memory to build it?

Can they actually get the logic to build it?

And this is really Nvidia's big moat for the next few years.

Well, it's one of the things that we can do that is hard for someone else to do.

The reason why we could— we've made enormous commitments upstream.

Some of it is explicit, these commitments that you mentioned.

Some of it is implicit.

For example, a lot of the investments that are upstream are made by our supply chain because I said to the CEOs, let me tell you how big this industry is going to be, and let me explain

to you why.

And let me reason through it with you and let me show you what I see.

And so as a result of that, that process of informing, inspiring, aligning with CEOs of all different industries upstream, they're willing to make the investments.

Now, why are they willing to make the investments for me and not someone else?

And the reason for that is because they know that I have the capacity to buy it, buy their supply and sell it through my downstream.

The fact that NVIDIA's downstream supply chain and our downstream demand is so large, they're willing to make the investment upstream.

And so if you look at GTC

and, you know, people are marveled by the scale of GTC and the people that go, it's a 360 degrees, the entire universe.

Of AI all in one place.

And they're all in one place because they need to see each other.

I bring them together so that the downstream could see the upstream, the upstream could see the downstream, and all of them could see all the advances in AI.

And very importantly, they can all meet the AI natives and all the AI startups that are all, you know, being built and all the amazing things that are happening so that they could see

firsthand all the things that I tell them.

And so I spent a lot of my time informing directly or indirectly our supply chain and our partners and our ecosystem about the opportunity that's in front of us.

You know, most of my keynotes, you know, some people always say, you know, Jensen,

in most keynotes it's like one announcement after another announcement after another announcement after another announcement.

Our keynotes are,

there's always a part of it that's a little torturous in the sense that it almost comes across like an edu— like education.

And in fact, that's exactly on my mind.

I need to make sure that the entire supply chain upstream and downstream, the ecosystem understands what is coming at us, why it's coming, when it's coming, how big is it gonna be,

and be able to reason about it systematically, just like I reason about it.

And so I think the

mode as you describe it, we're able to, of course,

build for a future.

If our next several years is $1 trillion in scale, we have the supply chain to do it.

Without our reach, the velocity of our business, You know, just as there's cash flow, there's supply chain flow, there are turns.

Nobody's going to build a supply chain for an architecture if the architecture, the business turns is low.

And so our ability to sustain the scale is only because our downstream demand is so great and they see it and they all hear about it.

They, they see it all coming.

And so that's— it allows us to do the things that we're able to do at the scale we're able to do.

I do want to understand more concretely whether the upstream can keep up.

For many years now, you guys have been 2x-ing revenue year over year.

You guys have been more than tripling the amount of flops you're providing to the world year over year.

And 2x-ing at the scale now, it's really incredible.

Exactly.

Yeah.

So then you look at Logixai, you're the biggest customer on TSMC's N3 node and you're one of the biggest on N2.

AI as a whole this year is going to be 60% of N3.

It's going to be 86% next year according to Semi Analysis.

How do you 2x if you're the majority?

And how do you do that year over year?

So are we, are we in a regime now where the growth rate in AI compute has to slow because of upstream?

Do you see a way to get around these?

You know, how do we build 2x more fabs year over year ultimately?

Yeah, at some, at some level,

the, the instantaneous demand

is greater than the supply

upstream and downstream, uh, in the world.

And, and it could be at any instant, any instance, we could be limited by the number of plumbers, which, which actually happens.

The plumbers are invited to next year's GTC.

Yeah.

You know, by the way, great idea.

But that's a good condition.

You, you want you want a market, you want an industry where the instantaneous demand is greater than the total supply of the industry.

The opposite is obviously less good.

If we're too far apart, if one particular item, one particular component is too far away,

obviously the industry swarms it.

So for example,

notice people aren't talking very much about co-ops anymore.

Yeah.

And the reason for that is because for 2 years we swarmed the living daylights out of it and we double, double, double on several doubles.

And now I think we're in a fairly good shape.

And TSMC now knows that co-was supply has to keep up with the rest of the logic demand and the memory demand.

And so they're scaling co-was and they're scaling, you know, future packaging technologies at the same level as they scale logic, which is terrific.

Because for a long time, co-was rather specialty and HBM memory was rather specialty, but they're not specialties anymore.

People now realize they're mainstream computing technology.

And then, and of course, we're now much more able to influence a larger scope of our supply chain.

In the past, in the past,

you know, in the beginning of the AI revolution, all the things that I say now, I was saying 5 years ago.

And some people believed in it and invested in it.

For example, Sanjay and the Micron team, I still remember the meeting really well where I was clear about exactly what's gonna happen and why it's gonna happen and the predictions,

the predictions that of today.

And they really doubled down on it and we partnered with them and across LPDDR, across, you know, HBM memories, they really invested in it and it obviously has been tremendous for the

company.

Some people came a little bit later, but now they're all here.

And so I think each

one of these bottleneck gets a great deal of attention.

And now we're prefetching the bottlenecks years in advance.

So for example, the investments that we've done with Lumentum and Coherent and all of the silicon photonics, ecosystem.

The last several years, we really reshaped the ecosystem and the supply chain of silicon photonics.

We built up an entire supply chain around TSMC.

We partnered with them on COUPP, invented a whole bunch of technology.

We licensed those patents to the supply chain, keep it nice and open.

And so we're preparing the supply chain through invention of new technologies, new workflows,

new testing equipment, double-sided probing, investing in companies, helping them scale up their capacity.

And so, so you could see that we're trying to shape the ecosystem so that it's ready, the supply chain, so that it's ready to support the scale.

It seems like some bottlenecks are easier than others.

And so scaling up co-ops versus scaling up— I went to the hardest one, by the way, which is plumbers.

Yeah, that's true.

Yeah, yeah, I actually went to the hardest one.

Yeah, yeah, plumbers and electricians.

And the reason for that is because Because, and this is one of the concerns that I have about all the doomers

describing the end of work and killing of jobs.

And one of the things that if we discourage people from being software engineers,

we're gonna run out of software engineers.

And the same prediction 10 years ago, some of the doomers were saying that we're telling people, whatever you do, don't be a radiologist.

And you might hear some of those, some of those videos are still on the web.

You know, radiology is going to be the first career to go.

Nobody's— the world's not going to need any more radiologists.

Guess what we're short of?

Radiologists.

Oh, but okay.

So going back to this point about, well, some things you scale, other things like how do you actually get— how do you actually manufacture 2x the amount of logic a year?

Ultimately, that's bottlenecked by memory and logic are bottlenecked by EUV.

How do you get to 2x as many EUV machines a year?

Yeah.

Year over year.

None of that's impossible to scale quickly.

You just need to, you could do, all of that is easy to do within 2 or 3 years.

You just need a demand signal.

It's not, once you can build 1, you could build 10.

And once you can build 10, you can build a million.

And so these things are not hard to replicate.

How far down the supply chain do you go?

Where do you go to?

ASML and say, hey, if I look out 3 years from now, for NVIDIA to be generating $2 trillion a year in revenue, we need way more AUV machines and— Some of them I have to directly, some

of them indirectly, and some of them, if I can convince TSMC, ASML will be convinced.

And so that's, you know, we have to think about the critical pinch points.

But if TSMC is convinced,

you'll have plenty of, EV machines in a few years.

And so none of that, my point is that none of the bottlenecks last longer than a couple, 2, 3 years, none of them.

And meanwhile, meanwhile we're improving computing efficiency by 10x, 20x, in the case of Hopper to Blackwell, some 30, 50x.

We're coming up with new algorithms because CUDA is so flexible.

We're developing all kinds of new techniques.

So that we drive efficiency in addition to increasing capacity.

And so those are things that none of that worry me.

It's the stuff that's downstream from us,

energy policies that prevent energy from, you know, you can't grow, you can't create an industry without energy.

You can't create a whole new manufacturing industry without energy.

We want to reindustrialize the United States.

We want to bring back chip manufacturing and computer manufacturing and packaging.

And we want to build new things like EVs and robots and we want to build AI factories.

And you can't build any of these things without energy.

And those things take a long time.

But more chip capacity, that's a 2-3 year problem.

More co-was capacity, 2-3 year problem.

Interesting.

I feel like I have guests tell me the exact opposite thing sometimes.

And I don't— in this case, I just don't have the technical knowledge to adjudicate.

But well, the beautiful thing is you're talking to the expert.

Yeah, true, true.

Okay, I want to ask about your competitors.

Yeah.

So if you look at TPU, arguably 2 out of the top 3 models in the world, Claude and Gemini, were trained on TPU.

What does that mean for NVIDIA going forward?

Well, we have a very different, we build a very different thing.

You know what NVIDIA built is accelerated computing, not a tensor processing unit.

And accelerated computing is used for all kinds of things, you know, molecular dynamics and quantum chromodynamics.

And it's used for data processing,

data frames, structured data, unstructured data.

It's used for

fluid dynamics, particle physics, you know, and In addition, we use it for AI.

And so accelerated computing is much more diverse.

And although AI is the conversation today, is obviously very important and impactful,

computing is much broader than that.

And what NVIDIA has done is reinvented the way computing is done from general-purpose computing to accelerated computing.

Our market reach is

far greater than any, any TPU, any ASIC can possibly have.

And so if you look at our position,

we're the only company that, that accelerates applications of all kinds.

We have a gigantic ecosystem.

And so all kinds of frameworks and algorithms all run on NVIDIA.

And because our computers are designed to be operated by other people.

Anyone who's an operator could buy our systems.

Most of these home-built systems, you have to be your own operator because it was never designed to be flexible enough for other people to operate.

And so as a result of the fact that anybody can operate our systems, we're in every cloud, including Google and Amazon and Azure and OCI and right?

And so whether you want to operate it to rent or operate it— if you want to operate it to rent, you better have large ecosystem of customers in many industries that be the off-takers.

If you're operating it,

if you want to operate it for yourself, we

obviously have the ability to help you operate yourself.

Like for example, for Elon with xAI.

And because we could enable operators in any company, in any industry.

You could use it to build a supercomputer for scientific research and drug discovery at Lilly.

And so we can help them operate their own supercomputer and use it for the entire diversity of drug discovery and biological sciences that we accelerate.

And so there are just, you know, a whole bunch of applications that we can address that you can't do so with TPUs.

Because NVIDIA's built CUDA as a fantastic tensor processing unit as well.

But it does, you know, it does every lifecycle of data processing and computing and AI and so on and so forth.

And so I, our market opportunity is just a lot larger.

Our reach is a lot greater.

And because we have such a large,

we basically support every application in the world now.

You could build NVIDIA systems anywhere and know that there will be customers for it.

And so it's a very different thing.

This is going to be sort of a long question, but, you know, you have spectacular revenue, um, and this revenue is mostly— you're not making $60 billion a quarter from, uh, pharma and,

um, quantum.

You're making it because AI is, uh, unprecedented technology that is growing unprecedentedly fast.

And so then the question is, what is best for AI specifically?

And I'm not in the details, but I talked to my AI researcher friends and they say, look, when I use a TPU, it's this big systolic array that's perfect for doing matrix multiplies, whereas

a GPU is very flexible.

It's great when you have lots of branching, when you have

irregular memory access.

But with these, you know, what is AI?

Just like these very predictable matrix multiplies again and again and again.

And you don't have to give up any die area for warp schedulers, for, you know, switches between threads and memory banks.

And so the TPU is really optimized for the majority, the bulk of this growth in revenue and use case for compute that is coming online right now.

Yeah, I wonder how you react to that.

Matrix multiplies is an important part of AI, but it's not the only part of AI.

And if you want to come up with a new attention mechanism, or if you want to disaggregate in a different way, if you want to come up with a whole new type of architecture altogether,

for example, you know, a hybrid SSM.

If you want to use a— you want to create a model that, that, um, that fuses diffusion and autoregressive somehow, uh, you want an architecture that's just generally programmable.

And, and we run everything you can imagine.

And so that's the advantage.

It allows for invention of new algorithms a lot more, a lot, a lot more easily.

And so, because it's a programmable system and, and the ability to invent new algorithms is really what makes AI advance so quickly.

You know,

TPUs, like anything else, is impacted by Moore's Law.

And we know that Moore's Law is increasing about 25% per year.

And so The only way to really get 10x leaps, 100x leaps

is to fundamentally change the algorithm and how it's computed every single year.

And that's NVIDIA's fundamental advantage.

The only reason why we were able to make Blackwell the hopper 50 times, you know, I said it was 35 times and, and, and when I first announced it was gonna, Blackwell was gonna be 35

times more energy efficient than Hopper.

Nobody believed it.

And then Dylan wrote an article, he said, in fact, I sandbagged, it's actually 50 times.

And you can't reasonably do that with just Moore's Law.

And so the way that we solve that problem is new models, MOEs,

parallelized and disaggregated and distributed.

Across a computing system and without the ability to really get down and come up with new kernels with CUDA, it's really hard to do.

And, and so the combination of the programmability of our, of our architecture,

the, the fact that NVIDIA is an extreme co-design company where we could even offload some of the computation into the fabric itself, NVLink, for example, into the network spectrum

X,

and that we could affect change across the processors, the system, the fabric, the libraries, the algorithm.

All of that was done simultaneously.

Without CUDA to do that, I wouldn't even know where to start.

My sponsor, Crusoe, was among the first clouds to offer NVIDIA's Blackwell and Blackwell Ultra platforms.

And they just announced their NVIDIA VERA Rubin deployment scheduled for later this year.

But access to state-of-the-art hardware is only part of the story.

For example, most inference engines already do KV caching for a single user's forward passes, but Crusoe does it across users and GPUs.

So if 1,000 agents are running on the same system prompt, Crusoe only has to compute the KV cache once for it to become available to every single GPU in the cluster.

This is especially important as systems get more agentic and require much longer prefixes in order to use tools, and access files.

In a recent benchmark, Crusoe was able to deliver up to 10 times faster time to first token and up to 5 times better throughput than VLLM.

This is just one among many reasons that you should run your inference workload with Crusoe.

And if you need GPUs for training, you don't need to switch clouds.

Crusoe's got you covered there too.

Go to crusoe.ai/thorcache to learn more.

So this gets at an interesting question about, um, Nvidia's clientele, where if 60% of your revenue is coming from these big 5 hyperscalers, you know, in a different era with different

customers, let's say it's professors who are running experiments and they are helped a bunch by— they need CUDA, they can't use another accelerator, they need to just run PyTorch with

CUDA and have everything optimized.

But if you've got these hyperscalers, they have the resources to write their own kernels.

In fact, they have to, to get that extra last 5% that they need for their specific architecture.

Anthropic, Google are mostly running their own accelerators or running TPUs

and Trainium.

But even OpenAI using GPUs has Triton, which they're like, we need our own kernels.

So they've down to CUDA C++, they've instead of using CuBLAS and NCCL and everything, they've got their own stack which compiles to other accelerators as well.

And so if most of your customers can, can and do make replacements for CUDA, to what extent is CUDA really the thing that is going to make frontier AI happen on NVIDIA?

CUDA, CUDA is,

is a rich ecosystem.

And so if you want to build on any computer first, building on CUDA first is incredibly smart.

And because the ecosystem is so rich.

We support every framework.

If you want to create custom kernels, if you need, for example, we contribute enormously to Triton.

And so the backend of Triton, huge amounts of NVIDIA technology.

We're delighted to help every framework become as great as it can be.

And there's lots and lots of frameworks.

There's Triton, there's VLM, there's SG Lang, and then there's more.

right?

And now there's a whole bunch of new reinforcement learning frameworks coming out.

You know, you got Vero, you got Nemo RL, you got a whole bunch of new.

And then the, now with post-training and reinforcement learning, that entire area is just exploding, right?

And so if you want to build on an architecture, building on CUDA makes the most sense because you know that the ecosystem is great.

You know that if something happens, it's more likely in your code and not in the mountain of code underneath.

You know, don't forget the amount of code that you're dealing with when you're building these systems.

When something doesn't work, was it you or was it the computer?

You would like it always to be you and to be able to trust the computer.

And obviously we still have lots and lots of bugs ourselves, but

our system is so well rung out that you could at least build on top of the foundation.

So that's number one.

Is that the richness of the ecosystem, the programmability of it, the capability of it.

The second thing is, is if you were a developer and you were building anything at all, the single most important thing you want more than anything is install base.

You want the software that you run to run on a whole bunch of other computers.

You don't want to build software.

You're not building software just for yourself.

You're building software for your fleet or for everybody else's fleet because you're a framework builder.

And NVIDIA's CUDA Ecosystem is ultimately its great treasure.

We are now, I don't know how many, several hundred million GPUs.

Every cloud has it.

Goes back to A10, A100, H100, H200, you know,

the L series, the P series.

I mean, there's a whole bunch of them and they're in all kinds of sizes and shapes.

And if you're a robotics company, you want that CUDA stack to actually run in the robot itself.

We're literally everywhere.

And so the install base says that once you develop the software, once you develop the model, it's going to be useful everywhere.

And so the install base is just too incredibly valuable.

And then lastly, the fact that we're in every single cloud makes us genuinely unique because, you know, you're an AI company and you're an AI developer.

You're not exactly sure which CSP you're going to partner with and where you would like to run it.

And we run it everywhere.

Including on-prem for you if you like.

And so, so I think that, that the, the richness of the ecosystem, the expansiveness of the, of the, of the install base and the versatility of where, where, where we are, that combination

is, is, uh, makes CUDA invaluable.

That makes a lot of sense.

I guess the thing I'm curious about is, um,

whether those advantages matter a lot to your main customers.

There's many people who they might matter for, for the kind of person who can actually build their own software stack, who will make up most of your revenue.

Especially if you go to a world where AI is getting especially good at the things which have tight verification loops where you can RL on them.

And then this question of how do you write a kernel that does attention or MLP the most efficiently across a scale-up, it's a very verifiable sort of feedback loop.

And so, oh, can everybody, can all the hyperscalers write these custom kernels for themselves?

And they might still— NVIDIA has, still has great price performance, so they might still prefer to use NVIDIA.

But then the question is, does it just become a question of who is offering the best specs, the best flops and memory and memory bandwidth for a given dollar, where historically NVIDIA

has just had and still has the best margins in all of AI across hardware and software, 70% plus, because of this CUDA mode.

And the question is, oh, can you sustain those margins if for most of your customers they can actually afford to build, build instead of the CUDA mode?

The number of engineers we have assigned to these AI labs is insane, working with them, optimizing their stack.

And the reason for that is because because nobody knows our architecture better than we do.

And these architectures are not as general purpose as a CPU.

The reason why a CPU is so, you know, CPU is kind of like a Cadillac, you know, it's just always, you know, it's a nice cruiser.

It never goes too fast.

Everybody drives it pretty well.

You know, it's got cruise control, you know, and everything's easy.

But in a lot of ways, NVIDIA's GPUs are accelerators, are kind of like F1 racers.

And yeah, I could imagine everybody's able to drive it at 100 miles an hour, but it takes quite a bit of expertise to be able to push it to the limit.

And we use a ton of AI to create the kernels that we have.

And I'm pretty sure we're gonna still be needed for quite some time.

And so our expertise, Helps our AI Labs partners get another 2x out of their stack easily, oftentimes.

It's not unusual that we, you know, by the time that we're done optimizing their stack or optimizing a particular kernel, their model sped up by 3x, 2x, 50%.

That's a huge number, especially when you're talking about the install base of the fleet that they have, of all the hoppers and blackwalls that they have.

When you increase it by a factor of 2, that

doubles the revenues.

That directly translates to revenues.

NVIDIA's computing stack is the best performance per TCO in the world, bar none.

Nobody can demonstrate to me that any single platform in the world today has better performance TCO ratio.

Not one company.

And in fact, In fact, the benchmarks are out there.

Dylan's, right, Inference Max is sitting out there for everybody to use.

And not one, TPU won't come, Trainium won't come.

I encourage 'em to

use Inference Max and demonstrate their incredible inference cost.

It's really, really hard.

Nobody wants to show up.

MLPerf?

I would welcome Trainium to demonstrate their 40% that they claim all the time.

I would love to hear them demonstrate the cost advantage of TPUs.

It makes no sense in my mind.

It makes absolutely zero sense.

On first principles, it makes no sense.

And so I think

the reason why we're so successful is simply because our TCO is so great.

There's a second, you say 60% of our customers are the top 5, but most of that business is external.

For example, most of AWS's, most of NVIDIA in AWS is for external customers, not internal use.

Most of our customers at Azure, obviously all of our customers are external.

All of our customers at OCI are external, not internal use.

The reason why they favor us is because our reach is so great, we can bring them all of the great customers in the world.

They're all built on NVIDIA.

And the reason why all these companies are built on NVIDIA is because our reach and our versatility is so great.

And so I think the flywheel is really install base, the programmability of our architecture, the richness of our ecosystem, and the fact that there's so many AI companies in the world.

There's tens of thousands of them now.

And if you were one of those AI startups, what architecture would you choose?

You would choose an architecture that's most abundant.

We're the most abundant in the world.

The one has the largest installed base.

We're the most largest installed base and one that has a rich ecosystem.

And so that's the flywheel.

That's the reason why between the combination of one, our perf per dollar is so great.

That, that they have the lowest cost tokens.

Second, our perf per watt is the highest in the world.

And so if, if one of these companies, if our partners built a 1 gigawatt data center, that 1 gigawatt data center better deliver the maximum amount of revenues that and number of tokens,

which directly translates to revenues.

You want it to generate as many tokens as possible, maximize the revenues for that data center.

We are the highest tokens per watt architecture in the world.

And then lastly, if your goal is to rent the infrastructure, we have the most customers in the world.

And so that's the reason why the flywheel works.

Interesting.

I guess the question comes down to what is the actual market structure here?

Because even if there's other companies, there could have been a world where there's tens of thousands of AI companies that have roughly equal share of compute.

But if even through these 5 hyperscalers, really the people on Amazon using the computer Anthropic, OpenAI, and these big foundation labs who can themselves afford and have the ability

to make different accelerators work.

No, I think your assumption is, premise is wrong.

Maybe.

Yeah, let me ask you a slightly different question.

Come back and make me correct your premise.

Okay, let me just ask you a different question, which is, okay, if everything you're saying— But still make sure to make me come back and fix, because it's just too important.

To AI.

It's too important to the future of science.

It's too important to the future of the industry.

That premise, the premise, look, let me just finish the question and then we can address it together.

So what do you think if

all these things are true about price performance and performance per watt, et cetera, are true, why do you think it is the case that say Anthropic, for example, just announced a couple

days ago they have a multi-gigawatt deal with Broadcom and Google for TPUs and majority of their compute.

Obviously for Google, it's TPUs are majority of compute.

So if I look at these big AI companies, it seems like a lot of their compute— there was some point where it was all NVIDIA and now it's not.

And so I'm curious how to square—

if these things are true on paper, why are they going with other accelerators?

Yeah, Anthropic is a unique instance and not a trend.

Without Anthropic, why would there be any TPU growth at all?

It's 100% Anthropic.

Without Anthropic, why would there be any Trainium growth at all?

It's 100% Anthropic.

I think that's fairly well known and well understood.

It's not that there's an abundance of ASIC opportunities.

There's only one Anthropic.

But OpenAI deals with AMD.

They're building their own Titan accelerator.

Yeah, but they're mostly, I think we could all acknowledge they're vastly NVIDIA and we're gonna still do a lot of work together.

Yeah.

And we're not, I'm not offended by other people using something else and trying things.

If they don't try these other things, how would they know how good ours is?

You know, and sometimes you gotta be reminded of it

and we gotta, and we have to continuously earn earn the position that we're in.

You know, there are always big claims and look at the number of ASICs that have been canceled.

Just because you're gonna build an ASIC, you still have to build something better than NVIDIA.

And it's not that easy building something better than NVIDIA.

It's not sensible actually.

You know, it's, we, NVIDIA's gotta be missing something seriously, you know, and because our scale, our velocity, we're the only company in the world that's cranking it out.

Every single year, big leaps every single year.

I guess their logic is that, hey, it doesn't need to be better.

It just needs to be not more than 70% worse because they're paying you 70% margins.

No, no, no.

Don't forget, even in ASICs, margins are really quite high.

NVIDIA's margin is 70%, let's say, but an ASIC margin is 65%.

What are you really saving?

Oh, you mean from Broadcom or something?

Yeah, sure.

You got to pay somebody.

Yeah.

And so I think the ASIC margins are incredibly good from what I can tell.

And they believe it so too.

And so they're quite proud of their incredible ASIC margins.

And so you ask the question why.

A long time ago, we just didn't have the ability to do it.

And this is, and at

the time, I didn't deeply internalize how difficult it would be to build a foundation AI lab like OpenAI and Anthropic.

And the fact that they needed huge investments from the supplier themselves.

We just weren't in a position to make the multi-billion dollar investment into Anthropic so that they could use our compute.

But Google and AWS were, and they put in huge investments in the beginning so that Anthropic in return used their compute.

We just weren't in a position to do so at the time, nor did I, I would say my mistake is I didn't deeply internalize that they really had no other options,

that a VC would never put in $5, $10 billion of investment into an AI lab with the hopes of it turning out to be Anthropic.

And so that was my miss.

But even if I understood it, I don't think we would've been in a position to do that at the time.

But I'm not gonna make that same mistake again.

And I'm delighted to invest in OpenAI

and I'm delighted to help them scale.

and I believe it's essential to do so.

And then, and then when, um, uh, when I was able to, uh, Anthropic, when Anthropic came to us, uh, I'm delighted to be an investor, delighted to help them scale.

And, um, uh, but we just weren't at, at the time able to do so.

If I, if I could, uh, rewind everything, uh, NVIDIA could have been as big back then as we are now.

I would have been more than happy to do it.

This is actually quite interesting, which is for many years Nvidia has been this,

the company in AI making money, making lots of money.

And now you're investing it.

It's been reported that you've done up to $30 billion in OpenAI and $10 billion in Anthropic.

But now their valuations have increased and I'm sure they'll continue to increase.

And so if Over these many years, you were giving them the compute, you saw where AI was headed, and then they were worth like 1/10 what they are now a couple of years ago, or even a

year ago in some cases.

And you had all this cash.

There's a world where either NVIDIA themselves becomes a foundation lab, does a huge investment to make that possible, or has made the deals you've made now at current valuations much

earlier on.

And you had the cash to do it.

So I am curious actually, why not have done it earlier?

We did it as soon as we could have.

We did it as soon as we could have.

And

if I could have, I would have done it even earlier.

At the time that Anthropic needed us to do it, we just weren't in a position to do it.

It wasn't, it wasn't, you know, it wasn't in our sensibility to do so.

How so?

Like a cash thing or just?

Yeah, the level of investment, you know, we had never invested outside the company at the time and not that much.

And we didn't realize we needed to,

you know, I always thought that they could just go raise VCs for God's sakes, like all companies do.

But what they were trying to

do Couldn't have been done through VCs.

What OpenAI wanted to do couldn't have been done through VCs.

And I recognize that now.

I didn't know it then.

You know, but that's their genius.

That's why they're smart.

You know, and so they realized that then that they had to do something like that.

And I'm delighted that they did, you know.

And even though

we caused Anthropic to have to go to somebody else, I'm still happy that it happened.

Anthropic's existence is great for the world.

I'm delighted for it.

Uh, I guess you still are making a ton of money and you're making way more money, um, quarter after quarter.

It's still okay to have regrets.

Um, so the question still arises, okay, well, now that we're here and you have all this money that you keep making, um, what should NVIDIA be doing with it?

And there's one answer which says, look, there's this whole middleman ecosystem that has popped up for converting

CapEx into OpEx for these labs so that they can rent compute.

Because the chips are really expensive, they make a lot of money over their lifetime because AI models are getting better.

The value that they generate through tokens is increasing, but they're expensive to set up.

Nvidia has the money to do the CapEx.

And in fact,

it's been reported you're backstopping CoreWeave up to $6.3 billion and have invested $2B.

but yeah, why doesn't NVIDIA become

a cloud themselves?

Why doesn't it become a hyperscaler themselves and rent this compute out?

You have all this cash to do it.

This is a philosophy of the company and I think is wise.

We should do as much as needed, as little as possible.

And what that means is the work that we do with building our computing platform, if we don't do it, I genuinely believe it doesn't get done.

If we didn't take the risk that we take, if we didn't build NVLink the way we built, if we didn't build the whole stack, if we didn't create the ecosystem the way we did it, if we didn't

dedicate ourselves to 20 years of CUDA while losing money most of that time, if we didn't do it, nobody else would've done it.

If we didn't create all of the CUDA-X libraries so that they're all domain-specific, you know, this is several, a decade and a half ago, we pushed into domain-specific libraries because

we realized that if we didn't create these domain-specific libraries, whether it's for ray tracing or image generation or even the early works of AI, these models, if we didn't create

'em for data processing, structured data processing or vector data processing, if we didn't create them, nobody would.

And I am completely certain of that.

We created a library for computational lithography called CuLitho.

If we didn't create it, nobody would have.

And so accelerated computing wouldn't advance the way it has if we didn't do what we did.

And so we should do that.

We should dedicate our company, all of our might, wholeheartedly go do that.

However, the world has lots of clouds.

If I didn't do it, somebody show up.

And so following the recipe, the philosophy of doing as much as needed, but as little as possible.

As low as possible.

That philosophy exists in our company today, and everything I do, I do it with that lens.

In the case of clouds, if we didn't support CoreWeave to exist, these neo clouds, these AI clouds, wouldn't exist.

If we didn't help CoreWeave exist, they would not exist.

If we didn't support Nscale, they wouldn't be where they are today.

If we didn't support Nibius, they wouldn't be what they are today.

Now they are, they're doing fantastically.

Is that a business model?

No, we should do as much as needed, as little as possible.

And so we're trying, we invest in our ecosystem because I want our ecosystem to thrive and I want our, our, I want, I want the architecture and I want AI to be able to connect with

as many industries as possible, as many countries as possible, and make it possible for, you know, the planet to be built on AI and to be built on the American tech stack.

And so, so that vision, I think, is exactly what we're pursuing.

Now, one of the things that, that you mentioned, um, there are so many great, amazing foundation model companies, and we try to invest in all of them.

And this is, this is another thing that we do.

We don't pick winners.

And we, we like, we, we, we need to support everyone and it's part of our, part of our, our, our joy of doing so.

It's, it's an imperative to our business, but we also go out of our way not to pick winners.

And so when I, when I invest in one of them, I invest in all of them.

Why do you go out of your way to not pick winners?

Because it's not our job to, number one.

Number two, when NVIDIA first started, there were 60 graphics companies.

60 3D graphics companies.

We are the only one that survived.

If you would have taken those 60 companies, 60 graphics companies, and asked yourself which one was going to make it, Nvidia would be the top of that list not to make it.

You know, this is long before you, but Nvidia's graphics architecture was precisely wrong.

It's not a little bit wrong.

We created an architecture that was precisely wrong.

And it was an impossible thing for developers to support.

It was never gonna make it.

We reasoned about it from good first principles, but we ended up in the wrong solution.

And everybody would've counted us out.

And here we are.

And so

I have enough humility to recognize that, you know, don't pick winners.

Either let them all take care of themselves or take care of all of them.

One thing I didn't understand is you said, look, we're not prioritizing these new clouds just because they're new clouds and we want to prop them up.

But you also said you listed a bunch of new clouds and you said they wouldn't exist if it wasn't for NVIDIA.

Yeah.

And so how are those two things compatible?

First of all, they, they need to want to exist and they come to ask us for help.

And when they want to exist and have, they have a business plan and they, you know, they have expertise and you know, they have the passion for it.

They obviously have to have some capabilities themselves.

But if at the end of the day they need some investment in order to get it off the ground, we would be there for them.

But the sooner they get their flywheel going, you know, your question was, do we want to be in the financing business?

The answer's no.

Yeah, we don't want to be, we want to, we, because there are people in the financing business and we'd rather work with all of the people who are in the financing business than to be

a financier ourselves.

And so, so I think the, the, our goal is to focus on what we do, keep our business model as simple as possible, support our ecosystem.

When someone like, like OpenAI needs an investment of $30 billion scale because it's still before their IPO,

And, and we deeply believe in them.

We deeply believe that, I deeply believe that, that they're gonna be, they're gonna be an ex— well, they're an extraordinary company already today.

They're gonna be incredible company.

The world needs 'em to exist.

The world wants 'em to exist.

I want them to exist.

And, and they have everything on, they have the wind at their back.

Let's, let's support them and let them scale.

And so, so to those, those investments will do.

Because we're— they need us to do it.

And, um, but we're not trying to do as much as possible, we're trying to do as little as possible.

I spend way too much time copy-pasting text back and forth from Google Docs to chatbots, and so I built what's basically a cursor for writing, which operates the way I think an AI co-researcher

should operate.

I can tag it and it can talk with me through inline comment threads and help me dig deeper and brainstorm.

I built this entire thing over the weekend with Cursor and their Composer 2 model.

With a lot of agentic coding tools, I feel like I have no idea what's going on under the surface.

I just have to relinquish control and hope for the best.

But Cursor let me try a bunch of different ideas while staying on top of the implementation.

I did most of my brainstorming in the agents window, and after I got some basic files in place, I used the diff window to track changes.

The few times that I needed to make a quick tweak by hand, I just used the editor.

If you want to try my AI code researcher yourself, I've linked the GitHub repo in the description.

And if you have a tool that you've been wanting to build, you should make it happen.

Go to cursor.com/thorcash to get started.

This may be sort of an obvious question, but we've lived many years in this situation where there's a shortage of GPUs, and it's grown now because models are getting better.

We have a shortage of GPUs.

Yes.

Yeah.

And

NVIDIA is known for divvying up the scarce allocation not just based on highest bidder, but rather on, hey, we want to make sure that these Neo Clouds exist.

Let's give some to CoreWeave.

Let's give some to Crusoe.

Let's give some to Lambda.

Why is it good for NVIDIA?

First of all, would you agree with this characterization of fracturing the market?

No, no, no.

Yeah.

Your premise is just wrong.

Yeah.

Yeah.

We're sufficiently

mindful about these things.

We're very mindful about these things.

First of all, If you don't place an order, if you don't place a PO,

all the talking in the world won't make a difference.

And so until we get a PO, what are we going to do?

And so the first thing is, is we work, we work really hard with everybody to get a forecast done

because these things take a long time to build and the data centers take a long time to build.

And so we align ourselves with demand and supply and things like that through forecasting.

Okay, that's job number one.

Number two,

everybody who, you know, we've tried to forecast with as many people as possible, but in the final analysis, you still had to place an order.

And maybe

for whatever reason you didn't place your order, what can I do?

And so at some point, first in, first out.

But beyond that, if you're not ready because your data center's not ready, or certain components aren't ready to enable you to stand up a data center,

we might decide to serve another customer first.

That's just maximizing the throughput of our own factory.

And so we might do some adjustments there.

Aside from that,

the prioritization is first in, first out.

Yeah, you got to place a PO.

If you don't place a PO.

Now, of course, there are stories about that.

You know, like for example, all of this kind of started from, from, uh, it was an article about Larry and Elon having dinner with me where they begged for GPUs.

That never happened.

We absolutely had dinner.

We absolutely had dinner.

Um, and it was a wonderful dinner.

In no time did they beg for GPUs.

And so it, they just had to place an order and once they place an order, we do our best to get the capacity to them.

Yeah.

We're not complicated.

Okay.

So it sounds like there's a queue and then, um, uh, based on whether your data center is ready and when you place a purchase order, you get them a certain time, but it still doesn't

sound like highest bidder just gets it.

Is there a reason to do it?

We never do that.

Okay.

We never do that.

Why not just do highest bidder?

Because it's a bad business practice.

You set your price, you set your price, and then people decide to buy it or not.

And I understand that others in the chip industry

change their prices when demand is higher, but we just don't.

We just don't.

That's just never been a practice of ours.

You can count on us.

You know, I prefer to be, to be

dependable,

to be the foundation of the industry.

And I— you don't need to— you don't need to second guess.

You know, if you— if I quoted you a price,

we quoted you a price.

That's it.

And if demand goes through the roof, so be it.

And on the other end, that's why you have a productive relationship with TSMC, right?

Yeah.

Yeah, yeah.

NVIDIA has been in business— we've been doing business with them for I guess coming up on 30 years.

And NVIDIA and TSMC don't have a legal contract.

There's always some rough justice.

And sometimes I'm right, sometimes I'm wrong.

Sometimes I got a better deal, sometimes I got a worse deal.

But overall in the whole, the relationship is incredible.

And I can completely trust them.

I can completely depend on them.

And our, one of the things that we, you can count on with NVIDIA is that next year— this year, Vera Rubin's going to be incredible.

Next year, Vera Rubin Ultra will come.

The year after that, Feynman will come.

And the year after that, I haven't introduced the name yet.

And so, so every single year you can count on us.

And this is an—

you're going to have to go find another ASIC team in the world.

Pick your ASIC team where you can say, I can bet the farm of— I can bet my entire business that you will be here for me every single year.

Your cost, your token cost will decrease by an order of magnitude every single year.

I can count on it like I can count on the clock.

Well, I just said something about TSMC.

No other foundry in history can you possibly say that.

You can say that about NVIDIA today.

You can count on us every single year.

If you would like to buy $1 billion worth of AI Factory compute, no problem.

If you'd like to buy $100 million, no problem.

If you'd like to buy $10 million or just one rack, not a problem.

Or just one graphics card.

Okay, no problem.

If you would like to place an order for a $100 billion AI Factory, no problem.

We're the only company in the world where you can say that today.

I can say that about TSMC as well.

I want to buy one, buy $1 billion, no problem.

We just got to go through the process of planning for it and, you know, all this, all the things that, that mature people do, you know?

And so, so I, I think the, the, uh, this ability for NVIDIA to be the foundation of the world's AI industry,

this is a, this is a position that has taken us decade, several, couple of decades to arrive at.

Enormous commitment, enormous dedication.

And, um,

the stability of our company, the consistency of our company is really, really important.

Okay.

I want to ask about China.

Yep.

And I always like to take, uh, I don't actually don't know what I think about whether it's good to sell chips to China or not, but I've like played devil's advocate against my guests.

So when Dario was on who supports export controls, I asked him, well, why can't America and China both have country of geniuses in a data center.

But since you're on the opposite side, I'll ask you in the opposite way.

And look, one way to think about it is Anthropic actually announced a couple days ago Mythos preview— this model Mythos, they're not even releasing publicly because they say it has

such cyber offensive capabilities that we don't think the world is ready until we get— we make sure these zero days are patched up.

But they say it found thousands of high severity vulnerabilities across every major operating system.

Every browser.

It found one in OpenBSD, which is this operating system that's been specifically designed to not have zero days, and it found one.

For 27 years it's existed.

And so if Chinese companies and Chinese labs and the Chinese government had access to the AI chips to train a model like Claude Mythos with these cyberoffensive capabilities and run

millions of instances of it with more compute, the question is, oh, is that a threat to

American companies to American national security?

First of all, um, Mythos was, was, uh, trained on fairly mundane capacity

and a fairly mundane amount of it,

um, by an extraordinary company.

Uh, and so the amount of capacity and the type of compute that it was trained on is abundantly available in China.

And so

you just have to first realize that chips exist in China.

They manufacture 60% of the world's mainstream chips, maybe more.

It's a very large industry for them.

They have some of the world's greatest computer scientists.

As you know, most of the AI researchers in all of these AI labs, most of them are Chinese.

They have 50% of the world's AI researchers.

And so the question is, if you're concerned about them,

what is the— considering all the assets they already have, they have an abundance of energy, they have plenty of chips, they got most of the AI researchers.

If you're worried about them, what is the best way

to create a safe world?

Well,

victimizing them,

turning them into an enemy likely isn't the best answer.

They are an adversary.

We want United States to win.

But I think having a dialogue and having research dialogue is probably the safest thing to do.

This is an area that is glaringly missing.

Because of our current attitude about

China as an adversary.

It is essential that our AI researchers and their AI researchers are actually talking.

It is essential that we try to both agree on how to, what not to use the AI for.

With respect to finding bugs in software, of course, that's what AI is supposed to do.

Is it gonna find bugs in a lot of software?

Of course.