Source URL: https://www.tomtunguz.com/nvda-2025-05-29/
Source: Tomasz Tunguz
Title: 1000x Increase in AI Demand
Feedly Summary: NVIDIA announced earnings yesterday. In addition to continued exceptional growth, the most interesting observations revolve around a shift from simple one-shot AI to reasoning.
Reasoning improves accuracy for robots – like telling a person to stop and think about an answer before they reply. Here’s an example where I asked Gemini to create a financial projection for NVIDIA for the next five years.
Reasoning is compute-intensive, requires hundreds to thousands more – thousands of times more tokens per task than previous one-shot inference.
Software engineers also use reasoning extensively as AI coding agents examine code bases, plan modifications, and execute them. Each time I watch one of these reasoning traces I wonder how many GPUs are firing to produce the result.
OpenAI, Microsoft and Google are seeing a step-function leap in token generation. Microsoft processed over 100 trillion tokens in Q1, a fivefold increase on a year-over-year basis.
In addition to increased demand and greater usage, these reasoning models are driving significant volume increases in tokens as we saw in the Microsoft earnings announcement a few weeks ago.
On average, major hyperscalers are each deploying nearly 1,000 NVL72 racks or 72,000 Blackwell GPUs per week and are on track to further ramp output this quarter. Microsoft, for example, has already deployed tens of thousands of Blackwell GPUs and is expected to ramp to hundreds of thousands of GB200s with OpenAI as one of its key customers.
72,000 GPUs deployed per week is quite a statistic!
The pace and scale of AI factory deployments are accelerating with nearly 100 NVIDIA-powered AI factories in flight this quarter, a twofold increase year-over-year, with the average number of GPUs powering each factory also doubling in the same period.
To match the demand, hyperscalers are deploying more than $300b in capex this year to fund data centers, which interestingly, NVIDIA calls AI factories. What is the marketing rationale behind this framing? A new industrial revolution?
To date, the algorithmic improvements that reduce the overall model sizes are helping to staunch some of the geometric explosion in demand for AI, but it’s clear that both the demand for AI and more sophisticated reasoning are outpacing those advances.
AI Summary and Description: Yes
Summary: The text discusses NVIDIA’s recent earnings announcement, emphasizing a significant shift in AI capabilities from simple inference to more complex reasoning tasks. This shift is characterized by expanded computational requirements, increased token usage, and infrastructural developments among major tech companies like Microsoft, OpenAI, and Google.
Detailed Description:
The text presents critical observations regarding the advancements in AI technologies and the infrastructure supporting this evolution, particularly focusing on NVIDIA’s role in facilitating these changes. Key points include:
– **Shift to Reasoning in AI:**
– Transition from basic one-shot AI responses to complex reasoning capabilities.
– Reasoning enhances the accuracy of AI models, comparable to human cognitive reflection.
– **Computational Demands:**
– Reasoning operations are compute-intensive, requiring thousands of times more tokens than traditional AI inference tasks.
– This surge in computational needs indicates significant investments in hardware, especially GPUs.
– **Token Generation Advances:**
– Major cloud service providers like Microsoft, Google, and OpenAI are witnessing exponential growth in token generation.
– Microsoft’s processing of over 100 trillion tokens in Q1 2023 signifies a fivefold increase from the previous year.
– **Hyperscale Infrastructure Developments:**
– Hyperscalers are deploying large quantities of GPUs, with nearly 1,000 NVL72 racks deployed weekly, translating into tens of thousands of Blackwell GPUs.
– Approximately 100 NVIDIA-powered AI factories are operational, marking rapid growth in AI processing capacity.
– **Industry Investment Trends:**
– An estimated $300 billion in capital expenditure is allocated by hyperscalers to expand data centers in 2023.
– The term “AI factories” reflects a significant marketing strategy illustrating the relationship between infrastructure growth and AI advancements.
– **Future Implications:**
– While algorithmic improvements are helping to manage AI demand, the need for more sophisticated reasoning in applications continues to escalate.
– The discussion hints at a potential industrial revolution driven by AI capabilities and the infrastructure that supports them.
This analysis highlights essential trends in AI development, the increasing complexity of AI tasks, and the corresponding need for robust computational resources. For security and compliance professionals, understanding these dynamics is vital, particularly in areas like cloud security and infrastructure management, to anticipate and mitigate risks associated with scalable AI deployments.