scripod.com
OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti

Overview

Shownote

Highlights

Transcript

Chapters

Pins

OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti

The MAD Podcast with Matt Turck

13 HOURS AGO
The MAD Podcast with Matt Turck

The MAD Podcast with Matt Turck

13 HOURS AGO
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Is the AI industry actually overbuilding, or is the physical world moving too slowly to keep up? In this episode of the MAD Podcast, OpenAI's Head of Industrial Compute, Sachin Katti, takes us inside the "belly of the beast" of what may be the largest infr...

Highlights

This podcast features Sachin Katti, OpenAI's Head of Industrial Compute, discussing the immense physical infrastructure required to support the AI industry. He details the challenges of building and powering massive data centers, the shift to custom silicon, and the strategic thinking behind OpenAI's massive compute investments.
00:00
Demand for AI compute far outstrips supply
00:30
Massive infrastructure buildout for AI
01:44
One of the largest infrastructure projects in history
03:41
Scale now requires building own infrastructure.
04:54
AI data centers transform electrons into tokens
05:27
Giant factories turning electrons into tokens
06:35
Better cooling allows chips to run hotter, directly increasing compute power.
08:10
They invest in grid infrastructure to add new generation and transmission capacity
10:43
Gas turbines are the primary power source for AI data centers.
11:02
Nuclear power offers density and scalability for AI.
11:49
Co-design hardware to maximize tokens per watt.
13:25
Maximizing tokens per watt is critical.
13:38
Inference now dominates AI compute usage
14:58
Demand far outstrips supply
16:47
AI compute demand is insatiable and growing
17:55
Data centers are net positive for rural communities
21:18
Texas and Abilene fit these criteria
25:19
A portfolio approach using hyperscalers and neoclouds
25:59
OpenAI is a unique combination of a research lab, startup, and fast-growing company
28:05
Future plans involve designing and building their own data centers.
29:38
Stargate is OpenAI's overarching compute strategy
31:22
OpenAI's massive AI infrastructure
32:48
They act as tenants with partners, committing to consume compute through offtake agreements.
34:07
Designed in nine months with AI assistance
35:59
AI will design its own chips and systems.
36:20
Multi-path spraying masks frequent failures.
38:47
Multiple bottlenecks in AI infrastructure
40:29
AI compute is a critical supply resource in a world of shortage
42:08
It's a future component of compute, not a replacement

Chapters

Cold open: “One of the largest things humanity has ever built”
00:00
Welcome: Sachin Katti, Head of Industrial Compute at OpenAI
00:30
Is this the biggest infrastructure buildout in history?
01:44
Why OpenAI is building a new industrial muscle
03:41
What an AI data center actually is
04:54
“Factories turning electrons into tokens”
05:27
Why AI data centers need liquid cooling everywhere
06:35
The power problem: grids, generation, transmission, substations
08:10
Behind-the-meter power and gas turbines
10:43
Why nuclear “can’t come soon enough”
11:02
Jalapeño: why OpenAI is designing its own AI chips
11:49
Tokens per watt: the new metric that matters
13:19
Why inference may now dominate AI compute
13:38
Is OpenAI overbuilding compute?
14:58
Why OpenAI thinks the bigger risk is not building fast enough
16:47
Communities, jobs, water, and the local data-center debate
17:55
How OpenAI chooses data-center sites
21:16
What “industrial compute” means inside OpenAI
22:25
Sachin’s path: Stanford, startups, Intel, OpenAI
25:59
OpenAI’s compute portfolio: Microsoft, hyperscalers, neoclouds
28:05
Stargate explained
29:37
Abilene, Oracle, and the next wave of AI data centers
31:21
How massive AI compute gets financed
32:48
How OpenAI designed Jalapeño so quickly
34:05
AI is starting to help design AI chips
35:59
MRC: the networking problem behind 100,000 GPUs
36:20
Bottlenecks: transformers, turbines, electricians, supply chains
38:47
Guaranteed capacity: intelligence as a supply unit
40:29
Will AI data centers move to space?
42:08

Transcript

Speaker 3: Anytime we have thought we have enough compute, we can slow down, always negatively surprise us, like, oh, we should not have slowed down. Demand far outstrips compute supply today. So anything we can bring online, we consume immediately. Our bi...