OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti
The MAD Podcast with Matt Turck
14 HOURS AGO
OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti
OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti

The MAD Podcast with Matt Turck
14 HOURS AGO
This podcast features Sachin Katti, OpenAI's Head of Industrial Compute, discussing the immense physical infrastructure required to support the AI industry. He details the challenges of building and powering massive data centers, the shift to custom silicon, and the strategic thinking behind OpenAI's massive compute investments.
Sachin Katti describes the AI compute buildout as one of the largest infrastructure projects in history, with OpenAI planning to spend about $50 billion on compute this year. He explains that AI data centers are fundamentally different from traditional ones, acting as factories that turn electrons into tokens, requiring innovations like liquid cooling and behind-the-meter power. Katti discusses Project Jalapeno, OpenAI's custom chip designed to maximize tokens per watt, and notes that inference now dominates compute usage. He argues that the bigger risk is not building fast enough, as demand far outstrips supply. The conversation covers site selection, the Stargate strategy, and the use of offtake agreements for financing. Katti also addresses bottlenecks in the supply chain, including a shortage of skilled tradespeople, and the potential for AI to eventually design its own chips.
00:00
00:00
Demand for AI compute far outstrips supply
00:30
00:30
Massive infrastructure buildout for AI
01:44
01:44
One of the largest infrastructure projects in history
03:41
03:41
Scale now requires building own infrastructure.
04:54
04:54
AI data centers transform electrons into tokens
05:27
05:27
Giant factories turning electrons into tokens
06:35
06:35
Better cooling allows chips to run hotter, directly increasing compute power.
08:10
08:10
They invest in grid infrastructure to add new generation and transmission capacity
10:43
10:43
Gas turbines are the primary power source for AI data centers.
11:02
11:02
Nuclear power offers density and scalability for AI.
11:49
11:49
Co-design hardware to maximize tokens per watt.
13:25
13:25
Maximizing tokens per watt is critical.
13:38
13:38
Inference now dominates AI compute usage
14:58
14:58
Demand far outstrips supply
16:47
16:47
AI compute demand is insatiable and growing
17:55
17:55
Data centers are net positive for rural communities
21:18
21:18
Texas and Abilene fit these criteria
25:19
25:19
A portfolio approach using hyperscalers and neoclouds
25:59
25:59
OpenAI is a unique combination of a research lab, startup, and fast-growing company
28:05
28:05
Future plans involve designing and building their own data centers.
29:38
29:38
Stargate is OpenAI's overarching compute strategy
31:22
31:22
OpenAI's massive AI infrastructure
32:48
32:48
They act as tenants with partners, committing to consume compute through offtake agreements.
34:07
34:07
Designed in nine months with AI assistance
35:59
35:59
AI will design its own chips and systems.
36:20
36:20
Multi-path spraying masks frequent failures.
38:47
38:47
Multiple bottlenecks in AI infrastructure
40:29
40:29
AI compute is a critical supply resource in a world of shortage
42:08
42:08
It's a future component of compute, not a replacement