scripod.com

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

In this episode, the founders of Etched, Gavin Uberti and Rob Wachen, discuss their journey to build a specialized AI inference chip that challenges industry giants like NVIDIA. They detail the technical bets behind their architecture, their unconventional hiring strategy, and their vision for the future of AI infrastructure.
Etched's founders explain that by focusing solely on inference, they can optimize for specific constraints like operating temperature, achieving radical performance improvements over general-purpose GPUs. They identified unexplored design spaces in thermal and voltage management, and developed cluster-scale memory and custom interconnects to reduce latency. The company's vertical integration strategy, building chips, racks, and cooling systems in-house, maximizes performance and speed. They emphasize that the future of AI hardware hinges on concurrency, not just raw speed, as their chips offer an order of magnitude more users per watt. The founders predict that cheaper, faster inference will lead to more AI agents than human knowledge workers by 2027, with inference becoming a majority of global GDP. They also discuss the inevitability of trillion-dollar data centers driven by economies of scale in token production.
00:00
00:00
Welcome to Invest Like the Best.
03:54
03:54
First hardware company founded after ChatGPT to tape out a working chip.
03:58
03:58
General-purpose design creates inefficiencies.
10:22
10:22
They developed a low-voltage power delivery method to avoid thermal throttling
14:17
14:17
Improving throughput and concurrency is critical to scaling AI for billions of users.
20:29
20:29
A cancer diagnosis gave him a drive to do something impactful.
31:08
31:08
Contrarian candidates self-select for risky startups
36:31
36:31
Flood the binding constraint.
38:02
38:02
Speed is critical in AI inference.
44:00
44:00
Concurrency is the key metric for AI hardware
55:15
55:15
Winning is existential for us.
57:47
57:47
Many experiments fail, but only a few need to succeed.
1:05:41
1:05:41
We raised the largest AI chip Series A ever
1:08:43
1:08:43
Variant perception creates opportunity.
1:15:48
1:15:48
Adding our hardware alongside GPUs is positive-sum, not zero-sum.
1:16:42
1:16:42
Math is cheap, memory is expensive.
1:20:05
1:20:05
Inference will become a majority of global GDP
1:23:44
1:23:44
Trillion-dollar data centers are inevitable.
1:26:23
1:26:23
Risking paralysis for a higher chance of survival.