The Shape of Compute (Chris Lattner of Modular)
Latent Space: The AI Engineer Podcast
2025/06/13
The Shape of Compute (Chris Lattner of Modular)
The Shape of Compute (Chris Lattner of Modular)
Latent Space: The AI Engineer Podcast
2025/06/13
In this episode, Chris Lattner of Modular discusses the company's efforts to revolutionize GPU programming and democratize AI. The conversation explores how Modular is breaking CUDA's monopoly, achieving high performance on AMD GPUs, and fostering a community of 'elite nerds' through open-source contributions and innovative technologies.
Modular focuses on simplifying GPU programming by developing tools like MAX, an inference framework that supports both NVIDIA and AMD hardware without relying on CUDA. The company also introduced Mojo, a new programming language designed for accelerated compute, offering performance improvements over Python while maintaining usability. By addressing challenges in both training and inference, Modular aims to empower more developers to participate in AI development. The discussion highlights the importance of community engagement and open-source contributions in driving innovation. Additionally, the podcast touches on personal reflections from Chris Lattner, including his experiences leading Modular, managing work-life balance, and engaging with AI coding tools. Modular's business model emphasizes free usage for certain platforms while charging enterprises for GPU support, aiming to democratize access to advanced AI technologies.
02:34
02:34
Focused on proving the impossible without NVIDIA and CUDA
06:55
06:55
Modular's three-year journey started with proving compilation philosophy on CPUs.
11:15
11:15
MAX is open-source, efficient, and avoids CUDA dependencies.
12:52
12:52
Mojo aims to expose the full power of hardware, be portable across vendors, and offer usability.
18:25
18:25
MAX focuses on inference and integrates with Mojo for automatic kernel fusion.
29:16
29:16
Building Flash Attention using Mojo beats reference implementations
32:25
32:25
Modular focuses on AI and Gen AI, inspired by the inaccessibility of large labs' resources.
46:17
46:17
Mojo had a soft start at version 0.1, focusing on real customer needs.
53:17
53:17
DeepSeek's open publication advanced AI progress significantly.
1:00:02
1:00:02
Inference is now part of training due to reasoning models
1:02:31
1:02:31
Losing team members feels more personal at a startup.
1:10:33
1:10:33
Mojo allows expressing all hardware capabilities with readable code.
1:13:24
1:13:24
Use Slack, Reddit, RSS, and arXiv to track research papers
1:15:30
1:15:30
Using a bandsaw is safe and great for teaching kids about woodworking.
1:17:05
1:17:05
More people should program GPUs as it's a huge industry opportunity.
