He Co-Invented the Transformer. Now: Continuous Thought Machines - Llion Jones and Luke Darlow [Sakana AI]
Machine Learning Street Talk (MLST)
2025/11/23
He Co-Invented the Transformer. Now: Continuous Thought Machines - Llion Jones and Luke Darlow [Sakana AI]
He Co-Invented the Transformer. Now: Continuous Thought Machines - Llion Jones and Luke Darlow [Sakana AI]

Machine Learning Street Talk (MLST)
2025/11/23
Shownote
Shownote
The Transformer architecture (which powers ChatGPT and nearly all modern AI) might be trapping the industry in a localized rut, preventing us from finding true intelligent reasoning, according to the person who co-invented it. Llion Jones and Luke Darlow, ...
Highlights
Highlights
The current trajectory of AI development, dominated by Transformer-based models, may be limiting the field's potential for achieving genuine reasoning capabilities. In this discussion, pioneers Llion Jones and Luke Darlow challenge the status quo, arguing that the industry’s reliance on a single architecture risks overlooking more biologically grounded and adaptive forms of intelligence.
Chapters
Chapters
Stepping Back from Transformers
00:00Introduction to Continuous Thought Machines (CTM)
00:43The Changing Atmosphere of AI Research
01:09Sakana’s Philosophy: Research Freedom
04:13The Local Minimum of Large Language Models
07:45Representation Problems: The Spiral Example
18:30Technical Deep Dive: CTM Architecture
29:12Adaptive Computation & Maze Solving
36:00Model Calibration & Uncertainty
47:15Sudoku Bench: Measuring True Reasoning
1:00:43Transcript
Transcript
Luke Darlow: This episode is brought to you by Indeed. You're ready to move your business forward, but first, you need to find the right team. Start your search with Indeed Sponsored Jobs. It can help you reach qualified candidates fast, ensuring your list...