Adam Marblestone — AI is missing something fundamental about the brain
Dwarkesh Podcast
2025/12/30
Adam Marblestone — AI is missing something fundamental about the brain
Adam Marblestone — AI is missing something fundamental about the brain

Dwarkesh Podcast
2025/12/30
In this deep exploration of neuroscience and artificial intelligence, Adam Marblestone examines the fundamental differences between how human brains and AI systems learn, focusing on the unique mechanisms that give biological intelligence its remarkable efficiency and adaptability.
The discussion centers on the idea that the brain's superior sample efficiency stems not from its architecture but from evolved reward functions—innate 'steering subsystems' that guide learning through hardcoded motivations. These reward mechanisms allow humans to develop desires for novel, evolutionarily unprecedented goals. The brain likely relies on amortized inference, encoding probable solutions rather than raw knowledge, with evolution optimizing reward circuits over detailed structures. Model-free reinforcement learning in the basal ganglia, driven by dopamine-based prediction errors, operates alongside model-based cortical predictions, mirroring societal cultural accumulation. Despite slower hardware, biological systems offer cognitive flexibility that AI lacks. Mapping full brain connectomes could unlock reverse-engineering efforts, while integrating neural data into AI training may improve generalization. Finally, automating formal mathematics with tools like Lean and LLMs promises to transform research, verification, and access to complex problem-solving, though preserving intuition remains a challenge.
09:12
09:12
The cortex learns to predict the steering subsystem's innate responses to guide learning.
41:36
41:36
Social reward functions in the steering subsystem require vision and audio to understand cues for learning
48:01
48:01
Human cultural evolution operates like model-free reinforcement learning over generations.
1:01:42
1:01:42
TD learning may be implemented in the brain through dopamine signaling
1:17:31
1:17:31
Gwern proposed using neural activity patterns as an auxiliary prediction task to improve model generalization