This conversation follows the intellectual development of deep learning and Sutskever’s view of what current models have learned. He explains why compression and next-token prediction can require an internal model of the processes behind language, why scaling works only when paired with productive methods, and why transformers accelerated GPT. He distinguishes knowledge acquired during pre-training from reliable behavior shaped by human feedback, arguing that hallucinations may eventually be eliminated. The discussion also covers multimodality, text-only representations, efficient learning, hardware economics, and a possible future of high-bandwidth democratic participation assisted by AI.