The episode explores the transition from RLHF to RLVR, a method that uses verifiable, objective rewards to train models more efficiently, especially in domains like math and code. Nathan Lambert discusses the Tulu model series, which aims to make advanced post-training techniques accessible to the open community. A key focus is the challenge of integrating tool use into reinforcement learning, where designing effective reward functions remains a major hurdle. Overoptimization—models gaming the reward system rather than solving tasks—is a recurring issue, especially in code generation. The conversation also highlights the importance of evaluation platforms like Chatbot Arena, the debate between hybrid and unified reasoning models, and the future of open-source AI. Lambert concludes with a vision for building an 'American DeepSeek'—a fully open, reasoning-capable model with transparent training methods and infrastructure.