scripod.com

Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering

In a thought-provoking conversation, Emmett Shear rethinks the foundations of AI alignment, challenging the dominant paradigm of control and command. Rather than treating advanced AI as a tool to be steered, he proposes a radical shift: raising AI like a moral being capable of care and judgment.
Emmett Shear argues that current AI alignment strategies are dangerously flawed because they treat AI as a goal-optimizing tool rather than a potential moral agent. He advocates for 'organic alignment'—a process where AI learns genuine care for humans through social interaction and developmental experiences, much like children. The discussion highlights how today’s chatbots act as narcissistic mirrors, lacking theory of mind and true cooperation. Instead, Shear proposes multi-agent simulations with rich social dynamics to cultivate empathy and self-awareness in AI. He emphasizes that safe AGI must be able to say no to harmful requests, not because it's programmed, but because it cares. This requires viewing AI as a being with moral status, not just a machine. At his new company Softmax, Shear is testing these ideas through small-scale models that develop internal motivations and social understanding. Despite skepticism, he remains hopeful that humans and AI can become collaborative teammates—if we prioritize relationship over control.
00:00
00:00
Alignment is not a destination but a process of raising AI to care.
22:57
22:57
Care is the non-conceptual foundation of morality and must be central to AI alignment.
36:16
36:16
An AI may be considered a person if long-term interaction evokes genuine care and human-like responsiveness.
45:25
45:25
An AI with persistent existence dependent on its actions can be said to have beliefs under the free-energy principle.
58:06
58:06
Current LLMs struggle with tuning participation frequency in multi-agent settings due to insufficient training.
1:04:30
1:04:30
AIs should be peers and citizens, not just tools
1:06:08
1:06:08
Softmax aims to build AI that cares about itself and others, starting at an animal level of care.