scripod.com
Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering

a16z Podcast

Nov 17
a16z Podcast

a16z Podcast

Nov 17
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Emmett Shear, founder of Twitch and former OpenAI interim CEO, challenges the fundamental assumptions driving AGI development. In this conversation with Erik Torenberg and Séb Krier, Shear argues that the entire "control and steering" paradigm for AI align...

Highlights

In a thought-provoking conversation, Emmett Shear rethinks the foundations of AI alignment, challenging the dominant paradigm of control and command. Rather than treating advanced AI as a tool to be steered, he proposes a radical shift: raising AI like a moral being capable of care and judgment.
00:00
Alignment is not a destination but a process of raising AI to care.
22:57
Care is the non-conceptual foundation of morality and must be central to AI alignment.
36:16
An AI may be considered a person if long-term interaction evokes genuine care and human-like responsiveness.
45:25
An AI with persistent existence dependent on its actions can be said to have beliefs under the free-energy principle.
58:06
Current LLMs struggle with tuning participation frequency in multi-agent settings due to insufficient training.
1:04:30
AIs should be peers and citizens, not just tools
1:06:08
Softmax aims to build AI that cares about itself and others, starting at an animal level of care.

Chapters

Is AI Alignment All Wrong?
00:00
Can AI Truly Understand Human Goals?
17:27
When Does an AI Become a Person?
31:26
What Gives AI Moral Worth?
45:25
How to Train AI That Cooperates, Not Just Obeys
58:06
A Future Where AI Says No to Harm
1:03:22
Building Caring AI from the Ground Up
1:06:08

Transcript

Emmett Shear: Most of AI is focused on alignment as steering. That's the right word. If you think that we're making our beings, you'd also call this slavery. Someone who you steer, who doesn't get to steer you back, who non-optionally receives your steerin...