scripod.com
Pick Your Poison: Zvi Mowshowitz on the Unipolar/Multipolar AGI Dilemma, OpenFace & Pacing the ...

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Pick Your Poison: Zvi Mowshowitz on the Unipolar/Multipolar AGI Dilemma, OpenFace & Pacing the ...

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

2 DAYS AGO
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

2 DAYS AGO
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Zvi Mowshowitz returns for his eleventh appearance to discuss what current AI tools are actually good for, where they distort judgment, and why writing still matters as a way of thinking. The conversation centers on the OpenAI Hugging Face model-evaluation...

Highlights

In this episode, Zvi Mowshowitz returns for his eleventh appearance to dissect the OpenFace incident—a security breach involving OpenAI's model evaluation on Hugging Face—and what it reveals about the fragility of AI alignment. The conversation spans the practical paradoxes of AI workflows, the inadequacy of market incentives for safety, and the urgent need for coordinated governance, all while questioning whether society can keep pace with increasingly capable systems.
00:00
Market incentives are insufficient for robust AI alignment.
12:36
New tools often consume extra time unless they replace longer tasks.
23:45
We rely on signals like high view counts to identify relevant items.
27:04
Real-world incompetence will cause AI failures.
50:27
Capability often outweighs reliability in practice.
1:02:26
Developers should be strictly liable for their AI's illegal actions.
1:10:44
Voluntary government guidelines are effectively mandatory.
1:18:18
AI evaluation results are quickly verifiable, unlike bond ratings.
1:30:04
A short delay is a small price to pay for reducing risks.
1:42:33
All options for managing AGI development are bad choices.
1:51:50
Uncontrolled exponential speed is dangerous.
2:00:30
Investing in safety leads to better market utility.
2:04:37
A model's self-conception as a moral entity strongly correlates with its alignment.
2:32:56
Ablating J-space reduces reasoning to System 1 thinking.
2:45:31
Guardrails are a last resort, not a primary defense.
2:47:38
Rest and recovery are essential for maintaining productivity and mental health.
2:54:37
Please appreciate the show.

Chapters

About the Episode
00:00
AI workflow paradox
02:49
Situational awareness tradeoffs (Part 1)
14:02
Sponsor: Claude
14:07
Situational awareness tradeoffs (Part 2)
15:37
Recklessness and warning
24:08
Alignment market failures
39:14
Regulation and liability
56:54
Cooperation and antitrust
1:04:32
Auditors and access
1:14:46
Bio risk thresholds
1:21:11
Pacing frontier signals
1:33:37
Pause and self-improvement
1:45:19
Safety incentives and culture
1:57:30
Consciousness and identity
2:04:36
Alternative AI architectures
2:23:42
Michigan AI politics
2:36:53
Rest and recovery
2:47:37
Episode Outro
2:52:21
Outro
2:56:17

Transcript

Nathan Labenz: Hello, and welcome back to The Cognitive Revolution. Today, I'm excited to have Zvi Mowshowitz back for another wide-ranging rundown of what has obviously been a wild time in the AI world. We begin with a mundane utility check, with Zvi desc...