scripod.com
Will scaling work? [Narration]

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Will scaling work? [Narration]

Dwarkesh Podcast

2024/01/19
Dwarkesh Podcast

Dwarkesh Podcast

2024/01/19
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

This is a narration of my blog post, Will scaling work?. You read the full post here: https://www.dwarkeshpatel.com/p/will-scaling-work Listen on Apple Podcasts, Spotify, or any other podcast platform. Follow me on Twitter for updates on future posts and...

Highlights

This podcast explores the central debate in artificial intelligence: whether simply scaling up current AI models with more data and compute will be enough to achieve human-level intelligence, or if fundamental breakthroughs are still needed.
00:00
We will soon run out of high-quality data
03:28
Scaling a simple transformer could produce AGI.
10:21
Benchmarks test memorization, not reasoning
17:14
Scaling has historically worked despite lacking full theoretical understanding
23:33
Insight-based learning is incompatible with gradient descent.

Chapters

The Great Scaling Debate: Will More Data and Compute Lead to AGI?
00:00
Overcoming the Data Wall: Can Synthetic Data and Self-Play Save the Day?
03:28
Benchmark Busters or Just Parrots? The Fight Over What AI Tests Really Measure
10:21
Is Compression Intelligence? Comparing AI Scaling to the Unstoppable March of Moore's Law
17:14
The Final Verdict: A 70% Chance of Transformative AI by 2040, But What Are the Hurdles?
20:30

Transcript

Dwarkesh Patel: Hey everyone, this is a narration of a blog post I wrote called Will Scaling Work? You can find the full version on my website, dwarkeshpatel.com. It was originally published December 26th, 2023. Will Scaling Work? When should we expect AGI...