scripod.com
Columbia CS Professor: Why LLMs Can’t Discover New Science

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Columbia CS Professor: Why LLMs Can’t Discover New Science

The a16z Show

2025/10/13
The a16z Show

The a16z Show

2025/10/13
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

From GPT-1 to GPT-5, LLMs have made tremendous progress in modeling human language. But can they go beyond that to make new discoveries and move the needle on scientific progress? We sat down with distinguished Columbia CS professor Vishal Misra to discus...

Highlights

The rapid evolution of large language models has sparked intense debate about their potential to transcend mimicry and contribute meaningfully to scientific discovery. In this conversation, a leading computer scientist unpacks the inner workings of LLMs, challenging assumptions about their capabilities and revealing why they may be fundamentally constrained in generating true innovation.
02:50
LLMs create Bayesian manifolds where low entropy means high confidence
12:45
RAC was accidentally invented while trying to fix StatsGuru and has been in production since 2021.
26:51
LLMs can only generate what they've been trained on, not truly self-improve.
30:40
AGI must create new science, not just interpolate training data.
44:12
An LLM creating a large software project without supervision would convince me we're close to AGI

Chapters

Can AI Really Discover New Science, or Is It Just Mimicking?
00:00
How an Accidental System Revealed the Hidden Potential of LLMs
12:45
What Matrix Abstraction Tells Us About How LLMs Learn From Context
19:44
Why No Amount of Data Can Enable LLMs to Invent Relativity
30:40
What Would It Take for an AI to Truly Break New Ground?
44:12

Transcript

Vishal Misra: Any LLM that was trained on pre-1915 physics would never have come up with a theory of relativity. Einstein had to sort of reject the Newtonian physics and come up with a space-time continuum. He completely rewrote the rules. AGI will be when...