Turso is rewriting SQLite in Rust (Interview)
Turso is rewriting SQLite in Rust (Interview)
Turso is rewriting SQLite in Rust (Interview)
In this episode, Glauber Costa, co-founder and CEO of Turso, delves into the ambitious Limbo project—a complete rewrite of SQLite in Rust. The conversation explores the technical and philosophical challenges of reimagining a widely-used database system, the innovative testing methodologies employed, and the broader implications for the future of database technology.
The podcast highlights Turso's Limbo project, which aims to rewrite SQLite in Rust to address limitations of the original SQLite, such as its closed contribution model. The team leverages deterministic simulation testing (DST) to ensure reliability and compatibility, collaborating with Antithesis for advanced testing. The project has gained significant traction, with contributions from nearly 70 developers, including running SQLite in the browser via WebAssembly. Turso plans to transition libSQL, a fork of SQLite, into a server implementation while focusing Limbo on client-side functionalities. The company is also simplifying its branding and product offerings, aiming for production readiness by 2026. The project's open-source nature and community-driven approach are central to its strategy, with a focus on scalability and innovation in database technology.
00:05
00:05
Deterministic simulation testing (DST) is a key innovation in Turso's Limbo project to ensure SQLite's reliability in Rust
01:31
01:31
Brex built around a thousand Retool apps, reducing internal tool development time from months to days
04:37
04:37
Turso's Limbo project aims to rewrite SQLite in Rust, leveraging Rust's safety and performance benefits.
05:10
05:10
SQLite's public domain nature and lack of external contributions make it unique but challenging to modify.
21:07
21:07
The announcement of a complete rewrite in Rust (Limbo) garnered significant interest, highlighting the demand for a better version of SQLite.
25:28
25:28
Deterministic simulation testing is crucial for rewrites, enabling easier debugging and reliability when integrated from the start.
28:44
28:44
Avoiding external crates with I/O operations is crucial for maintaining control in deterministic simulation testing.
30:58
30:58
Temporal abstracts scalability complexities, enabling developers to focus on reliable workflows
32:57
32:57
DST ensures confidence in the rewritten SQLite by generating random queries and injecting failures deterministically for replayable testing.
35:23
35:23
Limbo ensures bytecode compatibility with SQLite by generating random SQL statements for testing
40:08
40:08
Turso Cloud enables serverless SQLite with async operations, hosted on S3 for browser environments.
44:31
44:31
Limbo, a project rewriting SQLite in Rust, unexpectedly gained 1,000 GitHub stars and attracted top engineers, leading to a strategic shift in the company's focus.
46:49
46:49
Limbo's ability to run in the browser via WebAssembly marks a significant development
51:19
51:19
Timescale enhances PostgreSQL with extensions like pgvectorscale and pgai for AI use cases.
56:14
56:14
SQLite's compact codebase makes rewriting it in Rust within a year feasible, with a dedicated team of engineers.
57:22
57:22
The new Rust-based SQLite prioritizes read-heavy workloads and stability for early production use
58:36
58:36
The company plans to replace SQLite with Limbo, a Rust-based rewrite, aiming for production readiness in 9 to 12 months.
1:01:21
1:01:21
Turso rebrands its embedded database to align with its cloud offering, simplifying its brand identity.
1:03:37
1:03:37
Turso aims to replace SQLite with Limbo, a Rust-based rewrite, while keeping client-side features open-source
1:06:08
1:06:08
Turso's Limbo aims to reach a billion SQLite databases in a year, focusing on gradual adoption rather than immediate replacement.
1:07:18
1:07:18
Basic read operations from SQLite files are already functional in the Rust rewrite.
1:08:50
1:08:50
Turso is rewriting SQLite in Rust for scalability and testing, with a closed-source server and open-source client.
1:11:38
1:11:38
Excitement and support for the ongoing project, with plans to revisit the topic in a year
1:12:03
1:12:03
Limbo, a Rust-based rewrite of SQLite, is Turso's ambitious project to replace the database system.
1:14:33
1:14:33
Preston Thorpe, a prison contributor, worked on Limbo, a Rust-based SQLite project, through a remote work program
