HeadlinesBriefing favicon HeadlinesBriefing.com

DoltLite: SQLite Fork with Git-Style Version Control

Hacker News •
×

My baby is growing up. Just five months after launch, Dolt Lite is Beta, version 0.50.0. Dolt Lite started as a lark. I needed a pet project to test with Steve Yegge’s innovative agent orchestrator, Gas Town. I wanted a real problem, not a toy. We wanted an embedded version of Dolt for years but rewriting Dolt’s storage engine in C or Rust was a bridge too far. SQLite seemed like a logical host for that engine. Could a team of agents pull it off? It only took about 2,000 pull requests but Dolt Lite going Beta proves a team of agents certainly could.

What Is Dolt Lite#Dolt Lite is a fork of SQLite. Everything above the B-tree layer is the same. That means the SQL parser and analyzer, the file system interaction layer, and test harness are stock SQLite. With other Dolt products we had to implement a SQL engine on top of version-controlled storage. Building a SQL engine is hard. With Dolt Lite we got one for the cost of writing a version-controlled storage engine in C.The B-tree layer is swapped out for a Prolly Tree backed by a single file chunk store. Prolly Trees are content-addressed B-trees. This magical data structure powers the version control functionality in all Dolt products. That means you get all the version control features of Dolt and Git in a SQLite package. Have you ever wanted to branch, merge, and diff SQLite? Dolt Lite is for you. Or maybe even more importantly, have you ever wanted a conflict aware SQLite sync engine powered by Git-style push, pull, clone, and fetch? Dolt Lite is for you. Or maybe even more importantly, have you ever wanted a conflict aware SQLite sync engine powered by Git-style push, pull, clone, and fetch? Dolt Lite is for you. You can even use Dolt Hub as your sync backend.

What Does Beta Mean?#So what does Dolt Lite going Beta mean for you? Beta means four things: Storage Format Stability SQL Compatibility Full Version Control Production Performance Reports from the field on functionality and stability are universally positive. SQLite’s testing battery is truly impressive. Dolt Lite passes those tests as well as a custom suite of Dolt oracle tests. Dolt Lite is ready for you to try. Storage Format Stability#The biggest complaint about the development process for Dolt Lite was the storage format bumps. Storage format changes were not backward-compatible, requiring users to stay on the same version or manually dump and reimport their databases. It took 12 format changes to get to Beta. We’ve been on the current format for 57 releases, or over three months of calendar time. This format seems like one we can stick with. The storage format is now stable. Any future breaking changes will have a supported migration path. Dolt Lite Beta means you can adopt without fear of a breaking storage format change. SQL Compatibility#SQLite’s SQL layer is extremely well tested. Dolt Lite passes the vast majority of these tests as well, proving SQL compatibility. Dolt Lite passes 100% of sqllogictest, a suite of 5.8M complex queries. The query layer is the same as SQLite so this is expected. SQLite also ships with a suite of 892,277 TCL-based acceptance tests that test a broader surface of the SQLite API. Dolt Lite passes 99.46% with 4,809 known divergences. Each divergence must have a listed reason. The top reasons are: Tables are keyed by primary key, instead of rowid, to support version control functionality. Some tests directly inspect rowid. Dolt Lite has chunks, not pages. Some tests directly inspect pages. There is no WAL or journal sidecar. WAL and journal tests are skipped. SQLite users will feel right at home with Dolt Lite. Dolt Lite operates like SQLite with extra version control functions. Full Version Control#Dolt Lite implements the core suite of Dolt’s Git-style version control features. Local version control features are supported. Branches, merges, diffs, rebases, cherry-picks, and resets, to name a few. Remote version control is supported: push, pull, clone, and fetch from a custom remote or Dolt Hub. You even have the full Dolt Workbench GUI complete with agent mode. Unleash an a...