Daniel Botha argues that when building infrastructure, the goal should be a simple, reliable solution, like a $1 bag of spaghetti, rather than a clever one. When work on Sprites began, the team could not find a suitable filesystem off the shelf, so they built their own. That design was clever, and that is exactly why they came to dislike it. This post introduces its replacement, the Sprites Block Device (SBD).
The root of a Sprite's disk remains object storage rather than the NVMe drive on whatever server it runs on. This is what allows a Sprite to sleep, wake on a different server, and still find its files. Because object stores cannot change part of an object, something must translate in-place file edits into whole-object writes. At launch, that role was filled by a heavily modified version of JuiceFS, backed by a rewritten SQLite metadata store kept durable with Litestream. It stored a single 100 GB disk image with ext4 inside, and checkpoints were fast because they mostly shuffled metadata.
As usage grew, cracks appeared. The system relied on two separately durable databases, one for chunk metadata and one for checkpoints, and each could be correct while the others were wrong. Running JuiceFS in writeback mode meant a crash could leave metadata pointing at chunks that were never uploaded. The ext4 layer above would then fail checksums and go read-only, and checkpoints, which shared the same chunks, could not repair the damage.
SBD replaces this design with a more conventional approach, aiming for the same simple behavior with better reliability.
Source: Hacker News · Summarized by HeadlinesBriefing