HeadlinesBriefing favicon HeadlinesBriefing.com

State Deltas Cut Blockchain Storage 80%

DEV Community •
×

A developer reduced storage for a blockchain node by 80% by switching from full-state snapshots to state deltas. Instead of writing a 140 MB snapshot every 3.5 hours, the system stores one base state and encodes only the differences between subsequent states. This approach was viable due to deep knowledge of the data model and its evolution patterns.

The core problem was scaling storage for new nodes syncing the network. Storing full snapshots would require terabytes of disk space. The delta method, combined with an in-memory cache for the base state, cut runtime costs by 70%. Applying a delta takes only about 200 ms, making the tradeoff acceptable for this workload.

Further optimization targeted the balances field, which contains over two million values. Analyzing the change patterns revealed that most modifications were small or set to zero. This allowed encoding each change with just 2 bits using a custom `BitTagVec` structure, saving hundreds of kilobytes of data per state update and improving overall efficiency.