HeadlinesBriefing favicon HeadlinesBriefing.com

ZeroFS brings S3 buckets to POSIX via NFS/9P

Hacker News •
×

A new project, ZeroFS, exposes S3-compatible object storage as POSIX filesystems via NFS and 9P, or as raw block devices over NBD. Writes are handled as immutable objects, with compaction reclaiming deleted data. Before upload, data is compressed and encrypted, and local caching enables microsecond warm reads, a significant improvement over typical S3 latency. The project also includes a web UI for file management and a terminal with a bootable Linux VM.

ZeroFS undergoes rigorous testing, including the POSIX suite, xfstests, kernel builds, and Jepsen's model-based checking. These tests run across multiple protocols to ensure semantic correctness. The system supports various S3-compatible backends, including Amazon S3, Google Cloud Storage, and Azure Blob Storage. Its log-structured engine ensures data integrity and efficient storage management.

The system offers advanced features like immutable segments, honest fsync guaranteeing durability, and optional high availability with a standby server. Data is always encrypted using XChaCha20-Poly1305, with keys wrapped by Argon2id. Compression uses zstd or lz4. The project's architecture facilitates global pools by mirroring S3 regions across block devices, providing resilience against regional outages.

ZeroFS demonstrates impressive performance, with random reads on a warm cache hitting 1.6 microseconds. Small writes average 0.83 milliseconds. This utility allows developers to treat cloud object storage as local storage, simplifying application development and data access patterns.