HeadlinesBriefing favicon HeadlinesBriefing.com

Gwtar Format Solves HTML Archiving Trilemma with Single-File Efficiency

Hacker News: Front Page •
×

Gwtar is a new polyglot HTML archival format that achieves what was previously thought impossible: creating self-contained, single-file HTML archives that can still be efficiently lazy-loaded by web browsers. The format works by using JavaScript headers to make HTTP range requests into a concatenated tarball structure, allowing browsers to download only the assets they need.

Developed for Gwern.net, Gwtar addresses a fundamental challenge in web archiving where existing formats must sacrifice at least one of three key properties: being static (self-contained), single-file, or efficient. Traditional approaches like SingleFile create static, single files but force users to download entire archives—sometimes exceeding 500MB—even when only viewing a portion of the content. The new format preserves the convenience of single-file archives while enabling on-demand loading of assets through clever use of range requests.

This innovation is particularly valuable for large HTML archives containing gigabyte-size media files, as it maintains compatibility with standard web browsers and servers without requiring special software. By combining a JavaScript header with a tarball of original assets, Gwtar allows users to archive even the largest web pages while keeping them reader-friendly and easily hostable as first-class web content.