HeadlinesBriefing favicon HeadlinesBriefing.com

DjVu Format: Superior Document Compression from Deep Learning Pioneers

Hacker News: Front Page •
×

DjVu, created by Yann LeCun, Léon Bottou, and Yoshua Bengio alongside other researchers, remains a vastly superior document format for scanned books and mathematical papers compared to PDF. While PDF evolved from PostScript with metadata layers, DjVu excels at compressing mixed text and image content by intelligently discarding unnecessary data while preserving essential information like text and plots.

PDF typically stores scanned documents as uncompressed bitmaps or JPEG images, which perform poorly on text due to their cosine transform approach. In contrast, DjVu uses innovative compression techniques including JPEG2000-style wavelets for backgrounds and a clustering algorithm called JB2 for foreground text. The format also employs the ZP-coder for arithmetic compression, offering both efficiency and decoding performance.

Despite its technical superiority, DjVu failed to gain widespread adoption due to lack of native support in operating systems and e-readers. While PDF readers came pre-installed on most devices, finding and installing DjVu readers proved challenging. The format still finds use in specialized communities like Russian warez spaces and among users of rooted e-ink devices, but mainstream publishers rarely support it. As the article notes, scanned books in DjVu format may actually be more valuable than all PDFs combined, representing preserved knowledge from the pre-digital era.