HeadlinesBriefing favicon HeadlinesBriefing.com

27.5KB WebGPU Syntax Highlighter gpu-lexer

Hacker News •
×

gpu-lexer is a 27.5KB language-agnostic WebGPU syntax highlighter that splits source code into parts and uses a tiny GPU model to label each token from surrounding context, without relying on language-specific grammars. It guesses token types even for unseen languages or syntax. On held-out files, 12.57% of its token labels differ from Shiki, measuring agreement—not correctness.

Performance benchmarks on 10× three.min.js (5.56M characters) show competitive browser runtime. Bundle size remains minimal across all languages, using the same compressed bundle for JavaScript, TypeScript, CSS, HTML, JSON, and Markdown. Unlike Shiki, Prism.js, or Highlight.js, it does not return HTML but token spans.

The tool is an experiment, not a replacement for grammar-based highlighters. Scores are weighted by language popularity from GitHub Innovation Graph top 25 for 2026-Q1. Live demos support 75 languages.

Key metrics: 27.5KB bundle, 12.57% label divergence from Shiki, tested on MacBook Pro with M4 Pro GPU.