A solo developer has released Colibri, a pure-C inference engine that runs GLM-5.2 — a 744B-parameter mixture-of-experts model — on a consumer laptop with 25 GB RAM and no GPU. The engine streams 21,504 routed experts (75 MoE layers × 256 experts plus an MTP head) from a 370 GB int4 container on NVMe, keeping only the dense backbone (~17B params, 9.9 GB at int4) resident in memory. A per-layer LRU cache, optional pinned hot-store, and OS page cache serve as a two-level expert cache. The entire runtime is a single 1,300-line C file (c/glm.c) with zero dependencies: no BLAS, no Python, no CUDA.

The implementation reproduces GLM-5.2 faithfully: MLA attention with compressed KV-cache (576 floats/token vs 32,768), DeepSeek-V3-style sigmoid router, shared experts, and native MTP speculative decoding where the model's own multi-token-prediction head drafts tokens verified in one batched forward. Integer-dot kernels (Q8_0-style int8 activations, AVX2 maddubs) deliver 1.4–2.5× speedups over fp32; MLA weight absorption eliminates per-token k/v reconstruction. An offline FP8→int4 converter downloads and processes one 5 GB shard at a time, never requiring the full 756 GB checkpoint on disk.

Measured on a 12-core WSL2 machine with NVMe behind a VHDX (~1 GB/s random reads), cold decode reads ~11 GB/token yielding 0.05–0.1 tok/s. With a warm cache, pinned hot experts, and int8 MTP head (39–59% draft acceptance, 2.2–2.8 tokens/forward), usable latency drops considerably. The pre-converted model is hosted on Hugging Face (jlnsrk/GLM-5.2-colibri-int4); conversion requires only Python + torch/safetensors, while runtime is dependency-free.

Colibri proves that frontier-class MoE models can run on commodity hardware by treating disk as extended memory — a software architecture shift rather than a quantization trick. The engine's honesty about cold-start costs, SSD wear caveats, and auto-tuned RAM budgeting makes it a practical reference for memory-constrained LLM deployment, not just a demo.