Janas-LLM processes Qwen3-Next-80B (48 GB) at 37 tok/s on a 32 GB laptop, compared to llama.cpp's 2.8 tok/s. Reading a prompt hits 62 tok/s versus 16 tok/s. Tested on Debian 13 Linux with an Intel Core Ultra 9 185H and integrated Arc GPU, Janas-LLM outperformed llama.cpp on all 13 models compared in Q4_K_M.
Models larger than system RAM run by streaming experts from an NVMe SSD while keeping active ones in memory. The tool is written in C with no dependencies beyond the C library. Designed by Maurizio "camauri" Cammalleri, Janas is experimental software in early development.
Builds are available for Linux x86-64 and Windows 10/11 (x86-64). On Windows 11, Janas-LLM matched Linux performance. The public API and command line may change without notice, though published .jns model files remain compatible across versions.
Source: Hacker News · Summarized by HeadlinesBriefing