HeadlinesBriefing favicon HeadlinesBriefing.com

Slotstream Runs Qwen3.8-Flash-Next on Low-Memory Macs

Hacker News •
×

Slotstream enables running the Qwen3.8-Flash-Next 125B parameter Mixture-of-Experts model on Macs with limited RAM by streaming model experts from SSD. The model requires 104 GB on disk at 4-bit precision but runs using only a fraction of that in memory via MLX and Swift. It provides an Ollama-compatible API and is distributed as a single Swift binary with no Python dependencies.

On a 48 GB M5 Pro, it achieves ~12 tokens/second with a peak memory usage of 32 GB in auto mode. Minimum requirements include Apple Silicon, macOS 14+, and ~110 GB of free disk space, with a 512 GB Mac being the realistic baseline. The tool includes a 'doctor' mode to simulate performance across different RAM configurations.

Installation is via a one-line shell script, and weights are downloaded on first use with integrity checks via SHA-256 hashes. Pull operations use multiple TCP connections for faster transfers, and interruptions are safe due to resumable downloads. Future work includes implementing the MTP module for speculative decoding.