HeadlinesBriefing favicon HeadlinesBriefing.com

Run 80B Qwen on Mac & 35B on iPhone in 4‑GB RAM

Hacker News •
×

Run 35B and 80B Qwen models on ordinary Apple devices, including iPhones. Swiftlet is a Swift + Metal runtime for the Qwen3‑Next and Qwen3.5/3.6 hybrid model family. It keeps only the small dense core of a model resident in memory and streams the routed Mixture‑of‑Experts weights from storage on demand. The result: 35B runs on an M5 Mac at 2.6 GB RAM and 7–11 tok/s; the 80B uses 4.3 GB RAM and 4.5–5 tok/s. The 35B also runs on an iPhone 17 with ~2.5 GB RAM at ~1 tok/s, the first native phone run of this class. Status: working end‑to‑end, correct output, focus on kernel speed. About 3 B parameters are active per token, so the models chat like large ones but recall facts like small ones.

Quick start: clone Swiftlet, build, download the 35B >< 80B container from Hugging Face, then chat or generate locally. Requirements: Apple Silicon, macOS 14+ or iOS 17+, free SSD space (18 GB for 35B, 42 GB for 80B). Swiftlet caches hot experts in a bounded pool with LFU plus recency eviction; fetch is one pread from SSD, no mmap. It runs the forward pass on Metal with runtime‑compiled shaders, so no toolchain required. 75 % of layers use gated Delta Net linear attention with fixed‑size recurrent state, eliminating growing KV cache.

Swiftlet offers a CLI, an OpenAI‑compatible server, and an iOS app (Priv AI) for on‑device chat. All kernels are validated against mlx‑lm references; containers are byte‑verifiable. The project is open source under Apache 2.0, built with ~10k lines of Swift and Metal.