SharpAI has released SwiftLM, a native Swift inference server that brings 100% Apple Silicon performance to large language models. The server eliminates Python dependencies and the Global Interpreter Lock, compiling directly to a single binary for bare-metal performance on Apple's M-series chips.

Built on Metal and Swift, SwiftLM offers OpenAI-compatible API endpoints with streaming support and smart model routing that loads HuggingFace models directly. The standout feature is its TurboQuant KV cache compression technology, achieving 3.5× memory savings through a hybrid V2+V3 approach that combines hardware acceleration with non-linear Lloyd-Max codebooks for near-zero accuracy loss.

SwiftLM includes experimental SSD streaming for 100B+ parameter Mixture of Experts models, preventing macOS kernel panics by zero-copy swapping from NVMe to GPU command buffers. The project also ships with a native iOS companion app for on-device inference, featuring a model catalog with RAM fit indicators and HuggingFace search. Tested on MacBook Pro with M5 Pro and 64GB unified memory, SwiftLM supports models like Qwen3.5-122B-A10B-4bit and maintains production standards with 4-bit quantization to avoid JSON grammar corruption.