HeadlinesBriefing favicon HeadlinesBriefing.com

Apple Silicon VMs Boost LLM Inference 11-16x with Llama.cpp

Hacker News •
×

Francesco Bonacci and Johnny Franks demonstrate their macOS virtualization stack unlocks 11-16x faster LLM inference in VMs using a Metal capability shim. The layer intercepts GPU capability queries, reporting Apple family 9 and 64KB threadgroup memory instead of stock limits, enabling newer Metal paths. On an M1 Ultra with TinyLlama 1.1B, prompt processing reached 11.08x speedup (98.25% bare-metal) and token generation 16.36x (72.06%).

Testing Google's Gemma 4 12B showed 7.20x prompt and 14.54x generation improvements. MLX-LM showed minimal gains as it was already optimized for stock VMs. The shim runs process-scoped within guests, maintaining paravirtualization while exposing modern GPU features through Apple's Virtualization.framework virtual device.