Francesco Bonacci and Johnny Franks demonstrate their macOS virtualization stack unlocks 11-16x faster LLM inference in VMs using a Metal capability shim. The layer intercepts GPU capability queries, reporting Apple family 9 and 64KB threadgroup memory instead of stock limits, enabling newer Metal paths. On an M1 Ultra with TinyLlama 1.1B, prompt processing reached 11.08x speedup (98.25% bare-metal) and token generation 16.36x (72.06%).
Testing Google's Gemma 4 12B showed 7.20x prompt and 14.54x generation improvements. MLX-LM showed minimal gains as it was already optimized for stock VMs. The shim runs process-scoped within guests, maintaining paravirtualization while exposing modern GPU features through Apple's Virtualization.framework virtual device.
Source: Hacker News · Summarized by HeadlinesBriefing