Running Large Language Models (LLMs) locally on your own hardware can seem free, with electricity costs being negligible. However, a recent analysis using Apple Silicon measured the actual energy consumption and cost per token.

The study used a tool called "Token Watt" to measure power draw on an M3 Ultra Mac Studio with 96 GB of unified memory. Five different LLMs were tested under sustained generation to capture real-world energy usage priced at $0.31/kWh.

A surprising finding was that parameter count does not directly correlate with cost. The most expensive model to run per token was a 27-billion-parameter dense model, costing significantly more than larger mixture-of-experts (MoE) models. This is because cost is determined by power draw divided by throughput, and dense models activate all parameters per token, leading to higher data movement and thus higher costs. MoE models, by contrast, activate only a fraction of their parameters, making them more efficient.

In real-world, less saturated traffic, the cost disparity widened. The dense 27B model proved to be roughly ten times more expensive per token than the MoE models, reinforcing the idea that model architecture and memory access patterns are critical factors in determining operational costs, not just sheer size.