HeadlinesBriefing favicon HeadlinesBriefing.com

Local LLM Costs Measured: RTX 3090 Energy per Million Tokens

Towards Data Science •
×

The author measured actual GPU electricity costs for eight local LLMs on a single RTX 3090 using real power sampling from nvidia-smi every 10 seconds, integrated over exact run windows at Bulgarian electricity tariffs (0.30 BGN day / 0.18 BGN night, 1 BGN ≈ €0.5113). The benchmark used a fixed 240-second workload of 256-token generations via ollama with Q4_K_M quantization.

Five of eight models came in cheaper than a hosted Flash-class API at €0.55 per million output tokens, while three were more expensive — and the cost ranking didn't follow parameter count. The cheapest was gemma3:1b; the priciest local model was GLM-4.5-Air (106B). Other models tested included gemma4:26b, gemma3:27b, DeepSeek-R1-Distill (32.8B), Seed-OSS (36B), Devstral (24B), and Qwen3-Coder (30.5B).

The method used a custom Home Lab Monitor dashboard to track GPU energy per run, converting kWh to euros via the day/night tariff. Results show local inference isn't universally cheaper; marginal energy cost varies significantly by model architecture and size, challenging the folk wisdom that local tokens are "basically free."