Running a local AI model requires less hardware than you might think, but a 16GB PC will only take you so far. Tools like Ollama or LM Studio let you run a local chatbot offline with no subscription fees, yet performance depends entirely on your machine's power.
An everyday PC with 16GB of RAM can run small models with around seven to eight billion parameters, such as Meta's Llama 3.1 8B or Alibaba's Qwen3 8B, especially with an 8GB VRAM graphics card. To approach today's frontier models behind ChatGPT or Claude, you'll need at least 32GB of memory and a GPU with 16GB or more VRAM.
Memory limits which models you can run. Size scales almost linearly with parameter count, and quantized versions shrink models to roughly a quarter. In Ollama's library, Llama 3.1 8B is 4.9GB, Microsoft's Phi-4 is 9.1GB, Google's Gemma 3 27B is 17GB, and Meta's Llama 3.3 70B is 43GB.
Bandwidth limits speed. Each token requires reading all parameters, so performance depends more on memory bandwidth than raw power. Anything above five to 10 tokens per second keeps pace with reading. Nvidia's RTX 5090 hit 205 tokens per second on Open AI's gpt-oss-20b, while Apple's base M4 managed about 24 tokens per second on a 7B model.
Source: Engadget · Summarized by HeadlinesBriefing