Two $2,000 machines illustrate unified memory’s advantage: a tower with an NVIDIA RTX 5090 offers 32GB of GDDR6X at 1,792 GB/s, while an AMD Ryzen AI Max+ 395 Strix Halo mini‑PC supplies 128GB of LPDDR5X at 256 GB/s. A 70‑B model needs ~40 GB at 4‑bit precision, exceeding the RTX 5090’s VRAM but fitting comfortably in the mini‑PC’s pool, allowing the model to run but at a token‑rate limited by memory bandwidth.

Unified‑memory architecture eliminates the CPU‑GPU VRAM split, letting the CPU, integrated GPU, and NPU share a single memory pool. This grants capacity up to 512 GB on Apple M3 Ultra and 128 GB on Strix Halo, far beyond discrete GPUs that cap at 32 GB until expensive workstations. However, the same shared pool restricts bandwidth to 120–270 GB/s, whereas GPUs reach 900–1,800 GB/s.

Roofline analysis shows text generation is memory‑bound: decode speed ≈ bandwidth ÷ 40 GB per token for a dense 70‑B model. Strix Halo tops ~6 tokens/s; M3 Ultra reaches ~20 tokens/s; an RTX 5090 could hit 45 tokens/s if the model fit. Mixture‑of‑Experts models mitigate this by activating ~3 B parameters, reducing per‑token bandwidth to ~2 GB and yielding ~72 tokens/s on Strix Halo.

The bottleneck shifts during prompt processing, which is compute‑intensive. Mini‑PCs’ integrated GPUs deliver 95 tok/s on Llama 2‑7B prefill, slowing long‑context workloads. The NPU’s advertised TOPS figures are irrelevant for local LLMs because decode remains bandwidth‑bound and the NPU shares the same memory pool.