Pair it with a local coding agent. Run llama serve, install the pi‑llama plugin, and launch Pi. It will automatically discover your local model. No config, no API keys. Files stay on your machine, requests never leave it.

Serve a model: llama serve

Install the pi‑llama plugin: `pi install git:github.com/huggingface/pi‑llama`

Run Pi, everything is set: `pi`

Optimized tú any hardware. From your laptop to a cluster, llama.cpp runs on whatever you have. The same binary ships with hand‑tuned kernels for every GPU and CPU, ensuring peak performance on Apple Silicon M Ultra, RTX 4090, A100, Jetson H100, MI300, Radeon RX B200, and Intel Arc. The framework automatically selects the best execution path, so you need not tweak any settings.

The setup is minimalist: no external services, no configuration fuss. Once Pi is launched, the plugin hooks into yourinline environment, loads the model, and serves requests instantly. It’s ideal for developers who want full control over data, speed, and hardware scaling, while keeping everything on‑premise and secure.