Building useful local LLM agents requires solving problems that cloud APIs hide. The author constructed an automated single-cell RNA-seq analysis agent using open-weight models on institutional HPC hardware, choosing local hosting for reproducibility and provenance tracking that scientific workflows demand. Unlike prompt-based Skills, this approach maintains structured records of parameters, filtering decisions, and clustering results.

The initial implementation struggled with performance: 50-80 tool calls per analysis meant the model processed roughly 36k tokens of system prompt and tool schemas on every single iteration. Each request took 10-15 seconds, and long sessions eventually crashed from context overflow errors. Two vLLM optimizations addressed these bottlenecks. CUDA graphs eliminated hundreds of GPU kernel dispatches per token, reducing latency by 20-25% through a single replayable instruction sequence.

Memory efficiency improvements came from FP8 quantization, which cut model weights from 56GB to 31GB for Qwen3.6-27B. This freed memory for KV cache storage, dramatically extending context capacity. While Gemma 4-31B requires 1.1MB per token, Qwen needs only 256KB, enabling roughly 320k tokens of context on the same hardware instead of 74k.

These infrastructure optimizations transform local LLM agents from experimental curiosities into practical tools for scientific computing, though developers must now solve problems that cloud providers previously handled. The trade-off between control and convenience becomes starkly apparent when building production-grade autonomous agents.