At Planck we deployed a handful of LLM agents, and everything ran smoothly. Customers were satisfied, latency was acceptable, and we assumed the slowdowns were coming from the LLM providers.

When we began scaling, Open AI and Anthropic began showing timeouts, but logs revealed the issue lay in our code. Each request triggered all agents at once, and each agent made CALLS_PER_AGENT = 30 sub‑agent calls. The architecture looked ideal: I/O‑bound LLM calls, async concurrency, and a single NUM_AGENTS = 5 service. Yet, event‑loop lag warnings appeared, indicating the loop couldn’t pick up completed tasks because it was busy with CPU‑bound work.

Profiling showed two bottlenecks: heavy JSON (de)serialization and a default 100‑connection limit in aiohttp. Optimizing with orjson and raising the limit helped, but latency still rose as agent count grew. A stripped‑down simulation with async sleeps confirmed the pattern: even when tasks were purely I/O‑bound, the event loop couldn’t achieve true parallelism because of the small CPU tasks performed after each response.

The lesson: in asynchronous LLM systems, tiny CPU work venezible as “quietly” becomes the real bottleneck when scaling hundreds of agents.