HeadlinesBriefing favicon HeadlinesBriefing.com

A Million Agents Is a Distributed Systems Problem

Hacker News •
×

I think about agents the way I think about people. One agent is a worker. A thousand agents is an organization, and an organization of machines is a distributed system. People keep telling me agents can run forever. Technically that's true. You can call a model in a loop until your credit card declines. But the agent still runs on finite stuff: tokens, context, compute, memory, tools, money. Something always runs out. Humans aren't that different. We work for a while and get tired. Our working memory is tiny. We forget things. So we write the important parts down, sleep, and come back the next morning with a clear head and the same identity. Machines have the same constraint in a different shape. A box runs out of CPU. A pod runs out of memory. Nobody fixes that by assuming every process should live forever. We schedule work over the resources we have. I don't see why intelligence gets a pass. An agent can work for a while, write down what matters, clear its context, and let itself or another agent pick up from there. We run our own coding agents on VPSes at Ins Forge, and they get killed, restarted, and run out of context all the time. The failures that actually hurt are the ones where the plan lived only inside the agent's context window.

Adding agents helped parallel work by up to 80.9% and hurt sequential work by 39% to 70% in Google Research's 180-configuration study. The shape of the task decides. An orchestrator cut error amplification from 17.2× to 4.4× in the same study. Coordination is a real job. In Silo-Bench (ACL 2026), teams of 2 to 100 agents talked plenty and reasoned badly. The hardest tasks hit zero success at 50 agents. So the durable thing should be the state, not the agent. Schedule agents like processes and recover them like nodes.

Agents are starting to look like processes. This stopped being an analogy a while ago. There's a whole research line building it. AIOS, an "LLM Agent Operating System" out of Rutgers, opens with the problem in one sentence: "Allowing unrestricted access to LLM or tool resources can lead to inefficient or even potentially harmful resource allocation and utilization for agents." Their answer is a kernel. Every agent request gets broken into system calls, a scheduler decides whose call runs next, and a context manager snapshots an agent mid-task so it can be interrupted and resumed. They report up to 2.1× faster execution when serving agents built on existing frameworks. Scheduling, context switching, memory management, storage, access control. That's an operating system. The processes just happen to think.

It pays one level up too. LLM-as-Scheduler, from ACL 2026, starts from the observation that most queries don't deserve a heavy multi-agent workflow, and lets a scheduler pick the workflow per query. They got 43% fewer tokens and more than 36% lower end-to-end latency, for at most a 1.4 percentage-point drop in accuracy against a strong fixed workflow. So one agent looks like a process, and thousands of processes need a scheduler. Fine. But scheduling compute is only half of it. You also have to coordinate the agents with each other, and that's where it gets interesting. Ten people coordinate. Ten thousand invent managers. Ten people can coordinate themselves in a room. A thousand people can't all talk to each other and independently decide what the company should do, so we invented teams, managers, departments, and eventually a CEO. Managers exist partly because coordination is itself work, and someone has to do it. I assumed agents would have the same problem. Now there's data. Google Research and MIT ran a controlled study of 180 agent configurations across five architectures and three model families. Their headline: "the 'more agents' approach often hits a ceiling, and can even ..."