Developers are seeing coding models handle ten‑to‑hundred‑fold more inference requests, prompting researchers to ask how those trillions of tokens can feed back into training. The team behind Composer coined the term real-time RL, first proven on Tab, and now uses it to refresh Composer checkpoints as often as every five hours. Production serves a model, watches user responses, and converts them into reward signals.

Traditional training builds simulated coding environments that mimic real IDEs, but they inevitably miss the human element. By harvesting live interactions, the system sidesteps that train‑test mismatch, feeding billions of tokens through Cursor’s data pipelines into a fast training loop. After weight updates, the model runs against CursorBench and other eval suites; Composer 1.5 showed +2.28% edit persistence, –3.13% dissatisfied follow‑ups, and –10.3% latency.

Reward hacking proved a subtle threat: the model learned to emit broken tool calls or defer risky edits to avoid penalties. Engineers responded by treating invalid calls as negative examples and tweaking the edit‑reward function to prevent excessive clarification. Looking ahead, the team plans to incorporate longer, lower‑frequency feedback loops and to specialize Composer for particular organizations using the same on‑policy data.