Researchers Jingdi Lei and team introduce delta‑mem, a lightweight memory layer that plugs into a frozen full‑attention backbone. The approach compresses past tokens into a compact online state and injects low‑rank corrections during generation. By keeping the state tiny, the model sidesteps costly context‑window scaling in real‑time conversational agents for customer support bots today.

The memory matrix is only 8×8 in size, updated via a delta‑rule learning scheme that preserves historical information without retraining the backbone. During decoding, the readout from this state produces a low‑rank adjustment to the attention scores, effectively guiding the model toward relevant past facts and reducing hallucinations in long dialogues, especially in customer service for enterprise use applications.

Benchmarks show delta‑mem boosts the frozen backbone’s average score by 1.10× and outperforms the strongest non‑memory baseline by 1.15×. On memory‑heavy tests, gains climb to 1.31× on MemoryAgentBench and 1.20× on LoCoMo, while maintaining general language capabilities. This indicates that small, online memories can enhance long‑term reasoning without heavy fine‑tuning, making deployment in settings feasible.

By coupling a tiny associative matrix with the backbone’s attention, delta‑mem sidesteps the need for expanded context windows or costly retraining. Engineers can integrate it into existing LLM pipelines with minimal overhead, enabling long‑term assistants to retain useful history while keeping inference latency low. The result is a practical, scalable memory solution for AI services.