Memorilla introduces a memory system designed for frozen large language models, enabling them to access information beyond their original parameters. The system encodes entire document collections, such as user chats or book chapters, into reusable embeddings once. It then learns to retrieve and place query-specific information within the LLM's input embedding space using a fixed number of query-dependent memory tokens.
A key advantage of this approach is maintaining a short context window while accessing broad knowledge. Using only 16 memory tokens, Memorilla achieves state-of-the-art performance against baselines like Mem0 and Memo RAG across eight benchmarks. It excels in personalization, long-document question answering, and knowledge-intensive tasks. Notably, on the Fact KG benchmark, it reaches 84.33% accuracy using full-context inference, while consuming 108x fewer tokens to represent source documents.
Additionally, Memorilla facilitates continual learning in agentic systems. In a Text World environment, the system reduces the agent's tendency to revisit explored areas, improving the average success rate from 18.6% with full-history in-context learning to 24.2%. Overall, Memorilla allows frozen LLMs to accumulate and use knowledge effectively across long-context interactions, representing a step toward building continually learning AI systems.
Source: Hacker News · Summarized by HeadlinesBriefing