Stanford's new 'Memorilla' crams a whole library into just 16 tokens for your frozen AI.

Researchers from Stanford's SNAP lab have open-sourced Memorilla, a 194-million parameter trainable memory module designed for large language models (LLMs) that are kept 'frozen' or untrained. The system works by having a separate, smaller module read an entire collection of documents and condense the most relevant information into just 16 tokens, which are then fed to the main, frozen LLM alongside a user's question. This allows a model like the frozen Qwen3-8B to answer complex questions without needing to re-process massive amounts of text for every query.

The approach solves a key bottleneck: using a full document's text as context is slow and costly, while simple retrieval can miss subtle clues. In tests on the FactKG benchmark, the frozen model with Memorilla scored 84.33%, outperforming the method of using every relevant document in context (80.43%) while using 108 times fewer tokens. The project, released under the Apache 2.0 license with full code and training recipes, represents a clever architectural hack that could make large, powerful models significantly more efficient and practical for real-world applications that require deep knowledge bases.