Meta FAIR’s Self-Improving Memory Test: Model Writes Its Own Memory Prompts, Outperforms Human-Optimized Solutions by 11.4 Points
Meta FAIR has launched a self-improving memory framework for large language model (LLM) continual learning, enabling Kimi-K2.6 to autonomously generate prompt rules for memory loading and offloading. The optimized logic is then distilled into the model weights. Benchmark tests show that this approach scores 56.0, 11.4 points higher than the human-written optimal memory framework.
Continual learning for large language models has long been plagued by the problem of catastrophic forgetting. When models learn new tasks continuously, they either fail to retain knowledge from new inputs or overwrite previously learned knowledge with new content.
Previously, a common industry solution involved adding a separate memory scheduling framework on top of the model, where algorithm engineers manually wrote prompts specifying when the model should load historical memory or unload irrelevant context. The effectiveness of this approach heavily relies on manual tuning experience, and rules often need to be revised when switching models or task scenarios.
Meta FAIR recently announced a new approach to self-improving memory technology.
Researchers used Kimi-K2.6 as the test base model, allowing it to write its own prompts for two types of memory operations: LOAD and OFFLOAD. During testing, the model iteratively adjusted its prompt logic to filter out the best-performing version, after which knowledge distillation was used to directly embed this optimized scheduling rule into the model weights. For subsequent inference, no additional external prompts are required; the model itself has built-in memory scheduling capabilities.
The comparative test was conducted on the Continual Learning Bench, a standard benchmark for continual learning. In the end, this self-improving approach achieved a score of 56.0, while the previously best-performing human-written memory framework only scored 44.6, a difference of 11.4 points.
Based on the existing test results, the memory scheduling logic the model independently develops adapts better to its own inference characteristics than manually designed rules.
Source: MarkTechPost
Author: Michal Sutter
Original Link
发布时间: 2026-11-20 05:46