AI Agents Need Memory Calibrated to Their Model

Giving an AI agent a memory of its previous work sounds like an obvious improvement: collect what it learned, place those lessons in its prompt, and let experience guide future decisions. The evaluation of ALTK-Evolve across eight models shows that the reality is more complicated. Agent memory works less like an on-off feature and more like a dose that must be adjusted for each model. Too little guidance may leave useful experience unused, while too much can distract or overwhelm a model that cannot reliably sort the important lessons from the less relevant ones.

ALTK-Evolve builds this memory without retraining the underlying model and without asking people to label examples. The agent first completes tasks, producing records of the steps it took and the results it obtained. The system then examines successful and unsuccessful attempts, turns them into reusable behavioral guidelines, and consolidates those lessons. These may describe strategies that worked, mistakes worth avoiding, or unusual situations that require special handling. During a later task, the guidelines are added to the agent's context either as a complete collection or as a smaller selection chosen for the current problem. In simple terms, the model itself does not change; what changes is the practical advice it receives before acting.

The experiments revealed three broad patterns. Strong models that still had room to improve benefited from receiving the complete memory, including uncommon edge-case lessons. DeepSeek-V3.2, a 671-billion-parameter mixture-of-experts model, improved task completion by 9.5 percentage points with its full self-generated guideline set. Smaller or weaker models performed better when given a compact set of dependable rules plus a few guidelines retrieved specifically for each task. For gpt-oss-120b, this selective method raised task completion by 16.1 percentage points while increasing token use by only 5%. Feeding it the entire guideline collection produced a smaller improvement and used roughly 50% more tokens.

A third group showed no measurable benefit. GLM-5 fell into this observed “saturated” pattern, although the researchers do not claim to know the exact cause. It may already have been close to its limit on the evaluated tasks, the extracted lessons may not have addressed its remaining errors, or the model may have failed to apply them. Model size alone does not determine the correct memory strategy. Available room for benchmark improvement, context-window capacity, model architecture, guideline quality, and the mix of tasks may all influence the result.

The practical lesson is that teams should test how much stored guidance each agent can actually use instead of automatically attaching every lesson it has accumulated. Selective retrieval can improve accuracy while controlling prompt length and operating cost, and prompt caching can make even a complete guideline set affordable in production. The most useful memory is not necessarily the largest one; it is the amount and form of guidance that match the model's capabilities and the task in front of it.