Hugging Face published an evaluation of agentic memory using ALTK-Evolve, a method where agents distill self-mined guidelines from past trajectories without weight updates or human annotation. Testing across eight models on the AppWorld benchmark revealed that agentic memory performance is directly tied to model calibration rather than simply turning memory on. Strong models with capacity benefit from the full guideline set; for instance, DeepSeek-V3.2 gained 9.5 percentage points in task completion when given all its guidelines. Conversely, weaker models become overwhelmed by large sets and perform better with a compact core combined with per-task retrieval, such as gpt-oss-120b gaining 16.1 percentage points while increasing token use by only five percent. Meanwhile, fully saturated models showed no measurable gains from added memory. The findings indicate that the optimal memory dose is shaped by capability, benchmark headroom, and architecture. Furthermore, strict scenario goal completion metrics often saw even larger jumps than task completion alone. While injecting full guideline sets inflates input tokens per step, production techniques like prompt caching can keep costs manageable. Future work includes developing outcome-trained selectors and exploring teacher-distilled memory for extremely weak models.
How Much Memory Does Your Agent Actually Need?
A new Hugging Face evaluation examining agentic memory shows that the optimal amount of past experience to provide an AI agent depends heavily on the model's tier. Using the ALTK-Evolve framework, researchers tested eight models and found that strong models benefit from full guideline sets, weaker models prefer targeted retrieval, and saturated models show no gains.
Why it matters
Understanding how to calibrate memory doses allows developers to maximize agent accuracy and control token costs without requiring weight updates or human annotations.