Vector-store memory as the agent's long-term memory
“Remembered everything, retrieved nothing.”
Cause
The memory bake-off answered the question with a different design: structured episodic store with summarised recall beat raw vector recall on precision and latency above 50k tokens of history.
What was tried, what was hit, what it means
The original design was the obvious one: embed every turn of every session, store it in pgvector, retrieve top-k at each step. It was the default in three delivery patterns from February to June and it worked for agents with a handful of sessions. The field opened on the conviction that this shape would not survive contact with a long-running workload, and the bench was proposed to find out where it broke.
It broke at roughly 50k tokens of accumulated history. Retrieval latency crossed generation latency, and recall precision on the decision-consistency split fell to 0.61 because the top-k chunks were the ones that looked like the question rather than the ones that held the decision. Adding a re-ranker bought eight points and doubled the latency. The four-way bench ran for eight weeks and the episodic-plus-summary design won on both axes; the write-up became r-memory-layer.
This is a superseded entry, not a refuted one. Vector recall is still the right shape for short-lived agents and for semantic lookup inside the semantic store. What died is the claim that it is the long-term memory. Anyone reaching for it as a default for a multi-session agent should read the recommendation first.
Lessons
- 01Measure at the history length the workload actually reaches, not the one the demo uses.
- 02Top-k similarity finds things that look like the question; decisions rarely look like the question that revisits them.
- 03A design that is the default in three patterns needs a bench before it becomes doctrine, not after.
Record
Field
Agentic Memory System Lineage intact. The reasoning that killed this stays attached to everything it touched.Experiment
Memory layer bake-off on a 40-session support corpus Kill · Kill if episodic+summary trails vector recall on precision by day 14 on the 50k+ split, or if its p50 retrieval latency exceeds 1.5× vector recall at any history length.Superseded by
Use a structured episodic store with summarised recall, not a raw vector memoryTry again?
Proposals that touch this entry get its returned notes attached automatically.