← WTF is Agent?

Memory: remember, cite, forget

Part 2 of 13 · Jun 3, 2026 · 6 min read

In short Store the raw text, keep facts small, derive everything else, and make every belief point back to its source so it can be traced and forgotten. The design is built; two journeys pass, but a memory benchmark is still missing.Full verdict ↓

An agent that forgets is a chatbot, so memory was the first thing I built. It was also the thing I rebuilt most. Where I ended up is one rule: every belief the agent holds must point back to the words it came from.

3layers: raw text, facts, conclusions
1provenance table joining every kind of record
3clocks per fact: happened, told, stored
2memory journeys in the suite; the 35B passed all six samples

The design

Three layers, one trail Every record points to what it was formed from. Forgetting a payload walks that trail upward.
Derived: what the system concluded Base: what it was told, as small facts Raw: the text exactly as sent Observation Observation Entity profile factfact fact payload A payload B forget(payload A)removes the payload,the facts only it supported,and every conclusionformed from them.why do you believe this?walk the edges down tothe words you said.

Four operations sit on top of that: retain (store a payload, extract facts), recall (find what is relevant), reflect (answer a question over what is stored) and mental models (a written answer about the user that is refreshed and cites its sources).

The journey

  1. A mining pipeline. Extract entities and rules from conversations, cluster them into a knowledge graph, consolidate in the background.
  2. Rebuilt on a published design. Retain, recall, reflect and mental models, following the Hindsight paper, running as durable workflows.
  3. The first pipeline is removed. Mining, clustering and consolidation were taken out in full once the rebuild replaced them.
  4. Raw text first. Store every payload as sent, give each fact a source, add a separate layer for conclusions.
  5. One provenance trail. Profiles, relations and observations all cite their sources in one table, and forget walks it.
  6. Realigned with the paper. Observations are recomputed per entity from all of its facts.

Decisions that held

Decision Why The lesson behind it
Store the raw text as sent Without it a belief cannot be traced to anything Facts stored before this have no source, and there is no backfill because the text was discarded
Keep the base small, derive the rest A summary is a conclusion, not a fact A one-way rule: derived reads base, never the reverse, enforced by a test
One provenance table for every kind of record Profiles, relations and observations all need the same trail Two evidence tables turned out to be the same relation
The model proposes citations, code checks them A model can cite a fact it never saw Code accepts only ids that appeared in the evidence it was given
Three clocks: happened, told, stored A fact told about 2019 and stored today is new to the system I used the told time as a "new" marker in four places, so a fact told about 2019 would never have counted as new

What already exists

I read the main systems before rebuilding. What each keeps of where a memory came from:

System What it stores Source of a memory History when it changes Deletion
MemGPT / Letta Evicted messages, a rolling summary, archival store Only that messages are still stored Summary overwritten Not described
Mem0 Facts the model adds, updates or deletes No raw-text provenance described Graph variant marks relations invalid Model chooses DELETE
A-MEM Linked notes that evolve The original content stays on the note Evolved note replaces the original Not described
Zep A temporal knowledge graph over raw episodes Edges trace back to episodes Edges are invalidated with timestamps Not described
Hindsight Four networks of facts and observations Observations list their source fact ids (in the released code) Facts can be invalidated Deleting a document removes the observations built from it (in the released code)
Mine Raw text, base facts, a derived layer One table, for six kinds of record Each conclusion supersedes with history One operation walks the trail

This is from my reading notes of the papers and, for Hindsight, its released code. I first claimed in my own draft that nobody had lineage or deletion. That was wrong: Hindsight's code has both for observations. What remains different is breadth, one trail covering profiles, relations and written answers, and history for conclusions.

Where it stands, and my verdict

Question Answer
Does it remember across sessions? Retain, recall, reflect and mental models run yes
Can a belief be traced to its words? Yes, for everything formed after the raw layer yes
Can I forget something and what came from it? One operation does it yes
Does it help the agent decide better? Two journeys pass (a preference recalled, a fact saved); no with/without comparison thin
Do conclusions earn their cost? No with/without comparison yet unproven

Verdict: the design is sound and still under-tested. The traceability and forgetting work as designed, and the journey suite now gives the memory layer a first signal: two scenarios, and the 35B passes all six samples (the thinking 8B misses one). That is not a memory benchmark. I still have no measurement that the derived layers improve what the agent does, which is the next thing I owe this component. Next: the toolbox, which is how that agent reaches anything outside itself.