Agent memory
AI agents forget everything. Give them a slip box.
A semantic memory layer that grows a knowledge graph with every conversation.
Kevin Keller · kevinkeller.org
Why it matters
Every session starts from zero
- Context gathered in one session is gone when it ends
- Long context windows are a workaround, not a memory system
- They don't prioritise, connect ideas or decay gracefully
- Multi-day tasks and multi-agent teams lose what they learned
Today's options
Three common approaches, three gaps
| Approach | Gap |
|---|---|
| Context window stuffing | Truncates the oldest memories, which may matter most |
| Simple vector stores | Every memory is an island, no importance or decay |
| Key-value stores | No semantic search, you need the exact key |
The inspiration
Luhmann's Zettelkasten: connections over hierarchy
- Atomic notes: one idea per note
- Links, not folders: the value is in the web of connections
- Emergence: themes surface that nobody planned
Structure that emerges from the agent's work, not from a schema the developer imposes.
Storing a memory
Every new note weaves itself into the graph
- Generate an ID from a hash of content and timestamp
- Extract tags with TF-IDF
- Link to every note above a similarity threshold, both ways
- Store with metadata, tags, connections and an importance score
The key idea
Well-connected, recent, important memories surface first
Search ranks by similarity × importance × recency × connection boost. Isolated, stale notes fade away.
Self-maintaining
The graph walks, prunes and compresses itself
- Graph traversal: follow links up to N hops to find context nobody searched for
- Eviction: at 5,000 notes, the lowest survival score goes first
- TurboQuant: stored vectors 6–8x smaller, under 2% recall loss
- Persistence: embeddings saved, no re-embedding on restart
Plugs into what you use
Pluggable backends, three framework adapters
- TF-IDF backend: zero config, offline, no API keys
- Embedding backend: six providers, including Ollama and Snowflake Cortex
- CrewAI and LangGraph: drop-in storage and
BaseStore - Claude Code: an MCP server with six memory tools
What it solves
From forgetful sessions to shared knowledge
- Continuity across multi-day tasks
- Knowledge shared between agents in a crew
- Token-budgeted context instead of full history
- Decay without manual maintenance
- Non-obvious connections surfaced by the graph
Takeaway
Agent memory should prioritise, connect and forget, like a good slip box.
Open source: github.com/KellerKev/zettelkasten-memory