Agent memory

AI agents forget everything. Give them a slip box.

A semantic memory layer that grows a knowledge graph with every conversation.

Kevin Keller · kevinkeller.org

Why it matters

Every session starts from zero

  • Context gathered in one session is gone when it ends
  • Long context windows are a workaround, not a memory system
  • They don't prioritise, connect ideas or decay gracefully
  • Multi-day tasks and multi-agent teams lose what they learned

Today's options

Three common approaches, three gaps

ApproachGap
Context window stuffingTruncates the oldest memories, which may matter most
Simple vector storesEvery memory is an island, no importance or decay
Key-value storesNo semantic search, you need the exact key

The inspiration

Luhmann's Zettelkasten: connections over hierarchy

  • Atomic notes: one idea per note
  • Links, not folders: the value is in the web of connections
  • Emergence: themes surface that nobody planned
Structure that emerges from the agent's work, not from a schema the developer imposes.

Storing a memory

Every new note weaves itself into the graph

  1. Generate an ID from a hash of content and timestamp
  2. Extract tags with TF-IDF
  3. Link to every note above a similarity threshold, both ways
  4. Store with metadata, tags, connections and an importance score

The key idea

Well-connected, recent, important memories surface first

Search ranks by similarity × importance × recency × connection boost. Isolated, stale notes fade away.

Self-maintaining

The graph walks, prunes and compresses itself

  • Graph traversal: follow links up to N hops to find context nobody searched for
  • Eviction: at 5,000 notes, the lowest survival score goes first
  • TurboQuant: stored vectors 6–8x smaller, under 2% recall loss
  • Persistence: embeddings saved, no re-embedding on restart

Plugs into what you use

Pluggable backends, three framework adapters

  • TF-IDF backend: zero config, offline, no API keys
  • Embedding backend: six providers, including Ollama and Snowflake Cortex
  • CrewAI and LangGraph: drop-in storage and BaseStore
  • Claude Code: an MCP server with six memory tools

What it solves

From forgetful sessions to shared knowledge

  • Continuity across multi-day tasks
  • Knowledge shared between agents in a crew
  • Token-budgeted context instead of full history
  • Decay without manual maintenance
  • Non-obvious connections surfaced by the graph

Takeaway

Agent memory should prioritise, connect and forget, like a good slip box.

Read the full article

Open source: github.com/KellerKev/zettelkasten-memory