Agent memory
Agents that remember store embeddings. TurboQuant makes them 6–8x smaller.
Vector compression with under 2% recall loss and no training.
Kevin Keller · kevinkeller.org
Why it matters
Vector storage gets expensive fast
- Every memory, document chunk and saved tool result becomes a float32 vector.
- 10,000 memories at 1,536 dimensions: 60 MB of vectors.
- 100K memories across tenants: 600 MB, before metadata and indexes.
The result
3–4 bits
per element instead of 32, with under 2% recall loss.
Google's TurboQuant, published at ICLR 2026. No training required.
How it works
Two stages, no codebook
- PolarQuant: rotate each vector with a random orthogonal matrix, so every coordinate follows a known distribution, then quantise each coordinate.
- QJL: a 1-bit residual correction that preserves inner product accuracy for similarity search.
The rotation comes from a seed shared by writer and reader.
What changes for agents
Four reasons it fits agent memory
- Remember more: the same RAM holds 6 to 8x more memories
- No cold start: works on the first vector like the millionth
- Cheaper persistence: saved memory files 8x smaller than float32
- Quality kept: queries stay at full precision
No cold start
Data-oblivious compression suits incremental memory
You don't retrain a codebook every time a new memory arrives.
Product Quantization needs a representative sample first. TurboQuant fits the incremental add() pattern of agent memory.
The idea
Compress the stored vectors, keep the query at full precision.
Asymmetric search finds almost always the same memories at 4 bits as at 32.
In practice
One parameter in zettelkasten-memory
- A
TurboQuantCompressor(bits=4)sits between embedding and storage. - Usage of add and search stays identical.
- The MCP server enables it with a
--compressflag. - Six embedding providers, including Ollama and Snowflake Cortex.
Already spreading
TurboQuant is showing up in agent memory projects
- Prism MCP advertises it as a headline feature.
- LangChain has a community integration,
langchain-turboquant. - The
turboquantPyPI package has a first stable release.
The takeaway
If your agent stores embeddings, compression is the easiest win available.
Kevin Keller · kevinkeller.org