AI agents that remember things store embeddings. Every memory, every document chunk, every tool result that gets saved for later retrieval becomes a float32 vector — typically 768 to 3072 dimensions. At scale, this gets expensive fast.
A 10,000-memory agent using text-embedding-3-small (1536 dimensions) stores 60 MB of vectors. A production multi-agent system with 100K memories across tenants? 600 MB just in vectors, before you count metadata, connections, or indexes.
Google’s TurboQuant, published at ICLR 2026, compresses those vectors to 3-4 bits per element — a 6-8x reduction — with under 2% recall loss. And it requires zero training.
How It Works#
TurboQuant is a two-stage compression algorithm:
Stage 1 — PolarQuant. Multiply every vector by a random orthogonal matrix. This rotation is the key insight: it makes each coordinate follow a predictable Beta distribution regardless of the input data. Once the distribution is known, you apply an optimal scalar quantizer per coordinate — no codebook, no clustering, no training data needed. The rotation matrix is generated once from a seed and shared between writer and reader.
Stage 2 — QJL (Quantized Johnson-Lindenstrauss). A 1-bit residual correction that captures the error introduced by quantization. This preserves inner product accuracy for similarity search.
The result: vectors shrink from 32 bits per element to 3-4 bits, and cosine similarity search still works because queries stay at full precision (asymmetric search).
Why This Matters for Agent Memory#
Most agent memory implementations — RAG stores, vector databases, zettelkasten-style knowledge graphs — treat vector storage as a fixed cost. You embed, you store, you search. TurboQuant changes the math:
1. Agents can remember more. The same RAM budget holds 6-8x more memories. An agent capped at 5,000 memories can now hold 30,000-40,000 without changing hardware. For long-running agents that accumulate context over days or weeks, this is the difference between “memory full, evicting” and “plenty of room.”
2. No training means no cold start. Product Quantization (PQ) and other codebook-based approaches need to see a representative sample of vectors before they can compress. TurboQuant is data-oblivious — it works on the first vector the same as the millionth. This fits the incremental add() pattern of agent memory perfectly. You don’t retrain a codebook every time a new memory arrives.
3. Persistence gets cheaper. Agent memory that survives across sessions needs to be serialized. JSON with float32 vectors is bloated. Binary with float16 is better. TurboQuant at 4 bits per element means the serialized memory file is 8x smaller than float32 — faster saves, faster loads, less disk.
4. Asymmetric search preserves quality. The query vector stays at full precision. Only the stored vectors are compressed. This means retrieval accuracy barely degrades — the memories you find at 4 bits are almost always the same ones you’d find at 32 bits.
What It Looks Like in Practice#
For a pluggable memory system like zettelkasten-memory, TurboQuant fits as a transparent compression layer on the embedding backend:
from zettelkasten_memory import ZettelMemory
from zettelkasten_memory.backends import EmbeddingBackend
from zettelkasten_memory.compression import TurboQuantCompressor
mem = ZettelMemory(
backend=EmbeddingBackend(
embed_fn=my_embed_function,
compressor=TurboQuantCompressor(bits=4),
)
)
# Usage is identical — compression is transparent
mem.add("Customer requires PrivateLink with network policy blocking public access")
mem.add("Cortex agent role restricted to analytics schema only")
results = mem.search("network security configuration")
# Same accuracy, 6-8x less memoryThe compressor intercepts vectors after embedding and before storage. On search, queries stay full precision and use asymmetric dot product against the compressed index. No changes to the rest of the system.
The Competitive Landscape#
TurboQuant is already showing up in agent memory projects. Prism MCP (89 stars) advertises “TurboQuant 10x compression” as a headline feature. LangChain has a community integration via langchain-turboquant. The turboquant PyPI package shipped its first stable release this week.
For agent builders: if your memory system stores embeddings and you haven’t looked at compression yet, TurboQuant is the easiest win available. No codebook training, no GPU, no accuracy cliff — just smaller vectors.
