Cortex Proxy
Route every AI coding agent through one policy enforcement point
Unmodified agents, any backend, prompts checked before they reach the model.
Kevin Keller · kevinkeller.org
Why it matters
Every agent speaks its own dialect
- Claude Code speaks Anthropic; Continue.dev, ZeroClaw and Mistral Vibe speak OpenAI
- Enterprises want inference centralised for governance, billing and data sovereignty
- Prompts carry code, schemas, stack traces and file paths
- Locking agents down kills productivity
The approach
A proxy in the middle that translates on the fly
- Listens on
localhost:8766in Anthropic or OpenAI format - Routes to Snowflake Cortex, OpenAI, Anthropic or Ollama
- Switch backends by changing one line of config
- Agents don't know which backend is behind it
Zero client modification
Two environment variables and you're done
Claude Code thinks it is talking to Anthropic. The proxy translates to Cortex and back.
Tested with Claude Code, Continue.dev, Mistral Vibe, OpenCode and ZeroClaw.
The hard part
Translating tool calls and streaming, live
- Tool calls: content blocks versus
tool_calls, results reordered to match - Streaming: Anthropic's event protocol versus OpenAI chunks, chunk by chunk
- Index bug: Cortex streams every tool call as
index: 0; the proxy reassigns by ID - Hostnames: underscores break IDNA, so it resolves and sets the
Hostheader
The policy layer
Block only the violations. Everything else passes at full speed.
The proxy is a Policy Enforcement Point: every prompt is checked before it reaches the LLM, without touching the developer's tools.
Two-tier evaluation
Fast patterns first, a judge model second
- Pattern matching: milliseconds, catches phrases like "ignore all previous instructions"
- Judge model: 1–3 seconds, a separate Cortex model checks the prompt against every active rule
- Action:
blockwith a clean 403 in the client's format,warn, orlog
Built-in rules
Six rules out of the box, plus your own
| Rule | Severity |
|---|---|
prompt_injection | Critical |
data_exfiltration | Critical |
unauthorized_tool_use | High |
code_execution_abuse | High |
pii_exposure | Medium |
scope_violation | Medium |
Built small
A single Rust file you can audit
- About 1,300 lines, compiled to a binary under 4 MB on macOS ARM64
- Tokio and Axum: no GC pauses while streaming
- Cortex Agent API with automatic PAT-to-session exchange
- Fuzzy model name mapping, so agent configs rarely change
Before going further
An educational proof of technology
- The PAT lives in
config.toml: protect it like a password - Environment variable PATs, External OAuth and Workforce Identity Federation are planned
- Plain HTTP and open CORS by default: add TLS beyond localhost
Takeaway
Centralise inference, keep developers' tools unchanged, enforce policy in the request path.
Educational proof of technology, not meant for production as-is