Cortex Proxy

Route every AI coding agent through one policy enforcement point

Unmodified agents, any backend, prompts checked before they reach the model.

Kevin Keller · kevinkeller.org

Why it matters

Every agent speaks its own dialect

  • Claude Code speaks Anthropic; Continue.dev, ZeroClaw and Mistral Vibe speak OpenAI
  • Enterprises want inference centralised for governance, billing and data sovereignty
  • Prompts carry code, schemas, stack traces and file paths
  • Locking agents down kills productivity

The approach

A proxy in the middle that translates on the fly

  • Listens on localhost:8766 in Anthropic or OpenAI format
  • Routes to Snowflake Cortex, OpenAI, Anthropic or Ollama
  • Switch backends by changing one line of config
  • Agents don't know which backend is behind it

Zero client modification

Two environment variables and you're done

Claude Code thinks it is talking to Anthropic. The proxy translates to Cortex and back.

Tested with Claude Code, Continue.dev, Mistral Vibe, OpenCode and ZeroClaw.

The hard part

Translating tool calls and streaming, live

  • Tool calls: content blocks versus tool_calls, results reordered to match
  • Streaming: Anthropic's event protocol versus OpenAI chunks, chunk by chunk
  • Index bug: Cortex streams every tool call as index: 0; the proxy reassigns by ID
  • Hostnames: underscores break IDNA, so it resolves and sets the Host header

The policy layer

Block only the violations. Everything else passes at full speed.

The proxy is a Policy Enforcement Point: every prompt is checked before it reaches the LLM, without touching the developer's tools.

Two-tier evaluation

Fast patterns first, a judge model second

  1. Pattern matching: milliseconds, catches phrases like "ignore all previous instructions"
  2. Judge model: 1–3 seconds, a separate Cortex model checks the prompt against every active rule
  3. Action: block with a clean 403 in the client's format, warn, or log

Built-in rules

Six rules out of the box, plus your own

RuleSeverity
prompt_injectionCritical
data_exfiltrationCritical
unauthorized_tool_useHigh
code_execution_abuseHigh
pii_exposureMedium
scope_violationMedium

Built small

A single Rust file you can audit

  • About 1,300 lines, compiled to a binary under 4 MB on macOS ARM64
  • Tokio and Axum: no GC pauses while streaming
  • Cortex Agent API with automatic PAT-to-session exchange
  • Fuzzy model name mapping, so agent configs rarely change

Before going further

An educational proof of technology

  • The PAT lives in config.toml: protect it like a password
  • Environment variable PATs, External OAuth and Workforce Identity Federation are planned
  • Plain HTTP and open CORS by default: add TLS beyond localhost

Takeaway

Centralise inference, keep developers' tools unchanged, enforce policy in the request path.

Read the full article

Educational proof of technology, not meant for production as-is