lakesh

A DuckDB SQL shell for your data lake catalogs

Iceberg REST and DuckLake in one shell, with an MCP server for LLM agents.

Kevin Keller ยท kevinkeller.org

Why it matters

Catalog work is full of small frictions

  • Re-pasting ATTACH statements and OAuth tokens every session
  • Getting S3 credential plumbing right for cloud and MinIO
  • Switching between dev, staging and prod catalogs
  • Individually thin conveniences; together, meaningfully faster daily work

The division of labour

DuckDB does the heavy lifting, lakesh adds the ergonomics

  • DuckDB: Parquet reads, predicate pushdown, joins across namespaces
  • lakesh: named profiles, automatic OAuth2 token fetch and reuse
  • lakesh: pre-configured S3 credentials, psql-style meta-commands
  • lakesh: autocomplete scoped to the attached catalog

Multi-catalog workflows

Switching catalogs is one flag

lakesh -p prod

Works against any Iceberg REST catalog: Polaris, Nessie, Lakekeeper, managed REST, or any DuckLake catalog. Secrets can come from environment variables, so production profiles keep credentials out of config files.

Two profile types

Read through REST, write through DuckLake

Iceberg RESTDuckLake direct
Connects viaIceberg REST Catalog APIPostgres + S3
AuthOAuth2, handled per sessionPostgres DSN
WritesNot over RESTINSERT, UPDATE, DELETE

LLM agents over your lake

Give your AI assistant catalog access, read-only by default

lakesh mcp exposes profiles, namespaces, tables, schemas and queries to any MCP client. Writes need LAKESH_MCP_WRITE=1.

The MCP tools

Five tools, no custom tooling to build

  1. list_profiles: discover configured catalogs
  2. list_namespaces: list schemas
  3. list_tables: list tables, optionally scoped
  4. describe_table: columns, types, nullability
  5. query: rejects anything not starting with SELECT, SHOW, DESCRIBE, WITH or EXPLAIN

Scripted pipelines

Clean output for the next tool in the pipe

  • lakesh exec runs one-shot queries
  • Output as table, JSON or CSV
  • No REPL noise, so jq works as expected
  • lakesh doctor runs a connectivity smoke test

Takeaway

One shell for every catalog, and safe SQL access for your AI agents.

Read the full article

Open source: github.com/KellerKev/lakesh