Hybrid cloud, part 1
Query on-premise data from Snowflake, with every ingress port closed
An SSH tunnel from Snowflake Container Services to your DMZ.
Kevin Keller ยท kevinkeller.org
Why it matters
Use the cloud today, even for what can't move
- Reach resources that aren't migrated yet, during your move to the cloud
- Some APIs, AI models and data will never leave for compliance reasons
- Access from Snowflake can be cut at any time
The architecture
The DMZ is the bridge
- The private network makes only an outbound connection to the DMZ
- The DMZ opens one SSH port
- An SPCS container establishes the SSH tunnel at start-up
Ingress ports on the internal firewall stay closed.
Why SSH
VPNs need a privilege containers don't get
OpenVPN and WireGuard require NET_ADMIN for a container, which Snowflake Container Services doesn't grant today. SSH works without it.
The demo
Two engines, one Iceberg lake behind a firewall
- DuckDB and Spark in one container
- Iceberg REST Catalog and MinIO on private IPs
- Jupyter and VS Code in the container
- Streamlit app and a REST API callable as a UDF
The control
Cut the tunnel, lose the access
Spark and DuckDB run in the container, not a warehouse, so no result set cache holds on-premise data in Snowflake.
Three ways to query
Each with its own trade-off
| Path | Note |
|---|---|
| Jupyter notebook | Join on-premise and Snowflake dataframes freely |
| Snowflake UDF | Results are cached; scalar only, up to 16MB |
| Streamlit app | Edit live in the container's VS Code |
Setup
From a Linux VM to a running tunnel
- Run the Iceberg lake with Docker Compose and load sample data
- Close every port except 22
- Push the image to a Snowflake image repository
- Store the SSH user, DMZ IP and key in a Snowflake secret
- Create the service and open its endpoints
Before production
What the proof of concept doesn't do yet
- Replace SSH private keys with short-lived SSH certificates
- Broker credentials, for example Device Code OAuth for notebooks
- Fetch secrets from a secret manager instead of code
- Consider a catalog with role and policy controls, such as Apache Polaris
Beyond Iceberg
Any resource behind the firewall
On-premise databases, APIs and AI models work the same way. Part 2 reverses the direction, exposing services in Snowflake Container Services to apps outside Snowflake.
The takeaway
Reach data that will never move, and keep the switch in your hands.
A proof of concept, not a full production setup