Better together
Keep Splunk's detection. Keep every log for years.
A hybrid security data lake with Splunk, Snowflake and federated search.
Kevin Keller · kevinkeller.org
Why it matters
Not every log needs sub-second alerting
Log volume grows with the organisation. Web access, DNS, cloud audit and netflow are vital for investigations and compliance, but they don't all belong in the real-time detection tier.
The architecture
Everything to Snowflake. The critical subset to Splunk.
A routing layer such as Vector, Cribl, Kafka or Fluent Bit sends each event to one or both.
The routing question
One question decides where each log goes
Does this event need sub-minute detection and alerting?
- Splunk, 20-30%: EDR alerts, authentication failures, firewall denies, privilege escalation, DLP
- Snowflake, 100%: web, DNS, netflow, cloud audit, email, proxy, and the critical logs too
- Snowflake is always the complete record
The savings
70-80%
lower monthly spend at 100 GB/day
| Splunk-only | Hybrid | |
|---|---|---|
| Total monthly cost | ~$15,000-20,000 | ~$3,500-6,500 |
| Retention | 30-90 days | 30 days Splunk + years in Snowflake |
The compliance multiplier
Five years of logs becomes affordable
| Retention | Splunk-only | Snowflake storage |
|---|---|---|
| 1 year (36 TB) | Significantly higher licence | ~$830-1,440/mo |
| 2 years (73 TB) | Often cost-prohibitive | ~$1,680-2,920/mo |
| 5 years (182 TB) | Not practical for most | ~$4,200-7,300/mo |
Storage lifecycle policies can move older data to COOL and COLD tiers.
Federated search
Analysts query Snowflake without leaving Splunk
- Splunk DB Connect runs
dbxqueryagainst Snowflake - Aggregation pushdown: the SQL runs on Snowflake, only results return
- Hunt for SQL injection or XSS across months of logs
- Join today's EDR alerts with an IP's history in Snowflake
Detection in Snowflake
Dynamic Tables add a near-real-time layer
- Refresh on a target lag, typically 1 to 5 minutes
- Chain into multi-stage pipelines, such as brute force detection
- Declarative: define the SQL and the lag, Snowflake does the rest
- Separate
SECURITY_WHandSPLUNK_WHto track costs
Fit
Worth it at volume, not for everyone
| Good fit | Not a good fit |
|---|---|
| 50+ GB/day | Under 10 GB/day |
| Multi-year retention needs | Heavy reliance on ES notable models across all sources |
| Historical threat hunting | Real-time-only use cases |
| Snowflake already in use | No capacity for the routing layer |
Trade-offs
Know these before you split
- Federated queries take seconds, not milliseconds
- Two platforms to monitor, secure and run
- "Critical" needs ongoing tuning: start by sending more to Splunk
- Audit which ES searches depend on a source before rerouting it
The takeaway
Not a replacement story: each platform for what it does best.
Kevin Keller · kevinkeller.org