Skip to main content
  1. Posts/

A Layered Architecture for Python and Conda Supply Chain Security

On March 24, 2026, LiteLLM — a package with 95 million monthly downloads — was backdoored on PyPI. The TeamPCP group compromised the maintainer’s credentials through a chain of CI/CD attacks, published versions 1.82.7 and 1.82.8 with a multi-stage credential stealer, and had three hours before detection. Three hours where every pip install litellm delivered malware that harvested SSH keys, cloud tokens, Kubernetes secrets, and crypto wallets.

This wasn’t an isolated incident. TeamPCP also hit Telnyx the same week. In 2025, 500+ typosquatting packages flooded PyPI in a single campaign. Domain expiration attacks compromised 1,800+ PyPI accounts. A cross-ecosystem worm exfiltrated 3,325 secrets from 817 GitHub repositories, including PyPI publishing tokens.

The instinctive response is: we need package signing. But here’s the uncomfortable truth — signing would not have prevented the LiteLLM attack. The attacker compromised the maintainer’s CI/CD credentials and published through the legitimate pipeline. If signing had been enforced, the backdoored package would have been validly signed — because it came from the real account, through the real build system. The signature would have proven provenance. The provenance was the problem.

This means signing is necessary but not sufficient. Curation catches known CVEs but not zero-days. Scanning detects patterns but not novel payloads. No single layer is enough. This article explores what a complete, layered architecture looks like — from signing and curation to runtime containment — combining defenses from across the Python, conda, and container ecosystems.


How Packages Get Compromised
#

Four attack vectors dominate, in order of impact:

1. Account takeover — the dominant vector. Attackers compromise maintainer credentials through phishing (the July 2025 noreply@pypj.org campaign), expired email domain registration (1,800 accounts), or CI/CD credential theft (TeamPCP’s path through Trivy → GitHub Actions → PyPI tokens). Once in, they publish a malicious version of a legitimate, trusted package.

2. Typosquatting and slopsquatting. Register requ3sts or colorinal — close enough to a popular package that someone installs it by mistake. LLMs have made this worse: attackers now use model-hallucinated package names as typosquat targets, a technique called slopsquatting.

3. Dependency confusion. Publish a public package with the same name as a private internal package. If the package manager checks the public registry first, it installs the attacker’s version. 5,000+ copycats flooded PyPI and npm simultaneously in 2025.

4. Malicious updates. Once credentials are compromised (via any of the above), the attacker publishes backdoored versions of a package that thousands of projects already depend on. The trust is inherited.


What Signing Solves — And What It Doesn’t
#

Package signing proves provenance: this artifact was produced by this identity, at this time, through this build pipeline. It’s logged to a transparency log so the chain of custody is auditable.

Signing protects against:

  • Mirror tampering — a compromised CDN or mirror can’t substitute a different artifact
  • Man-in-the-middle — intercepted downloads are detected
  • Build tampering — if Trusted Publishing is used, the package must come from the registered CI pipeline

Signing does not protect against:

  • Compromised maintainer credentials — a signed package is still signed even if the maintainer’s account was hijacked. The signature proves it came from that account, which is exactly the problem
  • Malicious code in legitimate updates — if the maintainer publishes a backdoored version through the normal pipeline, the signature is valid
  • Typosquatting — the attacker signs their own malicious package with their own identity

This nuance matters. Signing is necessary but not sufficient. It needs to be combined with other defenses.


The State of Signing in Conda and Pixi
#

Conda’s Native Signing — conda-content-trust (TUF)
#

The conda client has supported package signature verification since version 4.10.1 through conda-content-trust, based on The Update Framework (TUF). This is the foundational signing mechanism for the conda ecosystem:

  • Anaconda builds and signs packages on a secure build network, producing signatures that ship alongside packages on Anaconda’s premium repository
  • Conda verifies that signatures come from trusted signers and match the package data, with trust established using root metadata that ships with conda
  • The TUF design protects against man-in-the-middle attacks, compromised mirrors, CDN tampering, and TLS misconfiguration
  • Anaconda has been working with the mamba and conda-forge communities to extend these verification capabilities to open-source repositories

This is production infrastructure — available to Anaconda Business and Enterprise customers today, with the conda-forge community working toward adoption.

Sigstore Attestations — CEP 27
#

Building on conda’s TUF foundation, the ecosystem is also adopting Sigstore for per-package attestations. CEP 27 (accepted, July 2025) standardizes the conda attestation format using in-toto Statement v1 with Sigstore bundles. The predicate ties a package filename and SHA-256 hash to a Sigstore-verified identity. This standard was developed by prefix.dev and Trail of Bits.

rattler-build supports Sigstore attestation generation:

rattler-build publish ./recipe.yaml \
  --to https://prefix.dev/my-channel \
  --generate-attestation

This creates a Sigstore bundle using the CI environment’s OIDC identity (GitHub Actions, GitLab, Buildkite, CircleCI, Codefresh), uploads both the package and attestation to prefix.dev. No key management — Sigstore’s keyless signing uses ephemeral keys bound to the CI identity.

pixi publish also supports --generate-attestation (merged in PR #5678).

Anaconda’s partnership with prefix.dev is bringing rattler-build into the conda client itself, which means Sigstore attestation support is heading into the broader conda ecosystem — complementing the existing TUF-based verification.

Verification
#

Both TUF and Sigstore verification are available today:

# conda TUF verification (conda 4.10.1+, Anaconda premium repository)
# Enabled via conda configuration — verifies signatures on install

# Sigstore verification via cosign
cosign verify-blob \
  --certificate-identity-regexp "https://github.com/my-org/.*" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  my-package-0.1.0-h123_0.conda

# Sigstore verification via sigstore-python
sigstore verify identity \
  --cert-identity "https://github.com/my-org/my-repo/.github/workflows/build.yml@refs/heads/main" \
  --cert-oidc-issuer "https://token.actions.githubusercontent.com" \
  my-package-0.1.0-h123_0.conda

Source attestation verification (experimental) in rattler-build can verify Sigstore attestations of upstream source tarballs during build:

source:
  url: https://files.pythonhosted.org/packages/.../flask-3.1.1.tar.gz
  sha256: "6489f1..."
  attestation:
    publishers:
      - github:pallets/flask

TUF and Sigstore are complementary — TUF protects repository metadata integrity (“is the index trustworthy?”), while Sigstore provides per-package provenance (“was this package built by who it claims?”). The conda ecosystem is investing in both.

How PyPI Compares
#

The PyPI community has made impressive progress here. PEP 740 (GA since November 2024) standardizes Sigstore-based attestations, with over 20,000 attestations already published. Trusted Publishing is well-adopted. pip doesn’t verify attestations by default yet — a plugin architecture is in development — but the foundation is strong and moving fast.


Where the Ecosystem Is Heading
#

The conda and pixi communities are actively building toward the same goal. It’s worth acknowledging how much progress has been made — CEP 27 is accepted, rattler-build has production Sigstore support, and prefix.dev is pioneering Trusted Publishing for conda. These are hard problems and the pace of innovation is impressive.

That said, there are opportunities ahead that the community is already aware of and working toward:

Install-time verification. pixi and rattler don’t yet enforce attestation verification at install time — this is a natural next step once CEP #142 (standardizing how channels serve attestation bundles) is finalized.

conda-forge attestation infrastructure. conda-forge already runs builds on GitHub Actions, which has the OIDC plumbing for Sigstore. The groundwork is there — it’s a matter of the community prioritizing attestation generation in feedstock builds.

Conda vulnerability tracking. OSV.dev doesn’t currently track conda as an ecosystem. Anaconda’s CVE curation fills this gap for commercial users, and broader coverage would benefit the entire community.

These aren’t criticisms — they’re the natural next milestones for an ecosystem that’s already moving in the right direction.


Defense in Depth — What You Can Do Today
#

The tools and practices available right now already provide strong protection:

Lockfiles are your best friend. pixi.lock pins every transitive dependency to an exact version and hash. If a malicious version is published after you lock, you don’t get it. This is the single most effective defense against supply chain attacks today.

Audit before you update. Don’t blindly pixi update. Review what changed. Check the diff. Look at the changelog.

Use Trusted Publishing for your own packages. If you publish to prefix.dev, use --generate-attestation. Your consumers can’t enforce verification yet, but the attestation is logged to Sigstore’s transparency log — it’s auditable retroactively.

Scan dependencies for CVEs before building. Trivy can scan filesystem paths. OSV.dev has an API. Neither tracks conda natively, but most conda packages have PyPI equivalents you can query.

Run SAST and secret scanning. Semgrep (OWASP rules), Bandit (Python AST), and Gitleaks catch hardcoded credentials and common vulnerability patterns before they ship.

Generate SBOMs. Syft produces CycloneDX SBOMs from images and directories. Required for DORA, NIS2, and EU AI Act compliance. More importantly, it gives you a manifest to audit.


Anaconda’s Approach — Curated Security for Conda
#

Anaconda has been a cornerstone of the Python and conda ecosystem for over a decade. On the security front, they’ve invested heavily in a problem nobody else has tackled at scale for conda: curated CVE management as a managed service.

Anaconda’s commercial offering (Business and Enterprise plans) provides:

  • Manual CVE curation — a dedicated team reviews NIST/NVD-flagged packages, verifies what software the CVEs actually affect, and assigns curated status categories: Reported, Active, Cleared, Mitigated, or Disputed. This human review layer eliminates the false positives that make raw CVE databases noisy.
  • Conda-forge CVE association — visibility into which CVEs affect conda-forge packages mirrored to your secure repository. This fills the gap that OSV.dev leaves (zero native conda coverage).
  • TUF-based package signing — packages on Anaconda’s premium repository are signed using conda-content-trust (The Update Framework). Conda 4.10.1+ can verify these signatures, establishing a chain of trust from root metadata shipped with conda.
  • Sigstore integration path — Anaconda has partnered with prefix.dev to integrate rattler-build into conda, bringing Sigstore attestation support into the broader conda ecosystem. They’re also working with the conda-forge community to extend signing capabilities to open-source repositories.
  • Security policy enforcement — license filtering and CVE-based blocking built into the repository pipeline.

The architecture is essentially the quarantine pattern implemented as a service:

  conda-forge / PyPI ──► Anaconda Mirror ──► CVE Curation ──► Secure Repository
                              │                   │                  │
                          mirror sync         human review       policy gate
                                              NVD/NIST check     license filter
                                              status assignment  signed packages
                                                                 CVE metadata

Enterprise Deployment Models
#

For organizations that need tighter control, Anaconda Server provides on-premise and air-gapped deployment options:

  • Private repository mirrors — create a local, curated mirror of conda-forge, PyPI, or CRAN. Mirrors can be complete, partial, or filtered by CVE score, license type, and package version. Users install from the mirror, never from the public internet.
  • Air-gapped deployments — for regulated environments (defense, finance, healthcare), Anaconda supports fully disconnected installations with offline mirror archives. Packages are vetted and transferred through a controlled ingress process.
  • SBOM generation — built-in Software Bill of Materials in SPDX format, providing component-level visibility required for DORA, NIS2, and EU AI Act compliance.
  • Audit trails — timestamped, searchable records of every package addition and removal, capturing mirror syncs, manual uploads, and policy-driven removals.

The quarantine pattern is built into the architecture: public packages flow through Anaconda’s CVE curation pipeline before reaching the enterprise mirror. Teams consume pre-vetted packages without building their own scanning infrastructure.

Private Channels With prefix.dev and Pixi
#

For teams that prefer a lighter-weight approach or need to host internal packages alongside public ones, prefix.dev offers hosted private channels with Trusted Publishing and Sigstore attestation support. Pixi integrates natively:

# pixi.toml — mixing public and private channels
[workspace]
channels = [
  "conda-forge",
  "https://repo.prefix.dev/my-org-internal",
]

Internal libraries are versioned, signed, and distributed through the same channel infrastructure as public packages. pixi auth handles authentication to private channels, and pixi publish --generate-attestation signs packages with Sigstore before uploading. This gives smaller teams and open-source projects the same provenance guarantees that Anaconda Enterprise provides at the platform level.


Chainguard — Rebuilt From Source for Containers
#

Chainguard has done something genuinely innovative for the container and Python ecosystem: rebuilding package distributions entirely from source.

Chainguard Libraries for Python rebuilds 16,000+ PyPI packages from their public, verifiable source code in a SLSA L3-compliant build system. The key insight: most PyPI supply chain attacks inject malicious code at the build or distribution stage — the published wheel contains code that isn’t in the source repository. By rebuilding from source, Chainguard eliminates that entire attack vector. In testing against 3,025 known malicious Python packages, their rebuilt-from-source approach prevented 98% of compromised packages from reaching users.

Every rebuilt library ships with Sigstore-signed provenance and a signed SBOM. Chainguard’s distroless container images (no shell, no package manager, minimal attack surface) further reduce the blast radius.

Where Each Approach Fits
#

ConcernAnacondaChainguard
FocusConda/Python package curationContainer images + PyPI rebuilds from source
Trust modelHuman-curated CVE review + TUF signingSLSA L3 build provenance + source verification
Malware preventionCVE scanning + policy gatesRebuilt from source (98% malware prevention)
SigningTUF (conda-content-trust)Sigstore + signed SBOMs
CoverageConda + conda-forge + PyPIPyPI (16K+ packages) + container base images
Best forData science teams using conda, enterprise Python environmentsProduction container deployments, regulated workloads (DORA/NIS2)
OpportunityExpanding to proactive malware detectionExtending coverage to conda-native packages

These are complementary approaches that strengthen different layers. Anaconda secures the package consumption pipeline (S2C2F) — invaluable for data science and enterprise Python teams. Chainguard secures the package distribution and runtime layer (SLSA) — essential for production container deployments. Both are doing important work for the ecosystem, and an ideal architecture can leverage both: consume curated packages from Anaconda, deploy in Chainguard containers.


Runtime Containment — When Prevention Fails
#

Signing, curation, and scanning reduce the probability of running compromised code. But no prevention layer is perfect — the LiteLLM backdoor was live for three hours despite PyPI’s security infrastructure. The final defense layer is runtime containment: ensuring that even if malicious code executes, it can’t exfiltrate data, move laterally, or persist.

This is where the compute environment itself becomes a security boundary. The architecture pattern:

  ┌─────────────────────────────────────────────────────────────┐
  │                   Runtime Containment Layers                 │
  │                                                             │
  │  ┌──────────────────────────────────────────────────────┐   │
  │  │  Layer 1: Network Isolation                          │   │
  │  │  Default deny egress. Only approved endpoints via    │   │
  │  │  allowlist. All traffic through egress proxy.        │   │
  │  │  No internet unless explicitly granted.              │   │
  │  └──────────────────────┬───────────────────────────────┘   │
  │                         ▼                                   │
  │  ┌──────────────────────────────────────────────────────┐   │
  │  │  Layer 2: Filesystem Isolation                       │   │
  │  │  chroot / overlay filesystem. Minimal shared libs.   │   │
  │  │  No access to host filesystem, other tenants, or     │   │
  │  │  data outside the execution scope.                   │   │
  │  └──────────────────────┬───────────────────────────────┘   │
  │                         ▼                                   │
  │  ┌──────────────────────────────────────────────────────┐   │
  │  │  Layer 3: System Call Restriction                    │   │
  │  │  seccomp-bpf filters. Only syscalls needed for       │   │
  │  │  execution are allowed. ptrace monitoring for        │   │
  │  │  anomaly detection.                                  │   │
  │  └──────────────────────┬───────────────────────────────┘   │
  │                         ▼                                   │
  │  ┌──────────────────────────────────────────────────────┐   │
  │  │  Layer 4: Identity & Authorization                   │   │
  │  │  Code executes as a specific role/identity.          │   │
  │  │  RBAC determines what data it can access.            │   │
  │  │  Masking and row access policies apply.              │   │
  │  └──────────────────────────────────────────────────────┘   │
  └─────────────────────────────────────────────────────────────┘

A Concrete Example: Snowflake’s Compute Isolation
#

Snowflake has invested deeply in secure compute isolation — their approach to running user-supplied code (Snowpark UDFs and Snowpark Container Services) is one of the most thorough implementations of this pattern in production today:

Snowpark UDFs (Python, Java, Scala) execute inside a multi-layered sandbox:

  • Linux namespaces isolate network, PID, mount, IPC, UTS, and cgroup domains
  • chroot restricts the filesystem to a minimal set of shared libraries — nothing outside the sandbox is visible
  • seccomp-bpf restricts the kernel system call surface to only what UDFs need
  • ptrace monitors system calls in real-time for threat detection
  • Egress proxy — all traffic between compute and the internet goes through a proxy that enforces access control and reports violations
  • The language runtime itself is built by Snowflake, complicating architecture-level attacks

Snowpark Container Services (SPCS) extends this to full containers:

  • Containers have no internet access by default — egress must be explicitly granted via External Access Integrations (EAIs)
  • EAIs are created by account administrators, not by the code running inside the container
  • Each service supports up to 10 EAIs with up to 100 hostnames each — a strict allowlist, not a blocklist
  • Network rules control which external destinations are reachable
  • Private connectivity (PrivateLink) can route egress through private endpoints instead of the public internet
  • All data access from containers is governed by Snowflake’s RBAC, masking, and row access policies via Horizon Note: SPCS is OCI-compatible, which means standard container scanning tools work against its registry. Vladimir Ozerov explored this in a practical walkthrough — running Grype as an SPCS job to scan images stored in the SPCS registry. This isn’t a platform feature — it’s a pattern customers can adopt. Continuous CVE scanning, image rebuilds, and vulnerability management still need to happen in your CI/CD pipeline outside of Snowflake.

Why this matters for supply chain security: If a compromised dependency is installed inside a Snowpark UDF or SPCS container, the malware faces:

  1. No network access (unless explicitly allowed) — credential exfiltration fails
  2. No filesystem access outside the sandbox — lateral movement fails
  3. Restricted system calls — privilege escalation fails
  4. RBAC-governed data access — the code only sees data its role permits, with masking applied

This doesn’t prevent the malicious code from running, but it contains the blast radius to effectively zero. The LiteLLM backdoor’s payload — harvesting SSH keys, cloud tokens, Kubernetes secrets — would fail at every step inside this architecture.

Generalizing the Pattern
#

This isn’t Snowflake-specific. The same containment architecture applies to any secure compute platform:

LayerSnowflake ImplementationGeneral Equivalent
Network isolationEgress proxy + EAI allowlistKubernetes NetworkPolicy + egress gateway
Filesystem isolationchroot + namespacesgVisor / Kata Containers / Firecracker microVMs
Syscall restrictionseccomp-bpf + ptraceseccomp profiles + Falco runtime detection
Identity-bound executionSnowflake RBAC + HorizonOPA/Gatekeeper admission + IRSA/workload identity

The principle: assume the code is compromised and design the environment so it doesn’t matter. Default-deny networking, minimal filesystem, restricted syscalls, and identity-scoped data access. Prevention (signing, scanning, curation) reduces probability. Containment limits consequence.


Putting It All Together — The Reference Architecture
#

The layers above — signing, curation, scanning, and runtime containment — map to three industry frameworks. Here’s the combined architecture applicable to conda/pixi projects:

  ┌─────────────────────────────────────────────────────────────────┐
  │                    PRODUCE (SLSA L3)                             │
  │                                                                 │
  │  Source ──► Build (isolated CI) ──► Provenance ──► Sign         │
  │    │            │                      │              │         │
  │  git commit  rattler-build          in-toto        Sigstore     │
  │  signed      hermetic build         statement      attestation  │
  │  tag/branch  reproducible           SHA-256 hash   Rekor log    │
  └──────────────────────────────────────┬──────────────────────────┘
                                         │ .conda + .sigstore.json
  ┌─────────────────────────────────────────────────────────────────┐
  │                    CHANNEL / REGISTRY                            │
  │                                                                 │
  │  prefix.dev / conda-forge / Anaconda / private channel          │
  │  Serves: package + attestation + repodata + CVE metadata        │
  └──────────────────────────────────────┬──────────────────────────┘
  ┌─────────────────────────────────────────────────────────────────┐
  │               CONSUME (S2C2F Quarantine Pattern)                │
  │                                                                 │
  │  ┌───────────┐    ┌──────────────┐    ┌───────────────────┐     │
  │  │ INGEST    │───►│ QUARANTINE   │───►│ PROMOTE           │     │
  │  │           │    │              │    │                   │     │
  │  │ pixi add  │    │ Typosquat    │    │ Approved deps    │     │
  │  │ or update │    │ CVE scan     │    │ pinned in        │     │
  │  │           │    │ Attestation  │    │ pixi.lock        │     │
  │  │           │    │ SAST scan    │    │                   │     │
  │  │           │    │ Secret scan  │    │ SBOM generated   │     │
  │  └───────────┘    └──────────────┘    └───────────────────┘     │
  │                         │ FAIL                                  │
  │                         ▼                                       │
  │                   ┌──────────┐                                  │
  │                   │ REJECT   │                                  │
  │                   │ + alert  │                                  │
  │                   └──────────┘                                  │
  └──────────────────────────────────────┬──────────────────────────┘
  ┌─────────────────────────────────────────────────────────────────┐
  │               RUNTIME CONTAINMENT                               │
  │                                                                 │
  │  Network isolation ──► Filesystem isolation ──► Syscall filter  │
  │  (default deny          (chroot / overlay)      (seccomp-bpf)   │
  │   egress proxy)                                                 │
  │                    Identity-bound execution (RBAC)              │
  └─────────────────────────────────────────────────────────────────┘

  ┌─────────────────────────────────────────────────────────────────┐
  │                 SIGSTORE TRUST CHAIN                             │
  │                                                                 │
  │  CI Identity ──► Fulcio (CA) ──► Ephemeral Cert ──► Sign        │
  │  (GitHub OIDC)   issues cert     bound to OIDC     artifact     │
  │                                  identity                       │
  │                                       │                         │
  │                                       ▼                         │
  │                                  Rekor (Log)                    │
  │                                  transparency                   │
  │                                  immutable record               │
  └─────────────────────────────────────────────────────────────────┘

Framework Mapping
#

FrameworkFocusWhat It CoversPixi/Conda Mapping
SLSA L1-L3Producing artifactsSource → Build → Provenance → Signingrattler-build + Sigstore attestation
S2C2F L1-L4Consuming dependenciesIngest → Inventory → Scan → Enforce → Auditpixi.lock + CVE scan + quarantine
SigstoreCryptographic trustKeyless signing, transparency log, verificationCEP 27 attestations on prefix.dev
NIST SSDFSecure developmentPractices across the SDLCSAST, secret scanning, SBOM

SLSA levels in context: SLSA L1 requires documented provenance. L2 requires a hosted build service. L3 requires the build to be hardened (isolated, hermetic). rattler-build on GitHub Actions with --generate-attestation achieves SLSA L3 for conda packages published to prefix.dev.

S2C2F practices mapped to the pipeline: The eight S2C2F practices — Ingest, Inventory, Update, Enforce, Audit, Scan, Rebuild, Fix — map directly to the quarantine pattern above. pixi add is Ingest. pixi.lock is Inventory. CVE scanning is Scan. Lockfile pinning is Enforce. SBOM generation is Audit.

Prevention reduces probability. Containment limits consequence. A complete architecture needs both.


What’s Next — An Agentic Approach
#

The frameworks, signing tools, and curation services described above are the building blocks. But assembling them into a single, automated pipeline — where typosquat detection, threat intelligence, CVE auditing, SAST scanning, Sigstore signing, and SBOM generation run as a coordinated workflow — remains a manual exercise today.

I’m exploring whether an AI agent can close that gap: an agentic builder that orchestrates the full SLSA + S2C2F pipeline as a build-time gate. The chain of thought looks something like this:

  1. Before resolving any dependency, verify every package name isn’t a typosquat
  2. Query live threat intelligence (CISA KEV, GitHub Security Advisories, OSV.dev) for active exploits
  3. Resolve the full transitive dependency tree, then audit each dependency for known CVEs and pin safe versions
  4. Run SAST (Semgrep with OWASP rules, Bandit) and secret scanning (Gitleaks) over the source
  5. Build with rattler-build, sign with Sigstore, scan the built artifact with Trivy
  6. Generate a CycloneDX SBOM and produce a compliance-ready report

The individual tools are mature. What’s missing is the orchestration layer that runs them in the right order, feeds findings between steps, and gates the build on security outcomes — while keeping humans in the loop for code-level decisions. More on this soon.


Related#

Kevin Keller
Author
Kevin Keller
Personal blog about AI, Observability & Data Sovereignty. Snowflake-related articles explore the art of the possible and are not official Snowflake solutions or endorsed by Snowflake unless explicitly stated. Opinions are my own. Content is meant as educational inspiration, not production guidance.
Share this article

Related