cicd · level 7

Artifact Management

Registries, signing, retention, and promotion.

200 XP

Artifact Management

Once your CI builds something, where does it go, who can pull it, and how do consumers verify it's the real thing? Three intertwined questions: registries, signing, promotion + retention.

Registries

Every artifact type has its own registry:

  • Container images → OCI registries (GHCR, ECR, GCR, Quay, Harbor, Artifactory).
  • npm packages → npm.pkg.github.com, npmjs.com, Verdaccio (self-hosted).
  • Python packages → PyPI, devpi (self-hosted).
  • Maven/Gradle → Maven Central, Sonatype Nexus, Artifactory.
  • Helm charts → OCI registries (the same OCI spec works for charts since Helm 3).
  • Generic build outputs → S3, GCS, plus a thin metadata layer.

The OCI registry spec is the modern lingua franca — even npm and Python are slowly adopting it via tools like ORAS.

Build once, promote everywhere

The classic anti-pattern:

dev:      build → push :v1.2.3-dev
staging:  build → push :v1.2.3-staging      ← rebuilds, may differ
prod:     build → push :v1.2.3              ← rebuilds again

Three artifacts, all "the same version", potentially differing in tiny ways (dependency micro-versions, base image patches, build env). The bug that fires in prod doesn't repro in staging because they aren't the same bytes.

The fix: build ONCE, promote the same artifact through environments. The artifact's identity is its digest:

dev:      build → push @sha256:abc...      tag as :dev
staging:  same digest                       tag as :staging   (no rebuild)
prod:     same digest                       tag as :prod      (no rebuild)

The tag is a label. The digest is the truth. If staging and prod resolve to the same digest, they're bit-identical.

# Promote staging → prod by re-tagging the same digest
cosign verify ghcr.io/me/app@sha256:abc... ...   # check provenance
docker pull   ghcr.io/me/app@sha256:abc...
docker tag    ghcr.io/me/app@sha256:abc... ghcr.io/me/app:prod
docker push   ghcr.io/me/app:prod

Or with a registry that supports digest-based tag updates (most do):

crane copy ghcr.io/me/app:staging ghcr.io/me/app:prod  # no re-pull needed

Signing — cosign and sigstore

Pulling an image by tag means "trust the registry to give me the right bytes". A compromised registry — or a typosquatted image name — gives you compromised bytes. Signing closes the gap.

cosign (part of sigstore) signs an image's digest with a short-lived key obtained via OIDC from your CI provider. The signature is published to a public transparency log (Rekor); anyone can verify it.

# CI build job
- uses: sigstore/cosign-installer@v3
- run: |
    cosign sign --yes ghcr.io/me/app@${{ digest }}

Verification before deploy:

cosign verify ghcr.io/me/app@sha256:abc... \
  --certificate-identity-regexp "https://github.com/me/app/.*" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

What this proves: "this digest was signed by a GitHub Actions run from the me/app repository". An attacker who pushes a malicious image to the registry cannot forge the signature unless they also compromise GitHub's OIDC issuer or your repository.

The trust anchor is OIDC, not a long-lived key you have to protect. Big improvement over PGP-signed releases.

Retention policies

Registries are gleeful about charging you for storage. Without a retention policy, you accumulate every PR build, every dev artifact, every "let me try one more thing" image. A 50MB image × 50 dev builds/day × 365 days = ~900GB/year. Multiply by 20 services and the bill is real.

Retention rules to apply:

  • Keep the last N tags per branch (typically 5-10 for non-main branches).
  • Keep all images tagged with a release version (e.g. v*, prod-*).
  • Delete untagged images older than N days (typically 7 or 14).
  • Delete tagged images older than N months UNLESS in production (typically 6).

Most registries support these as policy expressions:

// AWS ECR lifecycle policy
{
  "rules": [
    {
      "rulePriority": 1,
      "selection": { "tagStatus": "untagged", "countType": "sinceImagePushed", "countUnit": "days", "countNumber": 7 },
      "action": { "type": "expire" }
    },
    {
      "rulePriority": 2,
      "selection": { "tagStatus": "tagged", "tagPrefixList": ["dev-"], "countType": "imageCountMoreThan", "countNumber": 10 },
      "action": { "type": "expire" }
    }
  ]
}

Set policies day one, audit them quarterly. Set them up and walk away — and your registry costs flatten out.

Immutability — the npm tax

Some registries let you republish a version. Bad idea: a malicious actor (or honest mistake) can change what vendor-pkg@1.2.3 resolves to AFTER thousands of builds have pinned that version.

npm allows unpublish within 72 hours and deprecate afterward. The infamous left-pad incident (2016) was a 17-line package whose author unpublished it, briefly breaking large parts of the JS ecosystem. The fix npm rolled out: any package downloaded ≥X times in the last week cannot be unpublished.

Most modern registries enforce strict immutability:

  • Maven Central: immutable, period. To "fix" a version, publish a new one.
  • PyPI: immutable; deletion possible but discouraged and audited.
  • OCI registries: immutable digests; tags are mutable but the digest the tag points to is the source of truth.

Pin to digests where you can:

# Dockerfile
FROM node:20-alpine@sha256:abc...   # pinned to immutable digest
# package-lock.json — pinned versions plus integrity hashes
"dependencies": {
  "react": {
    "version": "18.3.1",
    "integrity": "sha512-..."
  }
}

The integrity field is a SHA-512 of the package tarball. If npm ever served a different file under the same name+version, the install would fail.

Provenance — SLSA

SLSA (Supply chain Levels for Software Artifacts) defines a hierarchy of build-integrity guarantees:

  • Level 1: provenance is generated automatically and recorded.
  • Level 2: the build runs on a hosted service (no developer can fake it from a laptop).
  • Level 3: the build is hardened — every step's source and toolchain is recorded, the build environment is non-falsifiable.
  • Level 4: full review and reproducibility.

GitHub Actions + cosign generates SLSA Level 3 provenance with no extra work:

- uses: actions/attest-build-provenance@v1
  with:
    subject-name: ghcr.io/me/app
    subject-digest: ${{ digest }}

The attestation lands in Rekor (transparency log) and ties together: this digest, this source commit, this CI run, this build environment. Verifiers can check all four.

Promotion gates

Promotion shouldn't be a manual docker tag command run by whoever's awake. It should be a gated workflow:

build (in CI) → push @digest → :dev tag
                    │
   automated test, integration test, security scan
                    ▼
                ✅ promote → :staging tag
                    │
   smoke test, load test, manual approval
                    ▼
                ✅ promote → :prod tag

Gates:

  • Provenance verification at each promotion step.
  • Vulnerability scan (Trivy, Snyk) against the digest before promotion.
  • Test results pinned to the digest, not the tag — so retests can be re-run.
  • Audit trail — who promoted what, when, with what justification.

Tools: ArgoCD, Spinnaker, Argo Workflows, or hand-rolled GitHub Actions. Whatever you pick, the gates are the value.

Summary

  1. Use one registry per artifact type with retention policies set on day one.
  2. Build the artifact once; promote by digest, not tag.
  3. Sign with cosign + sigstore for provenance.
  4. Pin dependencies by digest (containers) or integrity hash (packages).
  5. Treat versions as immutable; never republish.
  6. Generate SLSA provenance attestations and verify them at deploy gates.

The headline cost of NOT doing this is a supply-chain compromise — your build infrastructure or registry is briefly compromised, malicious bytes reach prod, you're in the news. The mundane cost is registry bills and tag confusion. Both pay for the work.

Tools in the wild

5 tools
  • cosignfree tier

    Sigstore CLI to sign + verify container images and OCI artifacts.

    cli
  • OCI registry tied to GitHub Actions; keyless cosign signing out of the box.

    service
  • Harborfree tier

    Self-hosted registry — RBAC, vuln scanning, replication, signing.

    library
  • Managed OCI registry with KMS encryption + lifecycle policies for retention.

    service
  • Universal artifact repo (npm/pypi/maven/docker/...). Most production multi-language registries.

    service