hmmnm.com — artifacts at a signature gate: verified certificate sealed teal, forged card rejected amber

Cosign Signing and Verification Lab: Sign, Verify, Enforce

📋 Key Takeaways
  • Traditional software supply chain security has a well-established playbook: scan for CVEs, pin base images, gate dependencies.
  • Here’s the attack you’re defending against: an attacker compromises a CI secret or a maintainer’s registry token, pushes a malicious image under a legitimate tag—app:latest, app:v2.4.1, whatever your deployment manifests reference—and waits.
  • Cosign’s keyless mode replaces the old model—”generate a keypair, stash it somewhere, pray”—with something closer to how TLS works
  • Download the binary from the official release so you’re not trusting a third-party mirror
12 min read · 2,220 words
Educational & Ethical Use Only — This article is provided for educational and ethical cybersecurity research purposes only. The techniques described should only be used on systems you own or have explicit permission to test. Always follow responsible disclosure and the laws applicable to you. Mitigations are included so engineers can harden real systems.
Security· 12 min read

🔎 Last updated · Editorial policy · Report an error

TL;DR: Sign and Verify Container Images with Cosign

Cosign Signing and Verification Lab: Sign, Verify, Enforce

Cosign Container Signing Lab: Sign, Verify, Enforce Images

Cosign signs container images with short-lived, keyless Sigstore identities tied to your OIDC provider, and a verifying admission webhook blocks any unsigned or untrusted image from deploying to your cluster. Sign once in your pipeline, verify everywhere at admission time—that’s the whole loop, and this lab walks you through both ends of it with copy-paste commands.

Traditional software supply chain security has a well-established playbook: scan for CVEs, pin base images, gate dependencies. None of that answers a simpler, nastier question—is this image the one your build pipeline actually produced? Registries are write-mostly systems. Anyone with push access (or stolen credentials) can overwrite a tag, and every cluster pulling that tag downstream gets the tampered artifact. CISA and NSA called this out explicitly in their software supply chain guidance: provenance and integrity verification must be cryptographic, not inferred from registry trust. Sigstore’s cosign is the de facto open-source answer, and in this lab you’ll sign an image, verify it, and enforce verification at the Kubernetes admission gate—where it actually matters.

Why Image Signing Matters on the Supply Chain Front Line

Here’s the attack you’re defending against: an attacker compromises a CI secret or a maintainer’s registry token, pushes a malicious image under a legitimate tag—app:latest, app:v2.4.1, whatever your deployment manifests reference—and waits. Your clusters pull on their normal cadence. No runtime exploit needed; the malicious code ships through your own trusted pipeline’s output.

OWASP’s Software Supply Chain Security guidance ranks exactly this class of attack—artifact tampering between build and deployment—among the highest-impact supply chain threats. Signatures close the gap because they bind an artifact to an identity cryptographically. If the image digest changes, the signature fails. If the signer isn’t the identity you expect, verification fails. The signature transforms “the registry says this is my image” into “the holder of a specific identity attests this exact digest is my image”—and you’re no longer trusting registry access control as your only line of defense.

How Sigstore and Cosign Work: Fulcio, Rekor, and Keyless Identity

Cosign’s keyless mode replaces the old model—”generate a keypair, stash it somewhere, pray”—with something closer to how TLS works:

  • Fulcio is the certificate authority. Instead of a long-lived private key, cosign requests an ephemeral keypair at signing time and gets it certified by Fulcio against an OIDC (OpenID Connect) identity—your Google account, GitHub account, or a CI workload identity. The resulting certificate says “this short-lived key belonged to alice@example.com at this moment.”
  • Rekor is the transparency log. Every signature and certificate gets recorded in an append-only, publicly auditable Merkle log—the same tamper-evidence model as Certificate Transparency. If someone forges a signature later, the discrepancy is provable.
  • The signature itself is stored in the registry alongside your image, attached to the image digest—not the mutable tag.

The practical consequence: signers never manage keys, and verifiers check identity plus transparency-log inclusion rather than distributing public keys by hand. Key-based signing still exists (we’ll cover when you want it), but keyless is the default path for teams.

Lab Prerequisites and Environment Setup

Get these ready before you start:

  • cosign CLI (v2.x—we’ll install it in step 1)
  • A container registry you can push to—this lab uses ghcr.io, but Docker Hub or any OCI registry works
  • A local Kubernetes cluster: kind or minikube, both fine
  • kubectl pointed at that cluster
  • An OIDC identity—Google or GitHub account for interactive signing; GitHub Actions OIDC if you later automate this in CI

Start your cluster so everything downstream has somewhere to run:

kind create cluster --name cosign-lab
kubectl cluster-info

Step 1: Install Cosign and Confirm Your Environment

Download the binary from the official release so you’re not trusting a third-party mirror:

curl -O -L "https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64"
sudo mv cosign-linux-amd64 /usr/local/bin/cosign
sudo chmod +x /usr/local/bin/cosign
cosign version

Confirm you see v2.x output (verified on the cosign v2 release current at publication; pin and record your exact version when you reproduce — flag and key-handling behavior shifts between minor releases). Then authenticate to your registry:

echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USERNAME --password-stdin

Push a test image so you have something to sign:

docker pull nginx:1.27-alpine
docker tag nginx:1.27-alpine ghcr.io/YOUR_USERNAME/cosign-lab:signed
docker push ghcr.io/YOUR_USERNAME/cosign-lab:signed

Also push one you’ll leave unsigned for the enforcement test later:

docker tag nginx:1.27-alpine ghcr.io/YOUR_USERNAME/cosign-lab:unsigned
docker push ghcr.io/YOUR_USERNAME/cosign-lab:unsigned

Step 2: Sign an Image with Cosign Keyless Signing

One command handles the entire keyless flow:

cosign sign --yes ghcr.io/YOUR_USERNAME/cosign-lab:signed

Two things happen:

  • Cosign opens a browser-based OIDC flow—you authenticate (Google, GitHub, or Microsoft), Fulcio issues a short-lived certificate (valid for minutes) bound to your email identity, and cosign signs the image digest with the ephemeral key.
  • The signature is uploaded to the registry as a tag-derived artifact (sha256-...sig), and an entry is written to the Rekor transparency log.

The --yes flag skips the interactive confirmation prompt. Inspect what you created—because you want the certificate and inclusion proof in hand before you rely on them:

cosign verify --certificate-identity-regexp ".*" 
  --certificate-oidc-issuer-regexp ".*" 
  ghcr.io/YOUR_USERNAME/cosign-lab:signed

You’ll see JSON output containing the Fulcio certificate, the identity it was issued to, and the Rekor inclusion proof. That JSON is the cryptographic receipt for your image.

Step 3: Verify the Signature Before You Install

Verification is where the trust decision happens, and the identity flags are everything. Verify against your exact identity so nothing else can satisfy the check:

cosign verify 
  --certificate-identity "YOUR_USERNAME@YOUR_EMAIL_DOMAIN" 
  --certificate-oidc-issuer "https://accounts.google.com" 
  ghcr.io/YOUR_USERNAME/cosign-lab:signed

Expected output: a JSON blob ending with a verified checksum line for the exact digest. Exit code 0.

What failure looks like: verify the unsigned tag and you’ll get Error: no signatures found—cosign found no signature artifact for that digest and exits non-zero. Verify the signed image with the wrong identity and you’ll get an identity-mismatch error. Both failures are the point: verification fails closed.

Pin to digest in real deployments—tags are mutable:

cosign verify 
  --certificate-identity "YOUR_USERNAME@YOUR_EMAIL_DOMAIN" 
  --certificate-oidc-issuer "https://accounts.google.com" 
  ghcr.io/YOUR_USERNAME/cosign-lab@sha256:$(docker inspect --format='{{index .RepoDigests 0}}' ghcr.io/YOUR_USERNAME/cosign-lab:signed | cut -d: -f2)

Step 4: Deploy a Cosign-Verifying Admission Webhook

Manual verification only helps if someone remembers to run it. Enforcement belongs in the cluster’s admission path. The Sigstore project’s own policy-controller is purpose-built for this; Kyverno is the other common choice. Here’s the Sigstore policy-controller on kind:

helm repo add sigstore https://sigstore.github.io/helm-charts
helm repo update
helm install policy-controller sigstore/policy-controller 
  --namespace cosign-system --create-namespace

Now define a verification policy scoped to a namespace. Labels matter—the policy-controller only enforces on namespaces labeled policy.sigstore.dev/include=true:

apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: require-cosign-signature
spec:
  images:
  - glob: "ghcr.io/YOUR_USERNAME/**"
  authorities:
  - keyless:
      url: "https://fulcio.sigstore.dev"
      identities:
      - issuer: "https://accounts.google.com"
        subject: "YOUR_USERNAME@YOUR_EMAIL_DOMAIN"
    ctlog:
      url: "https://rekor.sigstore.dev"

Apply it and label the namespace:

kubectl apply -f cluster-image-policy.yaml
kubectl label namespace default policy.sigstore.dev/include=true

The same enforcement logic exists in Kyverno as a verifyImages rule with verifyDigest and keyless issuer/subject fields—use whichever engine your platform team already runs.

Verification Approaches at a Glance

Approach Where It Runs Key Management Best For
Manual cosign verify Developer workstation None (keyless) Spot checks, debugging signature failures
CI pipeline gate Build pipeline None (keyless, OIDC workload identity) Catching breakage before deploy
Sigstore policy-controller Kubernetes admission None (keyless + Rekor) Cluster-wide enforcement (this lab)
Kyverno verifyImages Kubernetes admission None (keyless) or KMS/HSM keys Teams already running Kyverno
Key-based sign/verify Anywhere You own rotation and distribution Air-gapped, high-assurance environments

Step 5: Test Enforcement — Trusted vs. Unsigned Images

Deploy the signed image first:

kubectl run signed-test --image=ghcr.io/YOUR_USERNAME/cosign-lab:signed --restart=Never

The pod is admitted. Check with kubectl get pod signed-test—it’s running.

Now the unsigned image:

kubectl run unsigned-test --image=ghcr.io/YOUR_USERNAME/cosign-lab:unsigned --restart=Never

The webhook rejects it — the pod is never created, so kubectl run itself returns the denial:

Error from server (Forbidden): admission webhook "policy.sigstore.dev" denied the request:
signature/attestation for image ghcr.io/YOUR_USERNAME/cosign-lab@sha256:... not verified

That message answers “why was my unsigned image rejected”—the policy-controller queried the registry for a cosign signature on the exact digest, found none (or one that failed identity/Rekor checks), and the admission webhook denied pod creation. Fails closed, by design.

Troubleshooting Common Cosign and Webhook Errors

  • Expired or missing OIDC token: keyless certificates live for minutes. If signing fails with an OIDC error mid-flow, re-authenticate—don’t cache tokens across sessions.
  • Identity mismatch: --certificate-identity must match the email Fulcio certified exactly (use --certificate-identity-regexp only when you understand the blast radius). A trailing space or personal vs. work account silently breaks verification.
  • Rekor lookup failures: corporate proxies blocking rekor.sigstore.dev cause verify to fail. Fix egress, or use offline verification with a bundled certificate (see FAQ).
  • Registry permissions: cosign stores signatures as registry artifacts. Your credentials need push access for cosign sign and pull access for cosign verify—and the policy-controller’s service account needs pull access to both the image and the signature artifacts.
  • Webhook not enforcing: nine times out of ten, the namespace is missing the policy.sigstore.dev/include=true label.

Hardening Tips and CI/CD Integration

Once the loop works locally, productionize it:

  • Pin identities tightly. In policy, match issuer and subject exactly. For CI, use the workload-identity form: issuer https://token.actions.githubusercontent.com with a subject like https://github.com/ORG/REPO/.github/workflows/release.yml@refs/heads/main.
  • Sign in GitHub Actions. Keyless was designed for this—the runner’s OIDC token becomes the signing identity, no secrets stored. Same cosign sign --yes command, zero credential management.
  • Verify in the pipeline too. Add cosign verify as a gate before deployment, so you catch signature breakage before it hits the admission webhook.
  • Key-based fallback. Keyless depends on Sigstore’s public infrastructure. For air-gapped or high-assurance environments, cosign generate-key-pair with the private key in a KMS or HSM and --key flags on both sign and verify is the right pattern. Understand the tradeoff: you inherit key management, rotation, and distribution responsibilities.

Key Takeaways and Next Steps

The sign-verify-enforce loop is short and it compounds: sign at build time with keyless identity, verify at deployment time with exact identity matching, enforce at admission so no human can bypass it. Once that’s running, extend it:

  • SLSA — the SLSA framework layers build provenance on top of artifact signing; cosign attestations carry SLSA provenance natively.
  • SBOMs — attach and attest SBOMs with cosign attest, building toward the CISA SBOM guidance (cisa.gov/sbom).
  • Policy-as-code — expand ClusterImagePolicy rules to require attestations, not just signatures, and manage policies like any other code.

Better signatures, tighter identities, enforced verification, richer attestations—that’s the maturity path. Start with step 1 above and ship it this week.

Frequently Asked Questions

What is cosign keyless signing?

Keyless signing means cosign signs with a short-lived (minutes-long) keypair tied to an OIDC identity—your Google, GitHub, or CI workload identity—instead of a long-term key you manage. Fulcio issues the ephemeral certificate binding the key to your identity, and the signature is recorded in the Rekor transparency log for public tamper evidence. No keys to store, rotate, or leak.

Can cosign verify signatures without network access to Rekor?

Yes. Cosign supports offline verification when the signing certificate is bundled with the signature (via --bundle or offline artifacts), checking the certificate chain directly. You lose the transparency log’s tamper-evidence guarantees and any real-time inclusion proof, so weigh offline verification for air-gapped environments against the added assurance of a live Rekor check.

Which admission controllers or policy engines support cosign verification?

The three most common: the Sigstore policy-controller (purpose-built, used in this lab), Kyverno with verifyImages rules, and Connaisseur. All three enforce signature checks at pod admission time and support keyless identity matching.

Does keyless signing work in CI pipelines?

Yes—and it’s the primary use case. GitHub Actions and GitLab CI both provide OIDC tokens that cosign consumes automatically. In GitHub Actions with permissions: id-token: write, simply run cosign sign --yes $IMAGE; cosign detects the environment, requests the workload identity from Fulcio, and signs without any stored secrets.

How do I restrict verification to a specific identity?

On the CLI, use --certificate-identity (exact email or URI) and --certificate-oidc-issuer (the OIDC provider URL). In admission policies, set the same values in the identities block—subject and issuer—under the keyless authority. Never rely on “a valid signature exists” alone; the identity is the trust anchor.

Hmmnm
Published by Hmmnm

Hands-on cybersecurity tutorials, CVE breakdowns, and guided learning paths — written and lab-tested by the Hmmnm team.

Keep going — the structured way
This post is one step. The learning paths chain the next ones for you, with progress tracking and no account needed.
Follow a learning path →

Prabhu Kalyan Samal

Application Security Consultant and founder of Hmmnm. Certified Red Team Operator, Burp Suite Certified Practitioner, eWPTX, CompTIA SecurityX and Azure Security Engineer. Writes hands-on labs and threat analysis on web, cloud and AI-agent security.