Aller au contenu

ADR-032: Container image signing + SBOM (cosign + CycloneDX)

Status

Accepted — 2026-04-25 (Sprint 46 Stream A1)

Context

The CTO audit dated 2026-04-25 flagged "no image signing = supply chain attack possible" as a top-tier risk. AKKO publishes 14 custom images per release (harbor.akko-ai.com/akko/<image>:2026.04). They are pulled by every cluster — k3d dev, customer Netcup, future EKS / GKE — and the only thing that today proves authenticity is "we trust Harbor". A registry compromise, a stolen Harbor robot account, or a single malicious PR squash-merging into main would let an attacker ship code to every operator without anyone noticing.

Banking and healthcare prospects (the two pilot verticals tracked in akko-confidential/sales-playbook/) explicitly require:

  • DORA Article 6 — ICT supply-chain integrity controls.
  • NIS2 Annex II — software bill of materials for "essential entities".
  • HDS (Hébergeur de Données de Santé) Section 6.7 — provenance verification of every container deployed in the production data plane.

We need:

  1. A way to sign every image so operators can prove "this artefact was published by AKKO and was not tampered with after build".
  2. A way to enumerate every dependency (SBOM) so a CVE in libssl 3.0.7 surfaces in 14 builds without re-scanning each image.
  3. A way to verify both signals at pull time — eventually as an admission policy, today as a documented operator step.

Constraints:

  • 100 % open source, Apache-2.0-or-equivalent license. No Docker Scout, no Snyk Container, no Aqua. Sovereign-friendly.
  • Foundation governance preferred (ADR-029 rule). Single-vendor projects are excluded for load-bearing supply-chain bricks.
  • Must run inside the existing Woodpecker pipeline. Build runners have no daemon, no privileged access beyond what Buildah already requires (ADR-025).
  • Toggleable for air-gapped / offline customers. Not every operator can reach fulcio.sigstore.dev from inside their build network.
  • Idempotent. A Woodpecker re-run on the same digest must not duplicate signatures or fail.

Considered options

Image signing

Option License Governance Verdict
cosign (Sigstore) Apache 2.0 Linux Foundation (OpenSSF) Selected
GPG / docker trust GPL Docker Inc. (single-vendor) Workflow predates SLSA, key management is operator-owned, no transparency log. Rejected.
Notary v1 Apache 2.0 CNCF (deprecated) Replaced by Notary v2 / cosign. Rejected.
Notary v2 / notation Apache 2.0 CNCF Viable alternative. Smaller community than cosign, weaker tooling around SBOM attestation. Held in reserve as a second source.
Docker Scout Commercial (Docker Inc.) Single-vendor Violates the open-source constraint. Rejected.

Why cosign: Linux Foundation governance (rule from ADR-029), first-class CycloneDX + SPDX attestation support, mature ecosystem (Kyverno, Gatekeeper, Tekton Chains, ArgoCD all consume cosign signatures natively), and the keyless mode (Fulcio + Rekor) gives us a forward path to zero long-lived signing keys.

SBOM format

Option Maintainer License 2025+ industry default?
CycloneDX 1.5 OWASP Apache 2.0 Yes — adopted by NIST, OWASP, Anchore, Snyk, Dependency-Track
SPDX 2.3 Linux Foundation CC-BY-3.0 Common in Linux distro / SBOM-as-policy work, weaker tooling for container SBOMs
Syft native Anchore Apache 2.0 Internal only, not consumed downstream

Why CycloneDX: OWASP standard, Apache 2.0, the format every commercial SBOM scanner (Anchore Enterprise, Snyk, JFrog Xray, Dependency-Track) ingests natively. SPDX remains a strong second choice, but CycloneDX is the 2025+ default for container-centric SBOMs. We emit CycloneDX 1.5 JSON; we can add SPDX as a second predicate later (cosign supports both side-by-side on the same digest).

SBOM generator

syft (Anchore, Apache 2.0) — broadest ecosystem support (Alpine, Debian, RPM, Python, Java, Node, Go, Rust). The single-vendor Anchore caveat is mitigated by the BYOS pattern from ADR-029: SBOM generation is a pluggable build step, not load-bearing. Trivy can also emit CycloneDX SBOMs and is already in pipeline 06-trivy.yml — we keep it as a fallback.

Signing identity (keypair vs keyless Fulcio)

We ship keypair as the day-1 default and plan keyless Fulcio as the long-term default.

  • Keypair: deterministic, runs in air-gapped clusters, no external dependency. Downside: a long-lived private key in a K8s Secret is a juicy target. Rotation procedure is documented in helm/scripts/cosign-init.sh --rotate.
  • Keyless Fulcio: short-lived OIDC-bound certificates, transparency log via Rekor, no private key to leak. Requires the build runner to obtain an OIDC token bound to the pipeline. Woodpecker v3.4 lacks first-class OIDC issuer integration (tracked upstream: woodpecker-ci/woodpecker#3210); migration deferred until that ships. Toggle today via AKKO_SIGN_KEYLESS=true for operators whose build environment already has the OIDC plumbing.

Decision

  1. Sign every image published to harbor.akko-ai.com/akko/*:2026.04 with cosign.
  2. Attach a CycloneDX 1.5 SBOM as an in-toto attestation on the same digest (predicate type cyclonedx).
  3. Pipeline: new .woodpecker/10-image-sign.yml runs after the build steps in 08-deploy-netcup.yml succeed.
  4. Bootstrap: helm/scripts/cosign-init.sh generates the keypair and stores it as Secret/akko-cosign-keys in the harbor namespace. The public key is committed at helm/akko/charts/akko-cockpit/files/cosign.pub.
  5. Verification today (Sprint 46): documented operator command + a cosign verify step appended to 06-trivy.yml image-scan so a missing signature blocks merges to main.
  6. Verification tomorrow (Sprint 47): Kyverno or OPA Gatekeeper policy at admission time refuses any pod whose image lacks a valid AKKO signature.
  7. Kill switch: AKKO_SIGN_IMAGES=false (Woodpecker secret) skips the entire signing pipeline. Default is true.

Verification commands (operators)

# Pull the AKKO public key (committed in the repo, mirror-friendly)
PUBKEY="https://harbor.akko-ai.com/api/v2.0/projects/akko/repositories/cockpit/artifacts/2026.04/additions/cosign.pub"

# Verify signature
cosign verify --key cosign.pub harbor.akko-ai.com/akko/cockpit:2026.04

# Pull and validate the CycloneDX SBOM
cosign download attestation harbor.akko-ai.com/akko/cockpit:2026.04 \
  | jq -r .payload | base64 -d | jq '.predicate' > sbom-cockpit.cdx.json

# Forward to your SBOM scanner
curl -X POST -H "Content-Type: application/vnd.cyclonedx+json" \
  --data-binary @sbom-cockpit.cdx.json \
  https://your-dependency-track/api/v1/bom

Future ImagePolicyWebhook (Sprint 47)

A Kyverno ClusterPolicy (or OPA Gatekeeper ConstraintTemplate) will enforce, at admission time, that every pod whose image starts with harbor.akko-ai.com/akko/ carries:

  • A valid cosign signature against cosign.pub mounted from Secret/akko-cosign-keys.
  • A cyclonedx attestation on the same digest, generated within the last 90 days.
# helm/akko/charts/akko-opa/templates/clusterpolicy-image-signing.yaml (Sprint 47)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: akko-require-signed-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: verify-signature
      match:
        any:
          - resources:
              kinds: [Pod]
      verifyImages:
        - imageReferences:
            - "harbor.akko-ai.com/akko/*"
          attestors:
            - entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      ...
                      -----END PUBLIC KEY-----
          attestations:
            - type: cyclonedx
              attestors:
                - entries:
                    - keys:
                        publicKeys: |-
                          -----BEGIN PUBLIC KEY-----
                          ...
                          -----END PUBLIC KEY-----

This decouples signing (CI) from verification (cluster) so a future helm upgrade --set akko-opa.imagePolicy.enforce=true flip is a single value change, not a code change.

Consequences

Positive

  • DORA / NIS2 / HDS supply-chain checkbox addressed end-to-end.
  • Banking + healthcare prospects can audit AKKO's provenance trail with industry-standard tooling (no proprietary verifier required).
  • Future-proof: keyless Fulcio migration is one env flag away when Woodpecker OIDC lands.
  • Idempotent — re-running the pipeline costs ~2 s/image after the first run.

Negative / mitigations

  • Pipeline runtime +30-60 s per image on the first run (sign + syft scan + attest). Cached on subsequent runs. Acceptable.
  • Long-lived signing key in K8s Secret. Mitigated by (a) tight RBAC on the harbor namespace, (b) rotation procedure, (c) keyless migration plan.
  • Fulcio reachability in air-gapped builds. Mitigated by the AKKO_SIGN_IMAGES=false kill switch.
  • One more thing to maintain. Mitigated by the modular Woodpecker file structure — disabling the pipeline is a 1-line change.

References

  • Sigstore project: https://www.sigstore.dev (Linux Foundation, OpenSSF)
  • cosign GitHub: https://github.com/sigstore/cosign (Apache 2.0)
  • syft GitHub: https://github.com/anchore/syft (Apache 2.0)
  • CycloneDX spec: https://cyclonedx.org (OWASP)
  • SLSA framework: https://slsa.dev (Linux Foundation, OpenSSF)
  • Kyverno verifyImages: https://kyverno.io/docs/writing-policies/verify-images/
  • DORA Article 6: https://eur-lex.europa.eu/eli/reg/2022/2554/oj
  • NIS2 Annex II: https://eur-lex.europa.eu/eli/dir/2022/2555/oj
  • ADR-022: CI strategy
  • ADR-025: Buildah privileged build steps
  • ADR-029: Governance over license — tool selection rule

Migration / rollout plan

Step When Owner Status
Bootstrap key, K8s Secret, public key in repo Sprint 46 W1 Stream A1 Done (this ADR + cosign-init.sh)
New pipeline .woodpecker/10-image-sign.yml Sprint 46 W1 Stream A1 Done
cosign verify enforcement in 06-trivy.yml Sprint 46 W1 Stream A1 Done
Operator docs (docs/docs/admin/security.md) Sprint 46 W1 Stream A1 Done
Kyverno policy in audit mode Sprint 47 W1 Stream A2 Backlog
Kyverno policy in enforce mode Sprint 47 W2 Stream A2 Backlog
Keyless Fulcio migration Once Woodpecker #3210 lands Stream A1 Backlog