feat(ci): Tier-0 smoke — hardened containers can start (#104) #110

Sammanfogat

supernaut sammanfogade 1 incheckning från feat/104-tier0-container-smoke in i main

2026-07-18 16:41:26 +00:00

supernaut kommenterad

2026-07-18 16:38:01 +00:00

Ägare

First increment of the pre-prod validation epic (bitborg-docs#37), targeting the exact failure class that took prod down this week (#81/#94) and that ansible-playbook --check cannot catch — it never starts a container.

What it does

scripts/smoke-containers.py: for every roles/*/templates/*.container.j2 with NoNewPrivileges=true + DropCapability=ALL and a public image, it resolves the image + declared capabilities and does a real podman run with the same --cap-drop/--cap-add/--security-opt=no-new-privileges, then asserts the entrypoint could exec.

An OCI exec-permission failure (a file-cap binary missing a needed cap — e.g. Caddy's /usr/bin/caddy needing NET_BIND_SERVICE) is detected via State.Error / a 126–127 exit code.
App-level config exits (missing env/Caddyfile) pass — the binary exec'd, which is all Tier-0 asserts.
Internal-registry images (runner-controller, web) are skipped — Tier-1 (#105) covers those on a real host.

Wired into .forgejo/workflows/ci.yml; runs on the host-backend runner (ADR 0021 ci:host → the job has the VM's own podman). PyYAML comes from the existing pip install ansible step.

Validated locally (podman 5.8.3), both directions

Positive: all 4 hardened public units (caddy/forgejo/postgres/kanidm) PASS, exit 0.
Negative (teeth): re-introducing the #94 regression — dropping caddy's NET_BIND_SERVICE — makes the smoke FAIL with a non-zero exit and the #94 explanation. A test that can't fail is worthless; this one does.

Scope / remaining #104 work (tracked on the issue)

A textfile-metric readability/parse smoke for the #96 class (the token-audit.prom 0600 bug node_exporter rejected).
The full multi-service integration converge — that needs a real host, so it's Tier-1 (#105), not this cheap every-PR tier.

Had this existed, both prod incidents this week (#94 outage, #96 silent metric gap) would have been caught in CI.

First increment of the pre-prod validation epic (bitborg-docs#37), targeting the exact failure class that took prod down this week (#81/#94) and that `ansible-playbook --check` cannot catch — it never starts a container. ## What it does `scripts/smoke-containers.py`: for every `roles/*/templates/*.container.j2` with `NoNewPrivileges=true` + `DropCapability=ALL` and a **public** image, it resolves the image + declared capabilities and does a real `podman run` with the same `--cap-drop`/`--cap-add`/`--security-opt=no-new-privileges`, then asserts the entrypoint could **exec**. - An OCI exec-permission failure (a file-cap binary missing a needed cap — e.g. Caddy's `/usr/bin/caddy` needing `NET_BIND_SERVICE`) is detected via `State.Error` / a 126–127 exit code. - App-level config exits (missing env/Caddyfile) **pass** — the binary exec'd, which is all Tier-0 asserts. - Internal-registry images (runner-controller, web) are **skipped** — Tier-1 (#105) covers those on a real host. Wired into `.forgejo/workflows/ci.yml`; runs on the host-backend runner (ADR 0021 `ci:host` → the job has the VM's own podman). PyYAML comes from the existing `pip install ansible` step. ## Validated locally (podman 5.8.3), both directions - **Positive:** all 4 hardened public units (caddy/forgejo/postgres/kanidm) PASS, exit 0. - **Negative (teeth):** re-introducing the #94 regression — dropping caddy's `NET_BIND_SERVICE` — makes the smoke **FAIL** with a non-zero exit and the #94 explanation. A test that can't fail is worthless; this one does. ## Scope / remaining #104 work (tracked on the issue) - A textfile-metric readability/parse smoke for the **#96** class (the `token-audit.prom` 0600 bug node_exporter rejected). - The full multi-service integration converge — that needs a real host, so it's Tier-1 (#105), not this cheap every-PR tier. Had this existed, both prod incidents this week (#94 outage, #96 silent metric gap) would have been caught in CI.

supernaut lade till 1 incheckning

2026-07-18 16:38:01 +00:00

feat(ci): Tier-0 smoke — hardened containers can start (#104 )

ci / ci (pull_request) Successful in 2m20s

Detaljer

3656d9901c

First increment of the staging/validation epic (gitborg-docs#37): a CI
guard for the failure class that took prod down in #81/#94 and that
ansible-playbook --check cannot see (it never starts a container).

scripts/smoke-containers.py: for every roles/*/templates/*.container.j2
with NoNewPrivileges=true + DropCapability=ALL and a PUBLIC image,
resolve its image + declared caps and 'podman run' it with the same
--cap-drop/--cap-add/--security-opt, asserting the entrypoint can exec.
An OCI exec-permission failure (file-cap binary minus a needed cap, e.g.
caddy needing NET_BIND_SERVICE) is detected via State.Error / 126-127
exit; app-level config exits pass (the binary exec'd). Internal-registry
images (runner-controller, web) are skipped — Tier-1 (#105) covers those
on a real host.

Wired into .forgejo/workflows/ci.yml; runs on the host-backend runner
(ADR 0021 ci:host → the job has the VM's own podman).

Validated locally (podman 5.8.3) both directions: all 4 hardened public
units (caddy/forgejo/postgres/kanidm) PASS; re-introducing the #94
regression (drop caddy's NET_BIND_SERVICE) makes the smoke FAIL with a
non-zero exit — the guard has teeth.

Remaining #104 scope (tracked on the issue): a textfile-metric
readability/parse smoke for the #96 class, and the full multi-service
converge (Tier-1, #105).