backup-drill: restore and verify Kanidm on the scratch VM #458

Nytt ärende

Stängd

öppnade 2026-09-08 23:04:25 +00:00 av supernaut · 1 kommentar

supernaut kommenterad

2026-09-08 23:04:25 +00:00

Ägare

The weekly drill (ansible/roles/backup-drill/templates/restore-on-scratch.sh.j2) restores Postgres, Forgejo data and the restic storage, then runs forgejo doctor. It never touches kanidm-backup.json. The weekly verify only gzip-parses it.

Kanidm is the sole identity provider for Forgejo, the portal and Grafana. A kanidmd database restore that breaks on an image bump is found on the day it is needed.

Add a Kanidm restore step to the drill script: start a one-shot Kanidm container against the restored dump (the server must be stopped for database restore, so use the one-shot podman run pattern the runbook uses for domain rename), then probe /status and count persons and groups. Fail the drill if either is zero. Emit the counts as drill metrics.

Related: #157, #161.

The weekly drill (`ansible/roles/backup-drill/templates/restore-on-scratch.sh.j2`) restores Postgres, Forgejo data and the restic storage, then runs `forgejo doctor`. It never touches `kanidm-backup.json`. The weekly verify only gzip-parses it. Kanidm is the sole identity provider for Forgejo, the portal and Grafana. A `kanidmd database restore` that breaks on an image bump is found on the day it is needed. Add a Kanidm restore step to the drill script: start a one-shot Kanidm container against the restored dump (the server must be stopped for `database restore`, so use the one-shot `podman run` pattern the runbook uses for `domain rename`), then probe `/status` and count persons and groups. Fail the drill if either is zero. Emit the counts as drill metrics. Related: #157, #161.

supernaut lade till detta till projektet Bitborg Infrastructure

2026-09-08 23:04:45 +00:00

supernaut lade till etiketterna

2026-09-08 23:05:23 +00:00

supernaut refererade till detta ärende från bitborg/bitborg-docs

2026-09-24 13:04:40 +00:00

Data safety: recovery that has been proven, not assumed #107