fix(health-check): the concealment gate fails on the monitoring host #350

Nytt ärende

Stängd

öppnade 2026-08-02 16:25:28 +00:00 av supernaut · 0 kommentarer

supernaut kommenterad

2026-08-02 16:25:28 +00:00

Ägare

ansible-playbook site.yml --check fails on the monitoring host:

TASK [health-check : Health gate — identity concealment was verified against the deployed image tags (ADR 0038)]
[ERROR]: Task failed: Finalization of task args for 'ansible.builtin.assert' failed:
         Error while resolving value for 'fail_msg': 'kanidm_image_tag' is undefined
fatal: [gitborg-monitoring]: FAILED!

Cause

The concealment gate asserts the deployed Forgejo and Kanidm image tags. Those two variables are not
scoped the same way:

Variable	Defined in	Visible to
`forgejo_image_tag`	`ansible/group_vars/all/vars.yml`	every host
`kanidm_image_tag`	`ansible/roles/kanidm/defaults/main.yml`	only plays that include the kanidm role

The monitoring play runs base, podman, monitoring and health-check — no kanidm role — so
kanidm_image_tag is never defined there. health_check_concealment defaults to true, so the gate
runs anyway, and the assert dies while templating its fail_msg.

It is not specific to running without tags

health-check carries the always tag, so the gate runs on every invocation that reaches the
monitoring host. Reproduced with an explicitly tagged run:

ansible-playbook site.yml --check --limit monitoring --tags health-check
→ gitborg-monitoring : ok=1 changed=0 unreachable=0 failed=1

A tagged apply is not protection.

This is the second time a health gate has failed on the monitoring host

The task immediately above it carries this comment:

Read-only GET, so it is safe under --check and MUST run there: gated on not ansible_check_mode
this task was skipped in every dry-run, which is why a gate that fails on the monitoring host
reached production before anyone saw it. Same trap as #257.

Worth considering whether the role should distinguish services-host gates from gates that apply to
every host, rather than each one being scoped by hand as it is discovered.

Fix

Set health_check_concealment: false on the monitoring play's health-check invocation, next to the
health_check_services override that is already there.

Deliberately not guarded with kanidm_image_tag is defined inside the role: a gate that silently
skips when a variable is missing is exactly the trap the role's own comments warn about, and it would
hide the same mistake on the host where the gate does matter.

Done when

ansible-playbook site.yml --check completes without this failure
The gate still runs on the services host — it must not be disabled globally

`ansible-playbook site.yml --check` fails on the monitoring host: ```text TASK [health-check : Health gate — identity concealment was verified against the deployed image tags (ADR 0038)] [ERROR]: Task failed: Finalization of task args for 'ansible.builtin.assert' failed: Error while resolving value for 'fail_msg': 'kanidm_image_tag' is undefined fatal: [gitborg-monitoring]: FAILED! ``` ## Cause The concealment gate asserts the deployed Forgejo and Kanidm image tags. Those two variables are not scoped the same way: | Variable | Defined in | Visible to | | --- | --- | --- | | `forgejo_image_tag` | `ansible/group_vars/all/vars.yml` | every host | | `kanidm_image_tag` | `ansible/roles/kanidm/defaults/main.yml` | only plays that include the kanidm role | The monitoring play runs `base`, `podman`, `monitoring` and `health-check` — no kanidm role — so `kanidm_image_tag` is never defined there. `health_check_concealment` defaults to `true`, so the gate runs anyway, and the assert dies while templating its `fail_msg`. ## It is not specific to running without tags `health-check` carries the `always` tag, so the gate runs on every invocation that reaches the monitoring host. Reproduced with an explicitly tagged run: ```text ansible-playbook site.yml --check --limit monitoring --tags health-check → gitborg-monitoring : ok=1 changed=0 unreachable=0 failed=1 ``` A tagged apply is not protection. ## This is the second time a health gate has failed on the monitoring host The task immediately above it carries this comment: > Read-only GET, so it is safe under `--check` and MUST run there: gated on `not ansible_check_mode` > this task was skipped in every dry-run, which is why a gate that fails on the monitoring host > reached production before anyone saw it. Same trap as #257. Worth considering whether the role should distinguish services-host gates from gates that apply to every host, rather than each one being scoped by hand as it is discovered. ## Fix Set `health_check_concealment: false` on the monitoring play's `health-check` invocation, next to the `health_check_services` override that is already there. Deliberately **not** guarded with `kanidm_image_tag is defined` inside the role: a gate that silently skips when a variable is missing is exactly the trap the role's own comments warn about, and it would hide the same mistake on the host where the gate does matter. ## Done when - [ ] `ansible-playbook site.yml --check` completes without this failure - [ ] The gate still runs on the services host — it must not be disabled globally

supernaut refererade till detta ärende från en incheckning

2026-08-02 16:32:52 +00:00

fix(health-check): do not run the concealment gate on the monitoring host

supernaut hänvisade till detta ärende från en ändringsförfrågan som kommer att stänga det

2026-08-02 16:32:52 +00:00

fix(health-check): do not run the concealment gate on the monitoring host #351

supernaut lade till etiketterna

area/infra

area/observability

ready-for-implementation

type

bug