monitoring: Forgejo error-rate and latency alerts, Caddy 5xx rule #460

Nytt ärende

Öppen

öppnade 2026-09-08 23:04:25 +00:00 av supernaut · 0 kommentarer

supernaut kommenterad

2026-09-08 23:04:25 +00:00

Ägare

Forgejo /metrics is scraped (vmagent-scrape.yml.j2) but no alert rule reads it. Caddy access logs reach Loki but no LogQL rule exists. A partial outage that still answers up is invisible.

Add vmalert rules: Forgejo HTTP 5xx ratio over 5 minutes, p95 request latency, and a Loki rule on Caddy 5xx per vhost. Validate with scripts/check-alert-rules.py and add a negative-control query for each, as the existing rules do.

Forgejo `/metrics` is scraped (`vmagent-scrape.yml.j2`) but no alert rule reads it. Caddy access logs reach Loki but no LogQL rule exists. A partial outage that still answers `up` is invisible. Add vmalert rules: Forgejo HTTP 5xx ratio over 5 minutes, p95 request latency, and a Loki rule on Caddy 5xx per vhost. Validate with `scripts/check-alert-rules.py` and add a negative-control query for each, as the existing rules do.

supernaut lade till detta till projektet Bitborg Infrastructure

2026-09-08 23:04:48 +00:00

supernaut lade till etiketterna

2026-09-08 23:05:23 +00:00

Logga in för att delta i denna konversation.