caddy: tune ADR 0032 rate limits from Loki 429 data, and revisit the auth zone #302

Nytt ärende

Stängd

öppnade 2026-08-01 09:03:03 +00:00 av supernaut · 0 kommentarer

supernaut kommenterad

2026-08-01 09:03:03 +00:00

Ägare

Follow-up to the ADR 0032 Phase 2 rate limits applied in #296.

Tune from data

The ADR requires tuning from observed 429s and recording the final thresholds. Current values are
starting points, not measurements: Forgejo API 120/min, Forgejo general 300/min, auth 300/min, all
per client IP with ipv6_prefix 56. Query Loki for 429s by zone and vhost before tightening anything.

Revisit the auth zone specifically

It is deliberately 300/min rather than the ADR's ~20/min. Kanidm serves its SPA assets and several
XHRs from the same host, so one genuine interactive login is well over 20 requests, and a literal
20/min across all paths would lock real users out mid-sign-in on the only sign-in path there is.

If a tighter limit is wanted, scope a second zone to Kanidm's credential-submission endpoints rather
than the whole host. That needs someone to establish which paths those actually are.

Keep the internal exemption in mind

Every zone excludes private_ranges, and that is load-bearing. The first version of this change
throttled the reconciler into HTTP 429 within six minutes, because it reaches Forgejo over the
public URL (reconciler_forgejo_url) and all of its calls share one {remote_host} key. Any future
zone must carry the same exemption, and any new first-party caller that traverses the edge inherits
the same trap. Same shape as the fail2ban shared-SNAT ban in #195.

Follow-up to the ADR 0032 Phase 2 rate limits applied in #296. ## Tune from data The ADR requires tuning from observed 429s and recording the final thresholds. Current values are starting points, not measurements: Forgejo API 120/min, Forgejo general 300/min, auth 300/min, all per client IP with `ipv6_prefix 56`. Query Loki for 429s by zone and vhost before tightening anything. ## Revisit the auth zone specifically It is deliberately **300/min rather than the ADR's ~20/min**. Kanidm serves its SPA assets and several XHRs from the same host, so one genuine interactive login is well over 20 requests, and a literal 20/min across all paths would lock real users out mid-sign-in on the only sign-in path there is. If a tighter limit is wanted, scope a second zone to Kanidm's credential-submission endpoints rather than the whole host. That needs someone to establish which paths those actually are. ## Keep the internal exemption in mind Every zone excludes `private_ranges`, and that is load-bearing. The first version of this change throttled the reconciler into `HTTP 429` within six minutes, because it reaches Forgejo over the public URL (`reconciler_forgejo_url`) and all of its calls share one `{remote_host}` key. Any future zone must carry the same exemption, and any new first-party caller that traverses the edge inherits the same trap. Same shape as the fail2ban shared-SNAT ban in #195.

supernaut lade till etiketterna

area/infra

area/security

ready-for-implementation

type

task

2026-08-01 09:03:52 +00:00

supernaut lade till detta till projektet Bitborg Infrastructure

2026-08-01 09:04:09 +00:00

supernaut refererade till detta ärende

2026-08-01 14:09:52 +00:00

caddy: the signup rate-limit zone counts 4 requests per sign-up against a 10/min budget #304