bitborg/bitborg-infra

Bevaka

Avgrena 0

No fail2ban jail for git-over-HTTPS / API brute force (only sshd is jailed) #127

Nytt ärende

Stängd

öppnade 2026-07-19 06:58:03 +00:00 av supernaut · 1 kommentar

supernaut kommenterad

2026-07-19 06:58:03 +00:00

Ägare

Severity: MEDIUM — pre-onboarding infra audit (2026-07-19).

ansible/roles/base/templates/jail.local.j2 has only an [sshd] jail. HTTP Basic auth stays enabled (ENABLE_INTERNAL_SIGNIN=false only removes the web form), so git-over-HTTPS and API token auth can be brute-forced with no host-layer lockout — nothing tails Caddy/Forgejo logs for auth abuse.

Ask

Add a fail2ban jail on the Caddy access log (or Forgejo auth log) for repeated 401s, prioritising auth/signup/API endpoints. Effort S–M. (Related to edge rate limiting #48.)

**Severity: MEDIUM** — pre-onboarding infra audit (2026-07-19). `ansible/roles/base/templates/jail.local.j2` has only an `[sshd]` jail. HTTP Basic auth stays enabled (`ENABLE_INTERNAL_SIGNIN=false` only removes the *web form*), so git-over-HTTPS and API token auth can be brute-forced with **no host-layer lockout** — nothing tails Caddy/Forgejo logs for auth abuse. ### Ask Add a fail2ban jail on the Caddy access log (or Forgejo auth log) for repeated 401s, prioritising auth/signup/API endpoints. Effort S–M. (Related to edge rate limiting #48.)

supernaut lade till etiketterna

2026-07-19 06:59:36 +00:00

supernaut lade till detta till projektet Bitborg Infrastructure

2026-07-19 07:04:44 +00:00

supernaut refererade till detta ärende från en incheckning

2026-07-20 15:16:43 +00:00

feat(base): fail2ban jail for git-over-HTTPS / API 401 brute force (#127)

supernaut hänvisade till detta ärende från en ändringsförfrågan som kommer att stänga det

2026-07-20 15:18:11 +00:00

fail2ban jail for git-over-HTTPS / API 401 brute force #175

supernaut refererade till detta ärende från en incheckning

2026-07-20 18:07:36 +00:00

feat(base): fail2ban jail for git-over-HTTPS / API 401 brute force (#127)

supernaut stängde ärendet

2026-07-20 18:19:33 +00:00

supernaut refererade till detta ärende från en incheckning

2026-07-20 18:19:35 +00:00

fail2ban jail for git-over-HTTPS / API 401 brute force (#175)

supernaut refererade till detta ärende från en incheckning

2026-07-20 21:16:36 +00:00

fix(fail2ban): don't jail registry /v2 401 auth handshakes (#127)

supernaut refererade till detta ärende

2026-07-20 21:18:11 +00:00

don't jail registry /v2 401 auth handshakes — unblocks CI push (#127) #179

supernaut refererade till detta ärende från en incheckning

2026-07-20 21:20:57 +00:00

don't jail registry /v2 401 auth handshakes — unblocks CI push (#127) (#179)

supernaut kommenterad

2026-07-21 06:49:55 +00:00

Upphovsperson

Ägare

Follow-up: the jail (shipped in #175) caused a CI regression — it banned the ephemeral runner egress on the OCI registry /v2 401 auth handshake during image push (the registry 401s to advertise its token endpoint), which hung/failed the bitborg-web deploys (runs #165/#168) and blocked runners from even connecting.

Fixed in PR #179: added ignoreregex for 401s under /v2/ so registry handshakes are not counted, while real auth brute force (/user/login, /api, git-HTTPS Basic) is still jailed (verified against the live log + synthetic tests). Also had to fail2ban-client unban the stale runner-egress IP (158.174.210.225): fail2ban re-applies DB-stored bans across a restart within bantime, so the pre-fix ban survived the #179 reload and kept blocking runners until explicitly cleared.

Verified 2026-07-21: caddy-auth jailing real 401s, ignoring /v2; runner egress unbanned; CI run 173 + deploy run 174 both green with no re-ban. Learning: fail2ban bans persist across restart — clearing a bad ban needs an explicit unban, not just a filter fix + reload.

Follow-up: the jail (shipped in #175) caused a **CI regression** — it banned the ephemeral runner egress on the OCI registry `/v2` **401 auth handshake** during image push (the registry 401s to advertise its token endpoint), which hung/failed the bitborg-web deploys (runs #165/#168) and blocked runners from even connecting. Fixed in **PR #179**: added `ignoreregex` for 401s under `/v2/` so registry handshakes are not counted, while real auth brute force (`/user/login`, `/api`, git-HTTPS Basic) is still jailed (verified against the live log + synthetic tests). Also had to **`fail2ban-client unban` the stale runner-egress IP** (158.174.210.225): fail2ban re-applies DB-stored bans across a restart within `bantime`, so the pre-fix ban survived the `#179` reload and kept blocking runners until explicitly cleared. Verified 2026-07-21: caddy-auth jailing real 401s, ignoring `/v2`; runner egress unbanned; CI run 173 + deploy run 174 both green with no re-ban. **Learning:** fail2ban bans persist across restart — clearing a bad ban needs an explicit `unban`, not just a filter fix + reload.

supernaut refererade till detta ärende

2026-07-21 06:59:25 +00:00

Edge rate limiting: custom Caddy build + rate_limit block #48

supernaut refererade till detta ärende från en incheckning

2026-08-03 09:41:34 +00:00

fail2ban jail for git-over-HTTPS / API 401 brute force (#175)

supernaut refererade till detta ärende från en incheckning

2026-08-03 09:41:34 +00:00

don't jail registry /v2 401 auth handshakes — unblocks CI push (#127) (#179)

Logga in för att delta i denna konversation.