fix(caddy): socket-activate 80/443 so real client IPs reach caddy (#81)

supernaut kommenterad

2026-07-18 07:17:35 +00:00

Ägare

Fixes #81 — rootless Podman's pasta port forwarding rewrote every inbound source to the container network, so Caddy logged its own 10.89.0.15 for all external requests and forwarded it in X-Forwarded-For.

Approach: systemd socket activation (issue's candidate 1)

The systemd user manager binds 80/443 on the host (caddy.socket) and passes them into the container as inherited fds — pasta never touches inbound connections, so source addresses survive. Bonus: the sockets stay open across Caddy service restarts, so config deploys no longer drop connections; and Caddy now needs zero capabilities (NET_BIND_SERVICE dropped along with PublishPort).

caddy.socket (new template): 443/tcp, 443/udp (HTTP/3), 80/tcp; dual-stack via BindIPv6Only=both; fd order documented in the unit and mirrored by the Caddyfile (socket_bind) snippet — fd/3 h1+h2, fdgram/4 h3, fd/5 http.
caddy.container: Requires=caddy.socket + After= so fds are passed even on a direct service start (boot, deploys).
Caddyfile: each site imports (socket_bind); auto_https disable_redirects + explicit http:// server on fd/5 doing redir https://{host}{uri} permanent. ACME HTTP-01 is still solved (Caddy intercepts challenges on every server); TLS-ALPN-01 on fd/3 is the fallback.
tasks: install/enable the socket; migration choreography (stop service → rebind socket → start service) because on the migration apply the old published-port container still holds 80/443.
runbook: v16-specifics entry + verification steps.

Verified in the local podman machine (podman 5.8.3, systemd 259, caddy 2.11.4-alpine — prod tag)

Socket-activated Caddy logs the real source (127.0.0.1 from loopback curl); a published-port control container logs the bridge address (10.88.0.7) — the exact #81 symptom.
fds survive systemctl --user restart of the service (important: the #63 fix restarts Caddy on unit changes).
Full prod fd layout tested: two TLS hosts on fd/3 (HTTP/2 confirmed), http→https 301 from fd/5 ({host} strips the port).
The rendered production Caddyfile (real group_vars) passes caddy validate on the prod image.

Apply-day notes

Merge order: this PR was stacked on the #63 PR (now merged, #93); rebased onto main.
The migration apply briefly stops Caddy (seconds) while the sockets rebind — inherent to taking over ports pasta held.
Verify after apply (runbook § v16 specifics): systemctl --user status caddy.socket active; fresh external request shows a public remote_ip in ~gitborg/caddy/logs/access.log; Forgejo logs the real IP end-to-end.
h3/QUIC path (fdgram/4) not curl-testable in the VM — verify HTTP/3 works after apply (browser devtools or curl --http3 if available); if it misbehaves, dropping the fdgram/4 bind + ListenDatagram line falls back to h1/h2 only. Tracked in #101.

Unblocks / follow-ups

Unblocks #48 (edge rate limiting) — previously all external traffic looked like one client.
Forgejo's built-in SSH (port 22) is still pasta-published, so SSH client IPs are still rewritten — tracked in #91 (same socket-activation / PROXY-protocol options).
The monitoring VM's Caddy (grafana/stats/ntfy) has the same pasta rewrite in its access logs — tracked in #100.
Post-apply HTTP/3 verification over the inherited datagram fd — tracked in #101.

Fixes #81 — rootless Podman's pasta port forwarding rewrote every inbound source to the container network, so Caddy logged its own `10.89.0.15` for all external requests and forwarded it in `X-Forwarded-For`. ## Approach: systemd socket activation (issue's candidate 1) The systemd user manager binds 80/443 on the host (`caddy.socket`) and passes them into the container as inherited fds — pasta never touches inbound connections, so source addresses survive. Bonus: the sockets stay open across Caddy service restarts, so config deploys no longer drop connections; and Caddy now needs **zero** capabilities (NET_BIND_SERVICE dropped along with PublishPort). - `caddy.socket` (new template): 443/tcp, 443/udp (HTTP/3), 80/tcp; dual-stack via `BindIPv6Only=both`; fd order documented in the unit and mirrored by the Caddyfile `(socket_bind)` snippet — fd/3 h1+h2, fdgram/4 h3, fd/5 http. - `caddy.container`: `Requires=caddy.socket` + `After=` so fds are passed even on a direct service start (boot, deploys). - `Caddyfile`: each site imports `(socket_bind)`; `auto_https disable_redirects` + explicit `http://` server on fd/5 doing `redir https://{host}{uri} permanent`. ACME HTTP-01 is still solved (Caddy intercepts challenges on every server); TLS-ALPN-01 on fd/3 is the fallback. - tasks: install/enable the socket; migration choreography (stop service → rebind socket → start service) because on the migration apply the old published-port container still holds 80/443. - runbook: v16-specifics entry + verification steps. ## Verified in the local podman machine (podman 5.8.3, systemd 259, caddy 2.11.4-alpine — prod tag) - Socket-activated Caddy logs the **real** source (`127.0.0.1` from loopback curl); a published-port control container logs the bridge address (`10.88.0.7`) — the exact #81 symptom. - fds survive `systemctl --user restart` of the service (important: the #63 fix restarts Caddy on unit changes). - Full prod fd layout tested: two TLS hosts on fd/3 (HTTP/2 confirmed), http→https 301 from fd/5 (`{host}` strips the port). - The **rendered production Caddyfile** (real group_vars) passes `caddy validate` on the prod image. ## Apply-day notes - Merge order: this PR was stacked on the #63 PR (now merged, #93); rebased onto main. - The migration apply briefly stops Caddy (seconds) while the sockets rebind — inherent to taking over ports pasta held. - Verify after apply (runbook § v16 specifics): `systemctl --user status caddy.socket` active; fresh external request shows a public `remote_ip` in `~gitborg/caddy/logs/access.log`; Forgejo logs the real IP end-to-end. - h3/QUIC path (fdgram/4) not curl-testable in the VM — verify HTTP/3 works after apply (browser devtools or `curl --http3` if available); if it misbehaves, dropping the `fdgram/4` bind + `ListenDatagram` line falls back to h1/h2 only. Tracked in #101. ## Unblocks / follow-ups - Unblocks #48 (edge rate limiting) — previously all external traffic looked like one client. - Forgejo's built-in SSH (port 22) is still pasta-published, so **SSH client IPs are still rewritten** — tracked in #91 (same socket-activation / PROXY-protocol options). - The monitoring VM's Caddy (grafana/stats/ntfy) has the same pasta rewrite in its access logs — tracked in #100. - Post-apply HTTP/3 verification over the inherited datagram fd — tracked in #101.

supernaut lade till 1 incheckning

2026-07-18 07:17:35 +00:00

fix(quadlet): pull pinned images + restart on unit change (#63 )

ci / ci (pull_request) Successful in 1m40s

Detaljer

48b3f5731c

postgres/forgejo/caddy roles now:

- pull the pinned image explicitly before (re)starting, so a tag bump
  actually has the new image on the host (previously nothing pulled it)
- restart the container when its .container unit changed (monitoring
  role's pattern); forgejo folds daemon-reload into the Restart handler
  so the restart can never use a stale generated unit under --tags
- verify post-start that the running container is on the pinned tag,
  failing the play instead of green-lighting a silent no-op

Closes #63

supernaut tvångsskickade fix/81-caddy-socket-activation från b1df995766

ci / ci (pull_request) Successful in 1m32s

Detaljer

till 66d811a2b0

ci / ci (pull_request) Successful in 2m9s

Detaljer

2026-07-18 12:59:20 +00:00

Jämför

supernaut refererade till denna ändringsförfrågan

2026-07-18 14:28:09 +00:00

Monitoring VM Caddy also loses real client IPs to pasta (grafana/stats/ntfy) #100

supernaut refererade till denna ändringsförfrågan