zoekt index storage: per-owner accounting now; tier gating blocked on upstream (quota can't count it) #162

Nytt ärende

Stängd

öppnade 2026-07-19 22:50:09 +00:00 av supernaut · 1 kommentar

supernaut kommenterad

2026-07-19 22:50:09 +00:00

Ägare

Investigated whether Zoekt index storage (/var/lib/gitea/indexers/repos.zoekt, up to ~6× the repo footprint — #76) can be charged to user quota, and whether code indexing can be made opt-in / a tier entitlement.

Findings (Forgejo v16.0, verified in source)

1. Zoekt bytes cannot be counted in Forgejo's quota engine. The full v16 quota-subject list (models/quota/limit_subject.go: size:all, size:repos:*, size:git:all, size:git:lfs, size:assets:*) has no indexer subject, and usage is computed purely from DB columns (models/quota/used.go: repository.git_size, lfs_size, attachment/artifact/package sizes) — indexer disk is never measured. There is no quota rule we can write for it.

2. Per-repo attribution out-of-band is easy. Shard files are named by numeric repo ID (modules/indexer/code/zoekt/zoekt.go: zoekt repo name = repo.ID, deletes match the <repoID>_v prefix, ShardMax 512 MiB). So du over <repoID>_v* prefixes + repo-ID→owner mapping gives exact per-owner index footprints.

3. Opt-in / per-tier indexing is NOT possible in v16.

REPO_INDEXER_INCLUDE/EXCLUDE glob file paths inside repos, not owners/repos (modules/setting/indexer.go) — a reconciler-generated <user>/** pattern is a dead end.
REPO_INDEXER_REPO_TYPES filters only by category (sources/forks/mirrors/templates).
No per-repo unit/setting, no admin API for per-repo indexing.
Upstream: issues forgejo#157 and forgejo#7511 are open; PR forgejo#8106 ("per-repo option to disable code indexer", is_code_indexer_enabled column) is open/unmerged and targets post-16.0. When it lands (and if API-exposed), the reconciler can default-deny and enable per tier — the ADR 0018 actions-unit pattern.
Deleting shards out-of-band is not an exclusion mechanism (index state lives in repo_indexer_status; shards regenerate on next push).

Plan

Observability now: cron/textfile-collector on the services host sizing indexers/repos.zoekt/<repoID>_v* per owner → node_exporter metric (e.g. gitborg_zoekt_bytes{owner=…}) + Grafana alert on total indexers/ growth and per-owner outliers (N× tier repo quota). Read-only, ~half a day.
Follow-up trigger: revisit tiered/opt-in indexing when Forgejo merges PR 8106 (watch the v16.x release notes); check the toggle is exposed via the repo-edit API. No forked patches (plain-upstream-image principle).

Refs: #76 (zoekt rollout), ADRs 0014–0017 (tiers/quota via reconciler).
Sources: https://codeberg.org/forgejo/forgejo/raw/branch/v16.0/forgejo/models/quota/limit_subject.go · …/models/quota/used.go · …/modules/indexer/code/zoekt/zoekt.go · …/modules/setting/indexer.go · https://codeberg.org/forgejo/forgejo/pulls/8106 · https://forgejo.org/docs/latest/admin/quota/

Investigated whether Zoekt index storage (`/var/lib/gitea/indexers/repos.zoekt`, up to ~6× the repo footprint — #76) can be charged to user quota, and whether code indexing can be made opt-in / a tier entitlement. ## Findings (Forgejo v16.0, verified in source) **1. Zoekt bytes cannot be counted in Forgejo's quota engine.** The full v16 quota-subject list (`models/quota/limit_subject.go`: `size:all`, `size:repos:*`, `size:git:all`, `size:git:lfs`, `size:assets:*`) has no indexer subject, and usage is computed purely from DB columns (`models/quota/used.go`: `repository.git_size`, `lfs_size`, attachment/artifact/package sizes) — indexer disk is never measured. There is no quota rule we can write for it. **2. Per-repo attribution out-of-band is easy.** Shard files are named by numeric repo ID (`modules/indexer/code/zoekt/zoekt.go`: zoekt repo name = `repo.ID`, deletes match the `<repoID>_v` prefix, `ShardMax` 512 MiB). So `du` over `<repoID>_v*` prefixes + repo-ID→owner mapping gives exact per-owner index footprints. **3. Opt-in / per-tier indexing is NOT possible in v16.** - `REPO_INDEXER_INCLUDE`/`EXCLUDE` glob **file paths inside repos**, not owners/repos (`modules/setting/indexer.go`) — a reconciler-generated `<user>/**` pattern is a dead end. - `REPO_INDEXER_REPO_TYPES` filters only by category (sources/forks/mirrors/templates). - No per-repo unit/setting, no admin API for per-repo indexing. - Upstream: issues forgejo#157 and forgejo#7511 are open; **PR forgejo#8106** ("per-repo option to disable code indexer", `is_code_indexer_enabled` column) is open/unmerged and targets post-16.0. When it lands (and if API-exposed), the reconciler can default-deny and enable per tier — the ADR 0018 actions-unit pattern. - Deleting shards out-of-band is not an exclusion mechanism (index state lives in `repo_indexer_status`; shards regenerate on next push). ## Plan - [ ] **Observability now:** cron/textfile-collector on the services host sizing `indexers/repos.zoekt/<repoID>_v*` per owner → node_exporter metric (e.g. `gitborg_zoekt_bytes{owner=…}`) + Grafana alert on total `indexers/` growth and per-owner outliers (N× tier repo quota). Read-only, ~half a day. - [ ] **Follow-up trigger:** revisit tiered/opt-in indexing when Forgejo merges PR 8106 (watch the v16.x release notes); check the toggle is exposed via the repo-edit API. No forked patches (plain-upstream-image principle). Refs: #76 (zoekt rollout), ADRs 0014–0017 (tiers/quota via reconciler). Sources: https://codeberg.org/forgejo/forgejo/raw/branch/v16.0/forgejo/models/quota/limit_subject.go · …/models/quota/used.go · …/modules/indexer/code/zoekt/zoekt.go · …/modules/setting/indexer.go · https://codeberg.org/forgejo/forgejo/pulls/8106 · https://forgejo.org/docs/latest/admin/quota/

supernaut refererade till detta ärende från en incheckning

2026-07-20 08:42:41 +00:00

feat(forgejo): disable zoekt code search + delete indexes (#162)

supernaut hänvisade till detta ärende från en ändringsförfrågan som kommer att stänga det

2026-07-20 08:43:01 +00:00

disable zoekt code search + delete indexes (#162) #165

supernaut refererade till detta ärende från en incheckning

2026-07-20 08:52:01 +00:00

feat(forgejo): disable zoekt code search + delete indexes (#162)

supernaut stängde ärendet

2026-07-20 08:57:35 +00:00

supernaut refererade till detta ärende från en incheckning

2026-07-20 08:57:36 +00:00

disable zoekt code search + delete indexes (#162) (#165)

supernaut kommenterad

2026-07-20 09:19:30 +00:00

Upphovsperson

Ägare

Applied to prod 2026-07-20 (PR #165). Verified in the running container:

REPO_INDEXER_ENABLED = false
ls: /data/gitea/indexers: No such file or directory  →  INDEX_DIR_GONE

Forgejo restarted cleanly; health gate green (postgres/kanidm/forgejo/web/caddy active, all public endpoints serving). Zoekt is off and the shards are reclaimed. Closing — re-enable tracked by the "Forgejo PR 8106" trigger noted in the runbook.

**Applied to prod 2026-07-20** (PR #165). Verified in the running container: ``` REPO_INDEXER_ENABLED = false ls: /data/gitea/indexers: No such file or directory → INDEX_DIR_GONE ``` Forgejo restarted cleanly; health gate green (postgres/kanidm/forgejo/web/caddy active, all public endpoints serving). Zoekt is off and the shards are reclaimed. Closing — re-enable tracked by the "Forgejo PR 8106" trigger noted in the runbook.

supernaut lade till detta till projektet Bitborg Infrastructure

2026-07-20 09:31:12 +00:00

supernaut refererade till detta ärende

2026-07-21 12:18:19 +00:00

Code search: some queries 500 (Forgejo explore.Code panic, index out of range) #113

supernaut refererade till detta ärende

2026-07-21 19:09:17 +00:00

Exclude the zoekt index from backups (derived, regenerable data) #189

supernaut refererade till detta ärende från en incheckning

2026-08-03 09:41:34 +00:00

disable zoekt code search + delete indexes (#162) (#165)

supernaut lade till etiketten