research: two runner tiers, fast containers and isolated VMs #505

Nytt ärende

Öppen

öppnade 2026-09-30 11:38:07 +00:00 av supernaut · 0 kommentarer

supernaut kommenterad

2026-09-30 11:38:07 +00:00

Ägare

Question

Can Actions runners come in two tiers?

Basic: jobs run in containers on a long-lived VM. Fast pickup, since no VM boots per job. Weaker isolation.
Secure: one ephemeral VM per job, as today (ADR 0021). Slower pickup, strong isolation.

Why look at it

Pickup today includes a full VM boot per job. A container tier would cut that for users who accept the trade-off.

ADR 0021 rejected container-only isolation for untrusted jobs: containers share the host kernel, so they are a resource boundary and not a security boundary against hostile code. A basic tier has to answer that, not ignore it.

To research

Isolation options for the basic tier. Rootless podman, a user-space kernel such as gVisor, or micro-VMs such as Kata Containers or Firecracker. Measure the startup cost against today's VM boot.
Tenant separation. Can jobs from different tenants share a host at all, or does each tenant or org need its own host? What leaks through a shared host: cache, network, timing?
Image builds. Basic-tier jobs probably cannot build container images safely. Decide whether the basic tier refuses them or routes them to the secure tier.
Selection. How a workflow picks a tier (runs-on labels), and what the default is.
Controller changes. What the runner controller needs: a warm host pool, scaling, and reaping.
Numbers. Measured pickup time and cost per job for each tier against today.
Security review. Threat model per tier, and what we tell users about each.

Done when

A design note under docs/design/ answers the points above with measurements, recommends go or no-go, and, on go, drafts an ADR that amends ADR 0021.

Related: ADR 0018, ADR 0021, ADR 0033.

Epic: bitborg-docs#1

## Question Can Actions runners come in two tiers? - **Basic:** jobs run in containers on a long-lived VM. Fast pickup, since no VM boots per job. Weaker isolation. - **Secure:** one ephemeral VM per job, as today (ADR 0021). Slower pickup, strong isolation. ## Why look at it Pickup today includes a full VM boot per job. A container tier would cut that for users who accept the trade-off. ADR 0021 rejected container-only isolation for untrusted jobs: containers share the host kernel, so they are a resource boundary and not a security boundary against hostile code. A basic tier has to answer that, not ignore it. ## To research 1. **Isolation options for the basic tier.** Rootless podman, a user-space kernel such as gVisor, or micro-VMs such as Kata Containers or Firecracker. Measure the startup cost against today's VM boot. 2. **Tenant separation.** Can jobs from different tenants share a host at all, or does each tenant or org need its own host? What leaks through a shared host: cache, network, timing? 3. **Image builds.** Basic-tier jobs probably cannot build container images safely. Decide whether the basic tier refuses them or routes them to the secure tier. 4. **Selection.** How a workflow picks a tier (`runs-on` labels), and what the default is. 5. **Controller changes.** What the runner controller needs: a warm host pool, scaling, and reaping. 6. **Numbers.** Measured pickup time and cost per job for each tier against today. 7. **Security review.** Threat model per tier, and what we tell users about each. ## Done when A design note under `docs/design/` answers the points above with measurements, recommends go or no-go, and, on go, drafts an ADR that amends ADR 0021. Related: ADR 0018, ADR 0021, ADR 0033. Epic: bitborg-docs#1

supernaut lade till etiketterna

2026-09-30 11:45:22 +00:00

supernaut lade till detta till projektet Bitborg Infrastructure

2026-09-30 11:45:35 +00:00

supernaut refererade till detta ärende från bitborg/bitborg-docs

2026-10-02 23:04:25 +00:00

Dedicated runners & CI autoscale #1