Faster Rust builds: shared builders and smarter caching
Plan and findings · status as of 27 October 2026
In short. Our Rust services are rebuilt from scratch on every CI run, and developers build on machines of very different strength. The plan: send both developer and CI builds to a small fleet of strong "builder" machines, route each build to the machine that already has that project's cache warm, and fix a few cache settings that currently throw work away. Measured on real projects, most CI releases could drop from tens of minutes to a few minutes, and everyday rebuilds get several times faster, whichever cache tool we use.
1. The goal
- One way to build. Developer builds and CI builds go through the same path onto the same fleet of builder machines.
- Send each build to a warm cache. Each builder keeps its cache on its own fast disk. Builds go to the machine that recently built the same project, so only the changed parts recompile. No shared network cache.
- A personal "preferred builder". A developer can name their own strong machine as first choice, with one simple rule for spilling over to the shared pool when it gets busy.
- Build less. Skip work whose inputs did not change.
- Keep the "rolling release" model. Projects deliberately track the latest version of each other, so breakage shows up fast. Caching has to work within that.
2. Where we are today
14–46 minmedian CI release run, per repo (sample of 9 repos)
~720 hCI runner time per month, release workflow, those repos only
15%of that is nightly rebuilds of commits that had already built
7–26%of commits change no code at all, yet trigger a full release build
- CI starts cold every time. Each run happens in a fresh container with no cache, and compiles everything 3–4 times (lint check, tests, then two release targets).
- The builder service exists and already supports remote builds, sending uncommitted code, and reporting its load and which projects it has warm. But it puts every project in one build directory, which makes parallel builds wait on each other.
- Routing exists in a basic form, but ignores how recently a machine built the project and how long its queue is.
- The compiler cache (sccache) is set up in a way that defeats it: one environment variable makes every build directory look different, so nothing is shared between them, and incremental compilation is switched off.
3. What is on the table
Benchmarked on four representative projects on one 28-core machine, plus 30 days of CI history. The first two charts are gains we get whichever cache tool we use.
Other measured gains
| Change | Effect |
|---|---|
| Pass the build directory as a flag instead of an environment variable | Cache reuse across build directories: 0% → 82–100% |
| Only run the dependency update when an upstream actually moved | 3–14 s saved on most builds, often more than the edit itself costs |
| Skip CI runs where neither code nor dependency versions changed | 7–26% of runs skipped outright; most of the 15% nightly rebuild time |
| Macro caching, test-result caching | Small (under 1 s per crate; lib tests 2–42 s): not worth special machinery now |
Cache tool: sccache vs Mr. Boxington (mbx)
mbx is a new Rust build cache aimed at many agents and worktrees on one machine. Same code, same dependency versions, each tool with its own empty cache, sccache with the fixes above.
| sccache | mbx | |
|---|---|---|
| Fresh build directory, warm cache | 50–95 s | 1.6–11 s |
| Three agents start fresh worktrees at once | 105–255 s | 10–80 s |
| Cold build | about equal (mbx 5–30% faster) | |
| Normal edit-and-rebuild loop | the same | |
| First edit of a crate in a checkout | normal | +17–26 s, once |
| Caches build scripts, C code and final links | no | yes |
| Cache size per project | 1–3 GB | 5–11 GB |
| Maturity | ~9 years, widely used | ~2 months, mostly one author, fast-changing |
Caveat: about 11 of our 63 build scripts write generated files into the source tree rather than Cargo's official output folder. mbx caches build scripts, so with its defaults those projects fail to build in a fresh checkout; with that one feature switched off they work, and the gains above still hold.
4. The design
Developer or CI job→
local code service picks a builder→
builder with the warmest cache→
binaries come back / get published
Choosing a builder
- The local code service keeps a background snapshot of every builder (load, queue, which projects it built and when), so choosing takes milliseconds.
- Pick the machine expected to finish first: its queue wait plus a warm or cold build time learned from history. A warm idle machine wins; a warm machine with a long queue can lose to an idle cold one.
- Projects get a stable "home" machine through a simple hashing rule every client computes the same way, so they settle across the fleet without a central coordinator. Busy projects spill to a second home.
- Your own preferred builder comes first until its queue passes a number you choose.
On each builder
- One build directory per project, so different projects build in parallel.
- Admit builds by available memory, not a fixed count.
- Evict whole projects, least recently used first, when disk runs low.
Dependencies (rolling release)
- The code service already checks repositories against their remotes on a schedule. Extend it to keep a cache of upstream branch heads.
- Before a build, compare against that cache: if nothing moved, skip the update step; if something moved, update only that dependency. Releases always check live.
CI
- CI runners live on the builder machines and become thin clients that hand the build to the warmest builder.
- Skip runs where neither the code nor the dependency versions changed.
- Lint, tests and release share one warm build directory.
5. Steps and status
| # | Step | Status |
|---|---|---|
| 0 | Research current builder, build tool and CI; measure four projects; mine CI and commit history | done |
| 0b | Evaluate and benchmark mbx against sccache | done |
| 1 | Quick fixes: build directory as a flag, incremental on for dev builds, one directory per project on builders, memory-based admission | to do |
| 1b | Upstream-head cache; run the dependency update only when something moved | to do |
| 1c | Make the 11 build scripts write to the official output folder; adopt mbx on builders (pinned version) | proposed |
| 2 | Web UI builds: fast debug mode, skip when unchanged | to do |
| 3 | Smarter routing: snapshot, earliest finish, home machines, preferred builder | to do |
| 4 | CI on the builder fleet; trial on one core service first | to do, needs agreement |
| 5 | Shared upstream view across the fleet; push notifications to keep it fresh | later |
| 6 | Optional: cache test results; let CI reuse binaries a developer already built | later |
6. Risks
- risk Changing the agreed CI plan. The current approved plan builds CI inside each fresh CI container, which stays cold. Routing CI to the shared fleet changes that and needs the plan owner's agreement.
- risk mbx maturity. Young, mostly one author, frequent releases that discard cached state. Mitigation: pin the version, keep sccache as fallback, keep the cache layer swappable.
- risk Build scripts that write into the source tree break under build-script caching. Mitigation: fix them (they break Cargo's rules anyway) or keep that feature off.
- risk Disk. A warm project takes 4–40 GB per machine, so no builder can keep everything warm; routing to home machines is what makes this work.
- risk Rolling dependencies. A day of upstream changes still costs a large rebuild; caching avoids paying it twice, not paying it at all.
7. Open questions
- How often to refresh registry (crates.io) dependencies: daily or hourly?
- How stale a cached upstream head may be before checking live (suggestion: 5 minutes)?
- Should CI route to the fleet instead of building inside each CI container?
- Build the ARM target on every development push, or nightly / on demand?
- Keep full nightly release builds, or switch to a warm check-and-test when only dependencies moved?
- Which machines form the fleet, and who uses which as a preferred builder?
- Adopt mbx on builders now (with build-script caching off), then on developer machines?
8. Done vs pending
Done
- Mapped the builder, build tool, code service and CI
- Benchmarks on four projects; CI and commit history analysis
- Dependency-refresh cost model
- mbx evaluation and benchmark
- Found one project currently broken by an upstream change, and the build-script issue
Pending
- All implementation (steps 1–6)
- Decisions on the open questions
- Agreement on the CI change
Numbers are single runs on one machine and should be read as rough. CI durations include queue time. The "warm builder" CI figure is an estimate: two release targets rebuilt after a day of upstream changes, excluding queueing. Contains no credentials or internal addresses.