Skip to content

SYSTEM Cited by 1 source

bazel-remote

bazel-remote is an OSS implementation of the Bazel remote cache gRPC protocol, typically deployed as a local sidecar on each CI worker with a shared object-store (S3, GCS, etc.) as the global storage backend. Upstream: github.com/buchgr/bazel-remote.

Role

  • Implements the gRPC Remote Execution / Cache API that Bazel (and Buck, Pants) speaks natively.
  • Acts as a cache tier: incoming action-cache / content-addressable-store (CAS) requests from the Bazel client go to the local bazel-remote daemon; bazel-remote fronts the global backing store.
  • Local gRPC = fast handshake + fast lookup; global S3 backend = shared cache across all CI workers.

Canva's deployment

From the retrospective:

At Canva, we've long had a Bazel cache shared across all CI workers. It's a simple setup but amazingly effective. It's a service called bazel-remote, installed on every instance, backed by an S3 bucket as its storage. So, the gRPC communication between the Bazel server and the cache happens locally, but it's supported by global shared cache storage.

Properties:

  • On every instance. Pattern is sidecar, not shared-service — no single point of failure on the read path.
  • S3 as shared backend. bazel-remote fronts the S3 API, Canva's systems/aws-s3 provides durable global storage.
  • Same API surface as RBE. Because the Bazel protocol is the same, bazel-remote is the natural upgrade path to full Remote Build Execution.

Composes with

  • patterns/build-without-the-bytes — BwoB's --remote-download-minimal is a client-side flag that makes full use of a bazel-remote- style CAS: Bazel records digests from the cache without fetching payloads.
  • concepts/remote-build-execution — bazel-remote is the cache wire of RBE; adding workers on top moves to full RBE.

Why "local daemon + global storage" works

  • Handshake costs stay local. gRPC setup and many roundtrips happen on the same host.
  • Cache hits avoid public-internet bottlenecks. S3 is inside the VPC for Canva-scale CI; latency to bazel-remote in the local host is still lower.
  • Failures degrade well. Daemon restart → re-handshake; backend outage → cache miss, Bazel rebuilds locally, nothing fails-closed.

Cache-eviction caveat

Bounded storage means evictions. patterns/build-without-the-bytes amplifies the eviction hazard: if the client deferred fetching an artifact until a downstream action needs it, and that artifact was evicted in the meantime, the build fails. Canva's mitigation: retry-on-cache-check-failure (Bazel issue #10880).

Seen in

Last updated · 200 distilled / 1,178 read