Skip to content

CONCEPT Cited by 1 source

In-band signaling handshake

In-band signaling is the exchange of configuration / capability / policy information between a client and a server via the same RPC channel they use for request traffic — typically through a periodic handshake + per-request capability flags — rather than via static configuration files or out-of-band config services. Canonical shape from Netflix's KV DAL:

  • Server → client (on handshake, periodically): target + max latency SLOs, so the client can adjust timeouts and hedging policy dynamically.
  • Client → server (per-request): capability flags — compression support, chunking support, other features.

The Netflix anchor

"KV uses in-band messaging we call signaling that allows the dynamic configuration of the client and enables it to communicate its capabilities to the server. ... Without signaling, the client would need static configuration — requiring a redeployment for each change — or, with dynamic configuration, would require coordination with the client team."

"For server-side signals, when the client is initialized, it sends a handshake to the server. The server responds back with signals, such as target or max latency SLOs, allowing the client to dynamically adjust timeouts and hedging policies. Handshakes are then made periodically in the background to keep the configuration current. For client-communicated signals, the client, along with each request, communicates its capabilities, such as whether it can handle compression, chunking, and other features." (Source: sources/2024-09-19-netflix-netflixs-key-value-data-abstraction-layer)

Why in-band beats static or out-of-band

Three failure modes of the alternatives:

  1. Static config — every change requires a client redeploy. Rolling behavior / SLO adjustments become release-cycle bound.
  2. Out-of-band dynamic config — the client team has to be looped in for each config-service schema change; coordination tax scales with number of consumers.
  3. Per-request config — fine-grained but expensive (config payload on every request); harder to cache on the client side.

In-band signaling with a periodic handshake is a middle ground: the config lives in the RPC protocol, evolves without coordination beyond the protocol definition, and refresh cadence is tuned independently of request QPS.

What the client actually does with the signals

Once the server has advertised target_latency and max_latency SLOs, the client can:

  • Set its RPC timeout to max_latency (rather than a hard-coded value).
  • Fire a hedged retry at target_latency if the first attempt hasn't returned — a front-line tail-latency primitive (see concepts/tail-latency-at-scale).
  • Back off new requests if the server signals congestion (the protocol doesn't limit to just latency targets — any server- pushed operational signal is fair game).

Client → server capability signaling is simpler: the server decides whether to serve compressed responses / chunked reads based on per-request flags, with no ambiguity.

Trade-offs

  • Protocol surface grows. Every new signal is a new backward-compatibility concern; a versioned handshake schema is a must.
  • Stale signals. If the periodic refresh cadence is long and server-side SLOs change, clients operate on stale policy for a while. Tuning this cadence is an operational choice.
  • Partial adoption. Different clients in a fleet may support different signals — the server has to tolerate older + newer clients in parallel.
  • Still needs out-of-band for some things. Security credentials, routing topology, tenant auth — those typically don't live in data-path signaling.

Sibling concepts

  • gRPC deadlines — canonical in-band per-request timeout signal; the DAL's periodic SLO handshake composes with it (the handshake sets the default deadline; each request can override).
  • SlowStart / congestion signals — TCP-level signaling that's the network-stack analog of this application-level primitive.
  • HTTP Accept-Encoding: gzip — the oldest widespread capability-signal-per-request.

Seen in

Last updated · 319 distilled / 1,201 read