PATTERN Cited by 1 source

Async consistency-checker sidecar¶

Definition¶

Async consistency-checker sidecar is the mechanism used by Zalando's Returns parallel-run implementation to keep the comparison work off the client-facing hot path: the authoritative system (monolith) responds to the client first, then fires an async POST to a /consistency-checks endpoint on the new service, which immediately returns HTTP 202 Accepted and processes the comparison in the background.

Shape¶

Client ──(1) request──> Monolith ──(2) response──> Client
                            │
                            └──(3) async POST /consistency-checks──> Returns μs
                                                                       │
                                                                       └──(4) 202 Accepted
                                                                       │
                                     (5-6-7) re-issue real endpoint against localhost
                                                                       │
                                                                       └──(8) compare responses
                                                                                │
                                                                                └──(9) emit Matched/Unmatched/Failed

(1-2) Client receives the monolith's response with no added latency from the comparison path.
(3-4) Monolith's work is a fire-and-forget POST plus the receipt of a 202 ack. No parsing of a comparison result.
(5-7) New service re-issues the same request path against itself over localhost to get its own response.
(8-9) Compare along three axes (status, headers, body) per concepts/response-comparison-headers-body-status; emit Matched / Unmatched / Failed counters to Prometheus displayed in Grafana per operation_id.

Payload schema¶

The async POST body carries the full original request + the monolith's response:

{
  "request": {
    "url": { "path": "api/example?param=something" },
    "headers": {
      "Content-Type": "application/json;charset=UTF-8",
      "Accept-Language": "de-DE"
    },
    "method": "GET",
    "body": null
  },
  "response": {
    "status": 200,
    "headers": {
      "Content-Type": "application/json;charset=UTF-8",
      "transfer-encoding": "chunked"
    },
    "body": "json-response-body"
  }
}

(Source: sources/2021-11-03-zalando-parallel-run-pattern-a-migration-technique-in-microservices)

Why async¶

Client latency unaffected. Monolith is on the hot path; comparison runs later.
Monolith resources free up. No waiting on comparison → threads / connections return to pool immediately after the async POST.
Comparison can be slower than request handling without impact — PDF comparison, large JSON diff, header-set normalisation can all take time.
Decouples comparison failures from client-facing reliability. A bug in the consistency-checker doesn't crash the client path.

Trade-offs¶

Request doubling still happens — the new service must handle its own re-issued traffic on top of the /consistency-checks load. See concepts/parallel-run-request-doubling.
Async POST is not durable by default. If the monolith process dies between responding to client and firing the async POST, that sample is lost. Not a correctness issue (client got its answer) but is a measurement-completeness issue — match rates may be biased if loss correlates with certain request classes.
Comparison backlog can hide regressions. If the new service's /consistency-checks queue builds up, staleness of the match-rate metrics masks problems.
Headers like Date, X-Request-Id, transfer-encoding differ deterministically. The comparator must ignore them or normalise them out.

Architectural layers (Zalando's cleanup reveals)¶

Zalando names three layers added to the new service to support this pattern, all removed post-migration:

Use-cases layer — the handler that processes the async consistency-check POST.
Gateway layer — the localhost gateway that re-issues the real request against the same service's own real endpoint.
Entities layer — the domain model for the consistency-check logic.

Plus: the feature toggle gating it, the consistency-checker route in the router, the dependency-injection wiring in Main, and all tests for the consistency-check logic. ~700 LOC + ~1.3k LOC of tests removed at cleanup.

Seen in¶

Zalando Returns-service extraction (2021-11-03, sources/2021-11-03-zalando-parallel-run-pattern-a-migration-technique-in-microservices) — canonical.