Skip to content

PATTERN Cited by 1 source

Explicit Boundary Translation

Explicit boundary translation is the implementation pattern that realises concepts/boundary-as-feature: when two abstractions genuinely differ on load-bearing semantics, build an inspectable translation surface between them with (1) declared asymmetric consistency contracts, (2) a synchronisation mechanism with a stated cadence and conflict policy, (3) a visible failure mode when data can't cross the boundary, and (4) a programmable surface customers can observe and evolve against.

(Source: sources/2026-04-07-allthingsdistributed-s3-files-and-the-changing-face-of-s3)

The four design commitments

1. Declared asymmetric consistency contracts

Both sides retain their native semantics; neither is forced to approximate the other.

S3 Files example:

  • Filesystem side: NFS close-to-open consistency — "the file experience your tools already expect: NFS semantics, directory operations, permissions."
  • Object side: full S3 atomic-PUT strong consistency — notifications at-least-once delivered, CRR pipelines unchanged, conditional-writes continue to work.

Neither side compromises; the translation layer carries the mismatch.

2. Synchronisation mechanism with declared cadence and policy

Not "eventually"; a specific translation schedule and an explicit policy for conflicts.

S3 Files example:

  • Commit cadence: changes aggregated and committed back to S3 roughly every 60 seconds as a single PUT.
  • Bidirectional sync: S3-side mutations propagate to the filesystem view automatically.
  • Conflict policy: S3 wins; filesystem-side loser moves to lost+found; CloudWatch metric fires.

See concepts/stage-and-commit for the cadence + policy primitive itself.

3. Visible failure at the boundary

When data genuinely cannot cross the translation surface, surface an event — never silently drop or silently corrupt.

S3 Files example: object keys that aren't valid POSIX filenames, and files that aren't representable as valid object keys. The team's design principle:

"When objects or files are created that can't be moved across the boundary, we decided that (and wow was this ever a lot of passionate discussion) we just wouldn't move them. Instead, we would emit an event to allow customers to monitor and take action if necessary. This is clearly an example of downloading complexity onto the developer, but I think it's also a profoundly good example of that being the right thing to do, because we are choosing not to fail things in the domains where they already expect to run."

This is a security / correctness property: agents and humans reasoning about the system can trust that what crossed the boundary is what they think; what didn't cross produced a signal they can observe.

4. Programmable surface that can evolve

The boundary mechanism is published as part of the system — it isn't an implementation detail. Customers can reason about it, monitor it, and (over time) programmatically control it.

S3 Files example, Warfield:

"The specifics of how and when data transited the boundary would be published as part of the system, clear to customers, and something that we could actually continue to evolve and improve as a programmatic primitive over time."

At launch, that programmability is limited (fixed 60s cadence); the forward bet is that customers will get richer control (explicit commit API, pipeline integration, mid-commit visibility).

Applying the pattern

Step 1: Identify the load-bearing differences

Enumerate the axes on which the two sides differ — explicitly. S3 Files identified five: mutation granularity, atomicity primitives, authorization model, namespace semantics, namespace performance shape. See concepts/file-vs-object-semantics.

Step 2: Pick a source of truth

When conflicts happen, one side must be authoritative. S3 Files: "S3 is the source of truth." This should be the side with the larger existing installed base of applications that structurally depend on the current semantics.

Step 3: Design the translation for the common case

S3 Files' key insight: "very few applications use both file and object interfaces concurrently on the same data at the same instant. The far more common pattern is multiphase." Design for the 99% multiphase pattern (stage → commit → next-phase reads), not the concurrent-write case.

Step 4: Make the edges visible

  • Events for data that cannot cross.
  • Metrics for conflicts and losses.
  • Warnings for expensive operations at boundary scale (S3 Files warns at mounts > 50M objects because rename cost is O(objects)).

Step 5: Publish the cadence and policy

Customers cannot reason about a boundary whose specifics aren't documented. The 60-second window and conflict policy are customer-facing facts.

Contrast with convergence-layer design

The opposite pattern — hide the boundary behind a convergence layer — has a consistent failure mode: silent compromise of both sides. S3 Files' six-month "EFS3" design dead-end is the canonical warning story for this pattern. See concepts/boundary-as-feature.

Generalisability

The pattern applies wherever two abstractions must interoperate over the same physical data with load-bearing semantic differences:

  • File ↔ Object (S3 Files; this source).
  • Transactional ↔ Analytical (the OLTP/OLAP bridge — patterns/warehouse-unload-bridge is a specific instance with explicit cadence + queue-gated translation).
  • Streaming ↔ Batch (Lambda/Kappa architectures that admit the boundary instead of convergence).
  • Cloud A ↔ Cloud B (cross-cloud data exchange; patterns/cross-cloud-replica-cache is a closely-related instance, where open protocol + incremental replica cache is the translation).

Seen in

Last updated · 200 distilled / 1,178 read