Skip to content

CONCEPT Cited by 1 source

Boundary as Feature

A design principle crystallised in the 2026 S3 Files post: when two abstractions (e.g. file and object, stream and batch, SQL and key-value) genuinely differ on load-bearing semantics, the boundary between them is the right place to put the design effort — it should be explicit, inspectable, and programmable, not hidden behind a convergence layer that silently compromises both sides.

The origin quote, Warfield 2026:

"There is actually a pretty profound boundary between files and objects. File interactions are agile, often mutation heavy, and semantically rich. Objects on the other hand come with a relatively focused and narrow set of semantics; and we realized that this boundary that separated them was what we really needed to pay attention to, and that rather than trying to hide it, the boundary itself was the feature we needed to build."

(Source: sources/2026-04-07-allthingsdistributed-s3-files-and-the-changing-face-of-s3)

The convergence failure mode

The dominant industry instinct when customers complain about a choice between two abstractions is to build a convergence layer — a "best of both worlds" system that hides the boundary. The S3 Files team tried this for six months in 2024 under the nickname "EFS3":

"Every time we sat down to work through designs, we found difficult technical challenges and tough decisions. And in each of these decisions, either the file or the object presentation of data would have to give something up in the design that would make it a bit less good. One of the engineers on the team described this as 'a battle of unpalatable compromises.'"

The enumerated compromises were explicitly identified as "not the best of both worlds, but the lowest common denominator" — example workloads on both sides broke on the merged system.

The inversion

Post-holidays 2024, the team reframed the problem:

"The boundary between file and object didn't actually have to be invisible. And this time, the team started coming out of discussions looking a lot happier."

The design work was no longer about eliminating the boundary, but about designing a good boundary:

  1. Explicit — both sides retain their native semantics; neither is forced to approximate the other.
  2. Inspectable — when data can't cross the boundary (e.g. an S3 key that isn't a valid POSIX filename), an event is emitted so customers can see it.
  3. Programmable — concepts/stage-and-commit treats the boundary as a programmable primitive that can evolve over time (richer commit control, pipeline integration, multi-mount).
  4. Asymmetric — when a tiebreaker is needed (conflict policy in S3 Files), one side is the declared source of truth; the other loses visibly (lost+found + CloudWatch metric), not silently.

See patterns/explicit-boundary-translation for the generalised pattern.

Why this is a generalisable principle

The 2026 post explicitly generalises:

"Different ways of working with data aren't a problem to be collapsed. They're a reality to be served."

The same shape applies anywhere two abstractions differ on load- bearing semantics — axes where one side's customers have built applications that structurally depend on a specific behaviour:

  • Files ↔ Objects — 2026 S3 Files (this source).
  • Transactional ↔ Analytical — the OLTP/OLAP split; HTAP systems that try to converge often compromise both. See concepts/oltp-vs-olap.
  • Streaming ↔ Batch — similar convergence attempts (Lambda- architecture, Kappa-architecture); the most durable designs admit the boundary and invest in bridge primitives.
  • SQL ↔ KV — sometimes converged (CockroachDB SQL over KV), often explicitly separated (DynamoDB's deliberate non-SQL surface).

In each case, the convergence attempt pays a quiet tax — customers on both sides hit subtle failure modes — while the boundary-as-feature approach keeps each side uncompromised and spends the design budget on the translation mechanism instead.

Relationship to simplicity-vs-velocity

concepts/simplicity-vs-velocity says: racing to ship a merged design with known gaps backloads simplification work that is more expensive to do later. Boundary-as-feature is a specific answer to the "what merged design do we ship" question: sometimes the right merged design is no merge at all, with an excellent translation surface.

Warfield's hindsight on S3 Files:

"We spent months trying to make it disappear, and when we finally accepted it as a first-class element of the system, everything got better."

Failure modes to guard against

  • Pretending convergence on a boundary you didn't actually converge — silently compromises both sides and erodes trust.
  • Exposing the boundary without a translation mechanism — pushes all the work onto customers; no better than not building it.
  • Asymmetric conflict policy that's implicit — S3 Files' "S3 wins, filesystem-side goes to lost+found with a CloudWatch metric" is load-bearing; silent loss is unacceptable.
  • Treating the boundary as frozen — the programmable surface framing (stage-and-commit as a primitive that can evolve) protects against this.

Seen in

Last updated · 200 distilled / 1,178 read