Skip to content

PATTERN Cited by 1 source

Separate edge links from properties

Separate edge links from properties is a graph-storage layout pattern: the link (the existence of an edge between two nodes) and the property bag (everything attached to that edge) are stored in two separate KV namespaces, with two distinct identifier strategies and two distinct caching strategies, accepting non-atomic-multi-namespace-writes as the trade-off (resolved by an entropy-repair pipeline).

The pattern is the load-bearing structural disclosure of Netflix Graph Abstraction, where it powers a 10 M ops/sec / 650 TB / single-digit-ms p99 edge persistence workload.

Structure

┌─────────────────────────────────────────────────────────────┐
│ Edge "linked_to" between nodes A and B                      │
└─────────────────────────────────────────────────────────────┘
        ▲ separate
        │ namespaces
        ▼
┌─────────────────────────┐    ┌─────────────────────────────┐
│ Edge LINK namespace     │    │ Edge PROPERTY namespace     │
│  (per direction × type) │    │  (per edge type)            │
│                         │    │                             │
│ Forward index           │    │ Direction-agnostic ID:      │
│  partition: source A    │    │   sort([A, B]).concat()     │
│  cluster:   target B    │    │                             │
│  payload:   {last_write}│    │ Payload: full property bag  │
│                         │    │  registration_time: TIMESTAMP│
│ Reverse index           │    │  status:            STRING  │
│  partition: target B    │    │                             │
│  cluster:   source A    │    │ One record per edge pair    │
│  payload:   {last_write}│    │ — direction-agnostic        │
└─────────────────────────┘    └─────────────────────────────┘

Why this layout

Three structural problems get solved simultaneously:

  1. Wide-row prevention. A high-fanout node could otherwise blow Cassandra's partition-size envelope. With links and properties separated, the link partition stays narrow — each entry is just {target_node_id → last_write_time} — even for a node with millions of edges. "Decoupling edge links from their properties prevents large partitions in databases like Cassandra, enabling efficient storage and low-latency reads — even for edges with millions of connections." (Source: sources/2026-05-29-netflix-high-throughput-graph-abstraction-at-netflix-part-i)
  2. Efficient property upserts. Updating one property of an edge does not require reading and rewriting the whole property set — "Allows individual properties to be upserted over time without needing to read the entire property set for an edge." The property namespace is a regular KV record per edge pair; column-level upserts work directly.
  3. Direction-symmetric properties + direction-asymmetric links. Properties of an edge are the same regardless of which direction the user traversed; links are inherently directional (forward index ≠ reverse index). The two different identifier strategies (lex-sorted-concat for properties; natural endpoint-keyed for links) are the minimum-cost expression of this asymmetry.

Trade-off the pattern accepts

Cost Mitigation
Non-atomic multi-namespace writes. A write of (A→B) touches at minimum 3 records: forward link (partition A), reverse link (partition B), property bag. None of the 3 commits are bound by a distributed transaction. Kafka entropy repair picks up failed writes, retries until convergence; LWW + idempotency tokens guarantee the retry semantics.
Slightly higher write fanout: 3+ KV writes per edge mutation (or 2× more if forward+reverse indexes are also used). Caching: write-aside cache for edge links suppresses redundant writes when the link already exists.
Property bag is read-after the link record in some traversals (extra round-trip if not cached). Read-aside cache on EVCache; property-key pushdown so only requested fields cross the wire.

Composition with adjacent patterns

The pattern composes with three other patterns to make the graph layout work end-to-end:

Without all three, the pattern's correctness or scalability breaks somewhere.

When to use

  • High-fanout property graphs where some nodes have hundreds of thousands to millions of edges.
  • KV-substrate graph databases on stores like Cassandra where wide-row penalties are real.
  • Workloads with frequent partial property upserts (per- property updates dominate over full-edge-replace updates).
  • Workloads that already need an entropy-repair pipeline (the pattern's main cost is amortised once the substrate exists).

When not to use

  • Property bags are small and never updated independently. Storing them in the link record keeps writes single-namespace and atomic.
  • Edges are extremely sparse and there's no wide-row hazard.
  • The system has distributed transactions across namespaces — the trade-off this pattern resolves doesn't bind.
  • Pure relational workloads where edges and properties map to a flat table; SQL handles the access patterns better.

Risks

  • Long-tail entropy. If the entropy-repair pipeline is unhealthy, inconsistencies between link and property namespaces persist. Critical to monitor queue depth + lag.
  • Property-only update missing the link. A property write that succeeds while the link write fails leaves an unreachable property record. Symmetric scenario: link writes that succeed without property data leave skeleton edges. Both are repaired by the same retry pipeline but observable briefly to readers.
  • Lifecycle confusion on cascade delete. Deleting a node must cascade through both namespaces; partial cascades leave dangling property records. Mitigation: async cascade delete with the same LWW invariant.

Seen in

Last updated · 542 distilled / 1,571 read