Skip to content

System Design Wiki — Overview

A synthesis of the current corpus, not a claim about the industry as a whole.

Methodology. Node counts below are the number of distinct source pages that cite a node via a [[link]] in their body (counted once per file, via a parse over wiki/sources/ with frontmatter stripped). They are a coverage signal, not a relevance ranking — a higher count means the corpus talks about a node more, not that it matters more. Theme-cluster counts are non-exclusive: one article can count toward storage, streaming, reliability, and AI-infra at once. Corpus totals (concepts, patterns, systems, companies) are counted from files on disk.

Last refreshed: 2026-10-01 (753 sources / 42 company pages read; concept, pattern, and system pages counted from disk; per-node citations recomputed from source bodies).

Corpus at a glance

Corpus item Count
Ingested source articles 753
Company pages 42
Concept pages 282
Pattern pages 76
System pages 1,948
Source publication range 2021-03 – 2026-09

Sources by year: 2019 (1), 2021 (2), 2022 (4), 2023 (7), 2024 (57), 2025 (140), 2026 (542). The corpus is heavily weighted to the last ~18 months — ~90% of sources are 2025–2026 — so trends below reflect recent practice, and any "evolution over time" claim leans almost entirely on the 2024→2026 window. (One stray 2019 Figma multiplayer article predates the main range.) Within 2026, ingest spiked at 2026-04 (145) and stayed high through 2026-08/09 (~70/mo).

By company (source-article count): Cloudflare (115), Databricks (91), AWS (66), Redpanda (49), Meta (43), Netflix (39), Fly.io (32), PlanetScale (29), Zalando (25), Figma (23/24), Google (22), Pinterest / Atlassian (17), Dropbox (16), Yelp / GitHub (15), Slack / MongoDB / Instacart / Airbnb (14), Vercel (12), All Things Distributed (11), High Scalability / Datadog (10), Lyft (8), Spotify / Grafana / Expedia (6), Stripe (4), Canva (3), Wix / Shopify (2), Segment / eBay (1).

Note: the counts here are source articles actually distilled into wiki/sources/. The wiki/companies/index.md "sources" numbers can be larger for some vendors because that index counts a broader set (including raw/queued articles); the distilled-article counts above are the correct denominator for this synthesis.

Cloudflare + Databricks + AWS together supply ~36% of all distilled sources. This weights the synthesis toward public-cloud, data-platform/lakehouse, streaming, and edge-compute practice. It under-represents consumer-mobile backends, gaming, fintech ledgers (Stripe is only 4 sources), and hardware/embedded. Read the counts accordingly.

Most-cited nodes (drill-down entry points)

Systems (cited in N source bodies): systems/model-context-protocol (47), systems/aws-s3 (43), systems/unity-catalog (39), systems/cloudflare-workers (38), systems/redpanda (35), systems/kafka (30), systems/apache-iceberg (30), systems/lakebase (29), systems/mysql (29), systems/postgresql (28), systems/apache-spark (25), systems/kubernetes (25), systems/delta-lake (22), systems/aws-lambda (22), systems/planetscale (22), systems/databricks (20), systems/cloudflare-durable-objects (20), systems/vitess (20), systems/fly-machines (17), systems/dynamodb (17), systems/mlflow (16), systems/databricks-genie (16).

Concepts: concepts/blast-radius (46), concepts/observability (38), concepts/control-plane-data-plane-separation (35), concepts/context-engineering (30), concepts/change-data-capture (28), concepts/defense-in-depth (27), concepts/compute-storage-separation (24), concepts/llm-as-judge (24), concepts/tenant-isolation (23), concepts/scale-to-zero (22), concepts/vector-similarity-search (20), concepts/tail-latency-at-scale (20), concepts/durable-execution (20), concepts/cold-start (19), concepts/idempotent-operations (19), concepts/governed-agent-data-access (19), concepts/agentic-development-loop (18), concepts/schema-evolution (18), concepts/training-serving-boundary (17), concepts/retrieval-ranking-funnel (17), concepts/noisy-neighbor (17), concepts/rpo-rto (17), concepts/least-privileged-access (17).

Patterns: patterns/staged-rollout (46), patterns/specialized-agent-decomposition (34), patterns/shadow-migration (27), patterns/upstream-the-fix (26), patterns/central-proxy-choke-point (23), patterns/tool-surface-minimization (20), patterns/cheap-approximator-with-expensive-fallback (19), patterns/ai-gateway-provider-abstraction (19), patterns/tiered-storage-to-object-store (16), patterns/progressive-configuration-rollout (15), patterns/human-calibrated-llm-labeling (14), patterns/mcp-as-centralized-integration-proxy (13), patterns/snapshot-plus-catchup-replication (12), patterns/on-behalf-of-agent-authorization (12), patterns/fast-rollback (11), patterns/agent-sandbox-with-gateway-only-egress (10), patterns/telemetry-to-lakehouse (10), patterns/default-on-security-upgrade (10).

Recurring themes

Grouped by rough cluster size (distinct sources touching any node in the cluster; clusters overlap, so these do not sum to 753):

  1. AI / agent infrastructure — ~173 sources (largest single cluster). systems/model-context-protocol and concepts/context-engineering anchor it. The cluster spans agent decomposition (patterns/specialized-agent-decomposition), context handling (concepts/context-engineering, concepts/agent-memory), evaluation (concepts/llm-as-judge, patterns/human-calibrated-llm-labeling), retrieval (concepts/retrieval-ranking-funnel, concepts/vector-similarity-search), provider abstraction (patterns/ai-gateway-provider-abstraction), the coding-agent loop (concepts/agentic-development-loop), and agent security (patterns/on-behalf-of-agent-authorization, patterns/agent-sandbox-with-gateway-only-egress, patterns/tool-surface-minimization, concepts/governed-agent-data-access). This is the corpus's center of gravity in 2026 and still growing fastest.

  2. Reliability & safe change — ~126 sources. patterns/staged-rollout and concepts/blast-radius are the #1 pattern and #1 concept overall. Supporting cast: patterns/fast-rollback, patterns/progressive-configuration-rollout, patterns/cell-based-architecture-for-blast-radius-reduction, concepts/rpo-rto, concepts/durable-execution. This is the most cross-company theme — it shows up regardless of domain.

  3. Data platform / lakehouse — ~103 sources. Open table formats (systems/apache-iceberg, systems/delta-lake, concepts/open-table-format), catalogs (systems/unity-catalog), concepts/compute-storage-separation, systems/apache-spark, and the OLTP-in-the-lakehouse play (systems/lakebase). Heavily Databricks-driven; treat as a vendor-concentrated view.

  4. Streaming & CDC — ~89 sources. concepts/change-data-capture (28), systems/kafka, systems/redpanda, and patterns/tiered-storage-to-object-store / patterns/telemetry-to-lakehouse. The streaming-broker-as-lakehouse-source idea recurs.

  5. Multi-tenancy & isolation — ~76 sources. concepts/tenant-isolation (23), concepts/noisy-neighbor, concepts/scale-to-zero, and the patterns/central-proxy-choke-point family. Tightly coupled to the serverless/edge sources (Cloudflare, Fly.io, Vercel).

  6. Sharding & horizontal scale — ~50 sources. concepts/horizontal-sharding, systems/vitess, systems/planetscale, concepts/tail-latency-at-scale. The classic relational-scale core.

  7. Security & cryptography — ~48 sources. concepts/defense-in-depth, concepts/least-privileged-access, patterns/default-on-security-upgrade, plus a Cloudflare/AWS post-quantum-migration strand (concepts/post-quantum-cryptography, concepts/harvest-now-decrypt-later).

The temporal signal is real but rests on a thin 2024 base (57 sources), so read as directional, not precise.

Reading: the corpus's vocabulary shifted from "make one node fast" to "run many tenants safely and wire LLMs into production." Storage evolution: monolithic DB → concepts/compute-storage-separation + concepts/open-table-format on object storage (systems/aws-s3 remains the substrate under almost everything). Observability shifted from metrics-dashboards toward patterns/telemetry-to-lakehouse (logs/traces as queryable data). AI-infra went from near-zero to the largest cluster in ~18 months, and its security sub-theme (agent authz, sandboxing, governed data access) is now a distinct fast-growing strand rather than an afterthought.

Notable trade-offs & contradictions

The wiki has explicit ## Contradiction sections on these pages — surface them before trusting any single-source claim:

Other recurring tensions worth reading in-page: patterns/cheap-approximator-with-expensive-fallback (cost vs accuracy), concepts/tail-latency-at-scale (throughput vs p99), concepts/rpo-rto (durability vs latency).

Open questions / gaps in coverage

  • Vendor concentration: ~36% of sources are Cloudflare/Databricks/AWS. Themes like lakehouse and edge-compute may be over-weighted relative to the wider field.
  • Thin domains: fintech/ledgers (Stripe 4), consumer-mobile backends, gaming, and hardware are barely present. Don't generalize the wiki to these.
  • Taxonomy tail: 11 pattern pages are cited exactly once in bodies (e.g. patterns/hedged-reads-for-tail-latency, patterns/data-contract, patterns/dead-letter-queue) and many concept pages are cited ≤1 time. These are Lint candidates — either enrich with more sources or fold into a parent page. Conversely, patterns/upstream-the-fix and patterns/central-proxy-choke-point have grown into top-tier patterns and deserve richer pages.
  • System-page sprawl: 1,948 system pages against 753 sources means many systems are single-source proper nouns. That's expected (systems are exempt from the canonicalization gate), but it makes systems/ a long-tail reference, not a curated set.
Last updated · 766 distilled / 2,225 read