Skip to content

SYSTEM Cited by 19 sources

DynamoDB

Amazon DynamoDB is AWS's managed serverless key-value / document database. It's horizontally scalable by partition key, JSON-friendly, and commonly chosen when the workload needs fast single-item reads and writes at unbounded scale and you don't want to operate a sharding layer yourself.

Pattern of appearance

In system-design stories DynamoDB most often shows up as the scale-out replacement for a vertically-scaled OLTP store used as an event log or hot-write workload. The storage-scalability story is strong; the processing story is the same as any other OLTP store — per-record lookups are still per-record lookups.

Canva's Creators-payment counting pipeline used DynamoDB for this exact role: when MySQL RDS was doubling every 8–10 months, they moved raw usage events to DynamoDB to eliminate the storage growth problem. They explicitly chose not to move the rest of the pipeline onto DynamoDB, because the per-record database round-trip problem remains: running deduplication and aggregation over billions of events by reading each one is O(N) DB calls regardless of backend. Canva took this as a signal to move the processing into an OLAP engine (Snowflake), not just re-platform the OLTP layer. (Source: sources/2024-04-29-canva-scaling-to-count-billions)

Data shape notes

DynamoDB stores JSON-shaped items natively, which is ergonomic for event-producer code but not ideal for warehouse query performance. Canva's extract-load step into Snowflake projects the interesting JSON fields into typed SQL columns to make downstream GROUP BY / filter queries cheap — worth generalising: if DynamoDB is the upstream to an analytics workload, plan for a typed projection in the E/L.

Seen in

  • sources/2026-09-18-aws-how-csiro-built-scalable-cost-optimized-genomic-variant-querying-on-aws — DynamoDB as the ontology-term lookup store in sBeacon's genomic query path. The Microservice Lambda resolves a query's ontology terms to their hierarchical descendant terms and codes in DynamoDB (built during onboarding from CSIRO Ontoserver), then uses those codes to query metadata tables on Athena. This is what lets a query like "individuals with a specific BRCA1 mutation who have stage 3 cancer" match all narrower clinical/phenotypic concepts. Cost is negligible (~USD 0.0005 / 1000 queries), and DynamoDB has no public resource policy under the zero-trust boundary. (Source: sources/2026-09-18-aws-how-csiro-built-scalable-cost-optimized-genomic-variant-querying-on-aws)
  • sources/2026-09-10-aws-building-resilient-real-time-streaming-workers-with-amazon-dynamodb-leases — DynamoDB as a distributed-lease store (concepts/distributed-lease) coordinating a fleet of stateful WebSocket workers with no external lock service. Each connection is one item holding lock state (lease_owner, lease_expires_at_ms) + domain state (desired_state, ws_url, last_seq); conditional writes on update_item make acquire/renew/release atomic (loser gets ConditionalCheckFailedException). A GSI on (desired_state, lease_expires_at_ms) turns orphan reconciliation into a query for desired_state = STARTED AND lease_expires_at_ms < now (eventually-consistent reads → half the RCU). A second messages table captures received WebSocket payloads. Explicit clock-skew note: lease expiry is evaluated against the caller's now, not a DynamoDB server clock, so workers must have synchronized clocks (Amazon Time Sync on Fargate). Cost driver is heartbeat writes (1 WCU per renewal): ~$65/mo on-demand at 100 connections (5 s heartbeat); the post recommends provisioned capacity + autoscaling for sustained load. (Source: sources/2026-09-10-aws-building-resilient-real-time-streaming-workers-with-amazon-dynamodb-leases)
  • sources/2026-08-19-aws-ai-powered-clinical-trial-eligibility-and-safety-using-amazon-bedrock-agentcore — immutable audit records for every eligibility/safety decision (clinician ID, timestamp, patient/trial IDs, outcomes, AI recommendations, full workflow execution history), designed to support FDA 21 CFR Part 11; a control-plane/policy store distinct from streamed data paths.
  • sources/2026-08-21-aws-how-agentflo-built-ai-sales-agents-with-amazon-bedrock-agentcore-part-2 — AgentFlo's Reliability pillar uses a two-table design (Session Table + Cart Table) as the stateful-agent substrate: AgentCore Runtime loads the last 15 messages from the Session Table each turn, and loads the Cart Table only on cart-related intent (intent-gated-state-loading) so the model can't hallucinate prices/quantities — session continuity + autonomous context loading + structured ground-truth in one store (concepts/agent-memory).

  • sources/2026-08-26-aws-closing-the-ai-agent-trust-gap-with-graduated-autonomy — the trust-state and audit store for graduated autonomy: a current-state record plus a per-agent time-series history hold each agent's tier/score, read on every invocation (single-digit-ms lookup) to enforce the current tier; the Think→Plan→Act→Observe→Score audit records (with pre-action state for reversibility) also persist here.

  • sources/2026-08-18-aws-consistency-is-the-new-latency-ai-at-the-data-layer — DynamoDB Global Tables as Pattern B (multi-leader global availability) of the replication trinity for AI-agent memory; conditional writes (ConditionExpression / ConditionalCheckFailedException) prevent parallel agents from overwriting each other's reasoning (lost-update).

  • sources/2024-04-29-canva-scaling-to-count-billions — DynamoDB as the raw-event store for Canva's counting pipeline, replacing MySQL for that tier; explicit decision not to move dedup/aggregation onto it because DB round-trips still dominate.
  • sources/2024-07-29-aws-amazons-exabyte-scale-migration-from-apache-spark-to-ray-on-ec2 — DynamoDB as the durable job-lifecycle state store in Amazon BDT's 2021 serverless Ray job-management substrate (alongside systems/aws-sns, systems/aws-sqs, systems/aws-s3) — powering >1,600 Ray compaction jobs/day over exabyte-scale S3 tables.
  • sources/2024-08-01-segment-0-6m-year-savings-by-using-s3-for-change-data-capture-for-dynamodb — DynamoDB at petabyte scale, with a canonical cost-driven rejection of a Global Secondary Index. Segment's objects pipeline stores ~958 billion items in a ~1 PetaByte DynamoDB table (avg item size ~900 bytes; $0.25/GB·month ≈ ~$250K/month base-storage cost). The team wanted a secondary query surface — "items modified since timestamp T" — and explicitly rejected the in-database answer on cost grounds: "due to the very large size of our table, creating a GSI for the table is not cost-efficient." Instead they externalised the changelog first to Google Cloud Bigtable (V1, cross- cloud) and then to Amazon S3 (V2, ~$0.6M/ year savings, single-cloud). Canonicalised as gsi-cost-anti-pattern-at-petabyte-scale + concepts/secondary-index + object-store-as-cdc-log-store — the first wiki datum on DynamoDB-at-~1PB and the first canonical case where the secondary-index surface is materialised outside the DynamoDB store on cost grounds.
  • sources/2025-12-18-dropbox-feature-store-powering-real-time-ai-dash — DynamoDB as the API surface Dropbox's internal Dynovault is compatible with; Feast's DynamoDB adapter works against Dynovault unmodified. Confirms DynamoDB's value as a compatibility target even when the actual store is on-prem.
  • sources/2026-02-05-aws-convera-verified-permissions-fine-grained-authorization — DynamoDB fills three distinct control-plane roles in Convera's Verified Permissions architecture: (1) user/tenant attribute store read by the pre-token hook to enrich Cognito access tokens; (2) user-to-tenant mapping table used by the pre-token hook to look up tenant_id for the JWT claim; (3) tenant_id → policy-store-id mapping used by the Lambda authorizer to route each request to the right per-tenant AVP store. Plus a fourth role — Cedar policy source of truth — with DynamoDB Streams capturing changes and a sync pipeline continuously propagating them into AVP policy stores.
  • sources/2026-02-25-aws-6000-accounts-three-people-one-platform — named alongside Lambda as the canonical scale-to-zero serverless primitive that makes account-per-tenant economically viable at ProGlove's scale (~6,000 tenant accounts, ~1M Lambda functions). On-demand DynamoDB bills per request and has no per-account provisioned floor, so replicating "one table per service per tenant" across thousands of accounts doesn't multiply idle cost the way a per-account RDS instance would.
  • sources/2026-03-31-aws-streamlining-access-to-dr-capabilities — DynamoDB named as the canonical example of cross-Region backup coverage that AWS Backup added on top of a service that previously lacked it: "AWS Backup ... even enabled cross-Region backup for services like Amazon DynamoDB, which previously didn't have that capability."
  • sources/2026-04-21-figma-the-infrastructure-behind-ai-search-in-figma — DynamoDB as the metadata + embedding KV store for Figma AI Search's indexing pipeline. Chosen explicitly over Figma's existing RDS cluster because the workload "requires only a simple key-value store, writing and reading at high throughput. No transactions or foreign key relationships are required." Secondary role: the authoritative source for _source-slimmed OpenSearch fields — embeddings are removed from OpenSearch's _source for size, then re-fetched from DynamoDB on every update to prevent updates wiping the embedding (source-field-slimming-with-external-refetch).
  • sources/2026-04-08-aws-build-a-multi-tenant-configuration-system-with-tagged-storage-patterns — DynamoDB is the high-frequency per-tenant backend in the tagged-storage-routing split; keyed pk = TENANT#{id}, sk = CONFIG#{type} for composite tenant-scoped query shape + built-in multi-tenant isolation at the data model level. Stores per-tenant feature flags, payment-gateway preferences, etc. accessed "thousands of times per minute". For sub-millisecond at 1000+ RPS the post names DAX as the acceleration layer (5-10× over DynamoDB's single-digit-ms baseline). Multi- dimensional tenant context (e.g., pk = TENANT#{id}|SERVICE#{name}) named as the extension for service-level access-control beyond tenant-level.
  • sources/2022-12-02-highscalability-stuff-the-internet-says-on-scalability-for-december-2nd-2022 — DynamoDB at hyperscale (High Scalability Dec-2022 roundup). Two canonical disclosures: (1) Snap's Snapchat on AWS — 400 TB in DynamoDB, nightly scans running ~2 billion rows per minute for friend suggestions + ephemeral-data deletion. Snap's SnapDB is a transaction/TTL/ ephemeral-data abstraction on top of DynamoDB. (2) Instacart Postgres → DynamoDB push-notifications migration — migrated when projected 8× traffic growth would exceed the largest available EC2 instance size. Cut costs ~½ vs. Postgres and scaled on-demand with "no visible performance impact." Stated rule of thumb: new workloads where Postgres won't handle the fit go to DynamoDB from day one; existing Postgres tables have a higher migration bar. In 6 months they grew from 1 to 20+ tables serving 5–10 features. Thinly wrapped Dynamoid for ActiveRecord-style access.
  • sources/2026-01-06-lyft-feature-store-architecture-optimization-and-evolution — DynamoDB as the persistent backing store of Lyft's dsfeatures feature-serving layer. Primary key is composed of metadata fields; a Global Secondary Index (GSI) is provisioned specifically for GDPR-deletion efficiency (bounded-cost bulk-delete-by-user). A ValKey write-through LRU cache layers on top for ultra-low-latency hot-path reads; embedding features live in OpenSearch instead. Demonstrates the wrapper- over-heterogeneous-stores shape in production.
  • sources/2024-09-19-netflix-netflixs-key-value-data-abstraction-layer — DynamoDB named as one of the backing engines Netflix's KV Data Abstraction Layer can route namespaces to alongside Cassandra + EVCache + RocksDB. "The abstraction works with multiple data stores like EVCache, DynamoDB, RocksDB, etc..." Shown only in the list (no concrete DynamoDB-backed namespace example given in the post).
  • sources/2026-04-23-aws-modernizing-kyc-with-aws-serverless-solutions-and-agentic-ai — DynamoDB as the real-time decision store in IBM + AWS's KYC architecture, complementing Bedrock Knowledge Bases. Holds "sub-millisecond access to frequently accessed structured data, including current KYC decision status, risk scores, customer interaction history, and dynamic configuration parameters controlling agent behavior." The hot-state sibling to OpenSearch Serverless's regulatory- corpus vector store — DynamoDB for case state, OpenSearch for corpus knowledge.
  • — DynamoDB as the durable store under a cache-first serving tier at millions-rps scale. Zalando's PRAPI treats DynamoDB as the cache-miss fallback beneath a Caffeine async-loading cache ( 60s TTL / 15s stale window); "DynamoDB ensures high availability and fast lookups when cache misses occur." Ingestion knob: each Updater pod performs 10 concurrent batch writes of 25 items (250 items in flight), so cold-filling a new Market Group is bottlenecked on DynamoDB write-capacity units. Serving path uses a two-client fallback (10 ms primary timeout, 100 ms fallback for retries) with LIFO-disciplined queues to isolate DynamoDB latency spikes from the primary client. The canonical wiki instance of DynamoDB as cache backing store, not primary serving store.
  • — DynamoDB as the outbox of a transactional-outbox relay, via DynamoDB Streams (NEW_AND_OLD_IMAGES) + an AWS Lambda relay publishing to Nakadi. Zalando Payments's Order Store writes only to DynamoDB on the sync path; the stream is the outbox, so there is no separate outbox table and no dual-write risk. Canonical wiki example of DynamoDB table + Streams as a self-contained outbox substrate — canonicalised as dynamodb-streams-plus-lambda-outbox-relay. The redesign was motivated by the 99.9% × 99.9% = 99.8% availability arithmetic; removing Nakadi from the sync path restored the ceiling to 99.9%.
  • — DynamoDB as the originating failure of the 2025-10-20 AWS us-east-1 incident. The first wiki entry recording this outage as a named event on DynamoDB's own page. PlanetScale's control-plane dependency chain terminated at DynamoDB via four transitive hops (new-branch/resize service → internal secret-distribution → S3 → STS → DynamoDB), with an upstream DNS misconfiguration as the root cause. Phase 1 of the incident (~2 h 17 min window) took PlanetScale's control plane offline; customer database branches continued serving queries unaffected because the data plane did not call back into the DynamoDB-dependent control-plane chain (see concepts/control-plane-data-plane-separation). First explicit wiki record of DynamoDB sitting beneath STS + S3 in AWS's internal dependency DAG — "AWS STS which was impacted by the Amazon DynamoDB outage." Canonical substrate for runtime-dependency-on-saas-provider and the four-hop transitive-chain framing.

Global Tables as distributed lock store

DynamoDB Global Tables provide multi-region, active-active replication with strong consistency per region. This makes them suitable as a distributed lock substrate: a conditional write (PutItem with ConditionExpression: attribute_not_exists(pk)) ensures at most one writer acquires the lock, and cross-region replication propagates lock state to all regions. Used as both lock and idempotency store in AWS's reference architecture for multi-region CloudFormation custom resources (2026-07-22). Trade-off: replication lag means a secondary handler must wait (via queue delay) long enough for the lock to propagate. (Source: sources/2026-07-22-aws-building-multi-region-resiliency-for-cloudformation-custom-resources)

Conditional writes as optimistic concurrency for multi-agent memory

The AWS "Consistency is the new latency" post frames the same conditional-write primitive as the way parallel AI agents share mutable memory (session state, conversational history) without a lost update. A ConditionExpression checking a version timestamp or attribute state means an agent updates a record only if it hasn't changed since it was read; a failing condition returns ConditionalCheckFailedException, which the post calls "a critical signal: it tells the agent to re-read the current state and reconsider its decision, rather than blindly overwriting another agent's work." This prevents lost updates "without requiring synchronous global coordination" — the multi-leader Pattern B of the replication trinity (contrast the strong-consistency Aurora Global Database Pattern A and the leaderless-quorum Keyspaces Pattern C). See conditional-write-optimistic-concurrency and consistency-matched-to-truth-requirement. (Source: sources/2026-08-18-aws-consistency-is-the-new-latency-ai-at-the-data-layer)

Bosch L.OS policy and routing store

Bosch L.OS uses DynamoDB for business rules, security policies, authorization configurations, and routing rules. This is a policy/control-plane role distinct from the platform's streamed location-event path on MSK; the source does not disclose table schema, consistency mode, access pattern, or policy-cache propagation. (Source: sources/2026-08-14-aws-serverless-vehicle-tracking-at-scale-bosch-los-on-aws)

Seen in — hybrid cloud orchestration (2026-09-01)

DynamoDB is the Inventory Management System — the single source of truth — in the AWS hybrid-cloud orchestration engine: tables track sites, hardware (BIOS/firmware versions, encrypted credentials, IP/MAC, rack position, cluster membership), clusters, orders (operation lifecycle), and a catalog of reusable blueprints. Orders are created as DynamoDB records and updated asynchronously from workflow events. DynamoDB Streams trigger Lambda to keep dependent infrastructure in sync — Route 53 DNS records and Portworx storage provisioning (dynamodb-streams-triggered-dependent-infra). The inventory also backs conflict prevention (deny a new order if one is already running on the same resource). (Source: sources/2026-09-01-aws-hybrid-cloud-orchestration-modernizing-on-premises-infrastructure)

Last updated · 766 distilled / 2,225 read