Skip to content

SYSTEM Cited by 1 source

Marken — Scalable Annotation Service (Netflix)

Definition

Marken is Netflix's internal annotation service — the transactional persistence layer for machine-learning-model output describing media content (character recognition, scene detection, object tagging, embeddings, etc.). Every annotation is keyed by the source asset plus a time range and carries a typed label + optional embedding vector + confidence score (Source: sources/2026-04-04-netflix-powering-multimodal-intelligence-for-video-search).

Marken is backed by Apache Cassandra for "distributed storage"; the design posture is "data integrity and high-speed write throughput, guaranteeing that every piece of model output is safely captured".

Role in the multimodal video-search pipeline

Marken sits at stage 1 of Netflix's three-stage ingest-fusion- index pipeline (patterns/three-stage-ingest-fusion-index):

  1. Stage 1 — Transactional Persistence (this system). Raw per-model annotations land in Marken over high-availability pipelines.
  2. Stage 2 — Offline Data Fusion. On each write, Marken publishes an event via Kafka to trigger an asynchronous fusion job that discretizes annotations into fixed-size time buckets and computes cross-model intersections (concepts/temporal-bucket-discretization, concepts/multimodal-annotation-intersection). Enriched bucket records are written back into Cassandra as distinct entities.
  3. Stage 3 — Indexing for Search. A subsequent event triggers ingestion of enriched buckets into Elasticsearch as nested documents.

Annotation shape

Netflix documents one annotation type — SCENE_SEARCH — in the post:

{
  "type": "SCENE_SEARCH",
  "time_range": {
    "start_time_ns": 4000000000,
    "end_time_ns": 9000000000
  },
  "embedding_vector": [-0.036, -0.33, -0.29, ...],
  "label": "kitchen",
  "confidence_score": 0.72
}

A second type (CHARACTER_SEARCH) appears in the intersection- record example without its own schema disclosure. Annotations carry:

  • Annotation type (SCENE_SEARCH, CHARACTER_SEARCH, …) — one per producer model class.
  • Time range in nanoseconds — continuous-interval semantics.
  • Label — human-interpretable symbol.
  • Embedding vector — optional (present on scene search; absent on character search in the examples).
  • Confidence score — 0-1 model probability.

Seen in

Caveats

  • Marken was first described in a separate 2021 Netflix post ("Scalable Annotation Service: Marken" — linked from the 2026-04-04 post) that is not yet ingested on this wiki; the stub here captures only the role + surface disclosed in the 2026-04-04 post.
  • Schema details (Cassandra partition / clustering keys, retention TTL, consistency levels, write QPS) are not disclosed in the 2026-04-04 post.
  • The relationship between Marken-persisted annotations and MediaFM's shot-level embeddings is not disclosed; both are Netflix content-model output but at different altitudes (Marken per-model-per-span annotations vs MediaFM per-shot fused 2304-dim embeddings).
Last updated · 319 distilled / 1,201 read