SYSTEM Cited by 1 source
Metaflow Cache (Netflix)¶
metaflow.Cache is the Netflix-internal API for writing
precomputed results from a scheduled Metaflow
job to a low-latency key-value store. Paired with
Metaflow Hosting, it forms
Netflix's officially supported pattern for business-critical
applications where all or some predictions can be precomputed
(Source:
sources/2024-07-22-netflix-supporting-diverse-ml-systems-at-netflix).
When to use it¶
"Not all API-based deployments require real-time evaluation, which we cover in the section below. We have a number of business-critical applications where some or all predictions can be precomputed, guaranteeing the lowest possible latency and operationally simple high availability at the global scale."
"While the system relies on our internal caching infrastructure, you could follow the same pattern using services like Amazon ElastiCache or DynamoDB." See concepts/precomputed-predictions-api for the general design.
Example: content performance visualisation¶
┌──────────────────────────┐
daily cron ──►│ Metaflow scheduled job │
│ (parallel aggregate │
│ computations) │
└──────────┬───────────────┘
│ large volume
│ of results
▼
┌──────────────────────────┐
│ metaflow.Cache │
│ (online KV store) │
└──────────┬───────────────┘
│ read
▼
┌──────────────────────────┐ ┌──────────────────┐
│ metaflow.Hosting service│ ◄─ UI ──│ Streamlit app │
│ (lookup + compute + │ JSON │ (viz + params) │
│ return JSON) │ ───────►│ │
└──────────────────────────┘ └──────────────────┘
From the post:
"A daily scheduled Metaflow job computes aggregate quantities of interest in parallel. The job writes a large volume of results to an online key-value store using
metaflow.Cache. A Streamlit app houses the visualization software and data aggregation logic. Users can dynamically change parameters of the visualization application and in real-time a message is sent to a simple Metaflow hosting service which looks up values in the cache, performs computation, and returns the results as a JSON blob to the Streamlit application."
Canonical wiki instance of patterns/precompute-then-api-serve. See systems/streamlit for the front-end framework used.
Disclosure¶
No cache-size numbers, per-key size, read/write QPS, TTL policy, or internal KV backing technology are disclosed. Similarly, cost and cold-start characteristics of the paired Metaflow Hosting tier aren't quantified in this post.