CONCEPT Cited by 1 source
Unified storage and index¶
Definition¶
Unified storage and index is the managed-service property that a single write both stores the document and indexes it atomically, with no customer-visible sync pipeline, external CDC stream, or secondary store to wire up. The customer writes; the document is immediately (or within a bounded, awaitable poll cycle) searchable through the same primitive.
Contrast with the traditional shape where storage and index live in separate systems — object store / operational DB as source of truth, search engine as a derived index fed by an application-maintained pipeline (dual-write, CDC, batch ETL). That shape incurs the full synchronization tax.
Named in Cloudflare AI Search (2026-04-16)¶
"New instances come with their own storage and vector index. Upload files directly to an instance via API and they're indexed. No R2 buckets to set up, no external data sources to connect first."
"When you call
create(), the instance comes with its own storage and vector index built-in. You can upload a file, the file is sent to index immediately, and you can poll for indexing status all with oneuploadAndPoll()API. Once completed, you can search the instance immediately, and there are no external dependencies to wire together."
Why it matters¶
- Collapses the write-to-searchable latency budget to a single awaitable operation (patterns/upload-then-poll-indexing).
- Eliminates the cross-system consistency problem: there is no primary-replica drift because there is no separate primary and replica at the customer-visible layer.
- Eliminates the operational doubling: one capacity model, one backup strategy, one on-call rotation, one identity surface.
- Enables runtime-provisioned per-tenant instances: if storage and index are separate customer-visible primitives, runtime provisioning = provision both + wire a pipeline between them, a non-trivial operation; unified = one API call.
Contrast shapes¶
| Shape | Example | Sync tax |
|---|---|---|
| Separate storage + separate index, customer-maintained pipeline | App DB + Elasticsearch with CDC / dual-write | High; named in sources/2025-10-12-mongodb-cars24-improves-search-for-300-million-users-with-atlas |
| Separate storage + separate index, managed external-source crawl | AI Search instance pointed at an R2 bucket or website; crawl on schedule | Medium — one direction, managed, but staleness between source writes and crawl |
| Unified storage + index, single API | AI Search instance with built-in storage; items.uploadAndPoll(filename, content) |
None — atomic from the caller's perspective |
| Database with native search | Atlas Hybrid Search — one cluster, two index types | None; same principle at the document-DB-plus-search-engine layer |
Seen in¶
- sources/2026-04-16-cloudflare-ai-search-the-search-primitive-for-your-agents — AI Search instance with
uploadAndPoll()as the canonical API.
Related¶
- concepts/synchronization-tax — the cost class this property eliminates.
- concepts/three-database-problem — AI-era generalisation (operational + search + vector stores all needing sync); unified storage-and-index is one remediation axis.
- concepts/per-tenant-search-instance — runtime-provisioning is only cheap if storage and index are atomic.
- systems/cloudflare-ai-search — canonical instance (AI Search 2026-04-16).
- systems/atlas-hybrid-search — sibling realisation at the document-DB layer.
- patterns/upload-then-poll-indexing — the API-level realisation.
- patterns/consolidate-database-and-search — the broader remediation pattern for the synchronization tax.