Skip to content

DATABRICKS 2026-10-15

Read original ↗

Sizing Lakebase Search: lessons from a real app and 4 million passages

Summary

A follow-on to the Lakebase Search launch, this post is the sizing/capacity-planning companion: how to pick the compute size for a search bar or agent tool on Lakebase Postgres before production. It uses two workloads — Slide Hub (an internal Databricks app that finds single slides across ~700 internal decks / 23k slide rows via hybrid search, also exposed to agents over an MCP server) as the real-app baseline, and MIRACL English (a 32.9M-passage Wikipedia corpus, subsetted to 1M/2M/4M) as the scale sweep. The central result is a single sizing axis: the cache ratio = (compute file-cache size) ÷ (vector index size). Hit rate follows the cache ratio (not the row count), warm latency follows the hit rate, and each cache-missed 8 KB page costs ~0.3 ms. The second result is a two-stage scaling rule: below ~8 CU the database is the bottleneck (add Lakebase Compute Units); from ~8 CU the app is the bottleneck (add Databricks Apps workers). (Source: this article.)

Key takeaways

  1. Cache ratio is the sizing axis, and it is corpus-size-invariant. Define cache_ratio = file_cache_size / vector_index_size. Hit rate is a function of the cache ratio, not the number of rows: the 2M and 4M MIRACL curves fall on one line. So a sizing test at 2M predicts behavior at 4M — measure your index size, pick a CU for the ratio you want, read hit rate off the curve. Skewed (Zipf) traffic hit 45% at ratio 0.07, 86% at 0.57, 97% at 1.14 (index just fits); uniform (never-repeat) traffic sits a few points lower. (Source: this article.) See concepts/cache-hit-rate, concepts/working-set-memory.

  2. "Size the cache from the vector index, not the row count." Lakebase holds ~1.6 GB of Local File Cache (LFC) per CU; the LFC can use up to 75% of compute memory (1 CU ≈ 2 GB RAM). A cache about as large as the vector index serves 95–98% of pages without hitting storage. The lakebase_ann index grew 5.8 GB @ 1M, 11.5 GB @ 2M, 23.0 GB @ 4M — roughly linear, so the CU you need scales with the index, which with RaBitQ 1-bit codes grows far slower than full-precision vectors would. (Source: this article.)

  3. Latency follows hit rate; one storage miss ≈ 0.3 ms. A miss is one 8 KB Postgres page not in the LFC (one page holds 1–2 embeddings). Measured cost grew ~252 µs/missed-page for vector, ~316 µs for hybrid — ~0.3 ms either way. So a query that misses 2,000 pages pays ~0.5 s. Warm vector p50 fell 187 ms → 4 ms as the ratio went 0.07 → 2.3; hybrid stayed near 800 ms until the ratio approached 1, then dropped to 33 ms. The keyword half of a hybrid query reads the most pages, so it gains most from a full cache. (Source: this article.) See concepts/read-amplification, concepts/sequential-vs-random-io.

  4. With a full cache, warm latency barely grows with the corpus. Holding the cache ratio near 1.0 and scaling the corpus, warm vector latency stayed 11–23 ms from 1M → 4M and warm hybrid stayed near 276 ms. Only cache-missing (cold) queries get slower with corpus size: cold vector 44 ms @ 1M → 133 ms @ 4M, cold hybrid reaching 911 ms @ 4M. This is the honest form of "you don't need the whole index in memory" — RaBitQ (~1 bit/dim, 32× smaller than float32) + IVF (reads only a few partitions) keep compute memory flat; the index is storage-backed, and the cache ratio controls how often you pay the miss. (Source: this article.) See RaBitQ, IVF, concepts/compute-storage-separation.

  5. Two-stage scaling rule: size the query, scale the DB to 8 CU, then scale the app. At Slide Hub load, below ~4–8 CU the Lakebase compute is the limit — a LARGE app with 2× workers adds nothing at 1 CU. At 1 CU the search statement averaged 146 ms and read 14.9M pages from storage (1.5 GB cache); at 8 CU it averaged 34 ms (12 GB cache), ~50% more throughput. From 8 CU the app is the limit: 16 CU with the same MEDIUM app adds only 4% and DB connections sit idle; a LARGE app (4 workers) at 16 CU reached ≥662 req/s where the load generator itself was the bottleneck. (Source: this article.)

  6. Candidate pool size is a cold-start / page-count lever: ship ~200, not 1,000. The candidate limit sets how many pages a query touches, which on a cold cache sets latency. Measured over 20 queries vs the 1,000-candidate reference: 40 candidates → 74.5% top-10 overlap, 5,655 pages; 200 → 90.5% overlap, 10,401 pages; 1,000 → 100% (reference), 28,972 pages. Slide Hub shipped 1,000, moved to 200: keeps 9 of the top 10, reads ~⅓ the pages, halves cold latency. End-to-end a cold search at 200 candidates took 2.9 s (≈1 s endpoint wake + page reads from storage + ~75 ms first-query planning); a repeat took 40 ms. (Source: this article.)

  7. The connection pool is load-bearing; the embedding cache is not. At 50 users, without the pool: 45 req/s, 2.4 s p95; with it: 85 req/s, 0.7 s p95. Each Python worker holds a pool of 8 DB connections (2 workers → 16 conns, 4 → 32). The query-embedding cache made no measurable difference under load. (Source: this article.) See concepts/connection-pool-exhaustion.

  8. Recall is tunable via probes; build the index in quality mode. The lakebase_ann index is approximate (IVF partitions; lakebase_ann.probes sets how many partitions a query reads). At 4M, default 109 probes → recall@10 0.74 in 3 ms; 2,000 probes → 0.95 in 49 ms. Recall was corpus-invariant at default probes (0.74/0.75/0.75 for 1M/2M/4M). An index built with build_mode = 'quality' raised recall at every probe count — 0.80 at default probes (same 3 ms), 0.91 at 500 probes in 13 ms (vs 1,000 probes / 20 ms on the default build), 0.97 at 2,000 probes — at the cost of a 52-min build at 16 CU for 4M passages. (Source: this article.)

  9. Throughput scales with CU twice over (more CPU + bigger cache). Hitting Lakebase directly (no app), skewed-mix throughput on 4M: ~4 q/s @ 2 CU, 15 @ 8 CU, 65 @ 16 CU; at 64 concurrent connections median wait fell 14.5 s @ 2 CU → 5.2 s @ 8 CU → 343 ms @ 16 CU. The uniform (never-reuse) mix peaked at 30 q/s @ 16 CU. Real traffic repeats/clusters, so the warm hit rate — not the uniform worst case — sets steady-state capacity. (Source: this article.)

  10. Cost follows use; keep a small desk open in prod. Lakebase was ~1% of Slide Hub's 30-day bill; a suspended compute incurs no compute cost. A 14-minute in-region load ramp served 109,602 requests error-free and cost $0.13 at list ($0.11 app + $0.02 Lakebase, which autoscaled 0.5–1 CU / 0.21 CU-hr). 1 CU-hr ≈ 0.213 DBU ≈ $0.085 Premium / $0.11 Enterprise (AWS US East). Recommendation: set prod minimum to 0.5 CU (~$31–40/mo) rather than scale-to-zero, so searches don't wait ~1 s for wake + a cold cache; let dev/test branches scale to zero. Self-hosting the embedding of the whole 32.9M corpus on one A10G GPU (379 passages/s) took 2.95 GPU-hours vs $48.87 for a model-serving endpoint. (Source: this article.) See concepts/scale-to-zero.

Operational numbers

Slide Hub throughput sweep (p50 latency @ 500 users)

Lakebase CU App req/s p50
1 CU MEDIUM / 2 workers 196 1.8 s
0.5–1 CU LARGE / 4 workers 193 1.6 s
2 CU MEDIUM / 2 workers 201 1.8 s
4 CU MEDIUM / 2 workers 219 1.6 s
8 CU MEDIUM / 2 workers 336 1.0 s
16 CU MEDIUM / 2 workers 349 1.0 s
16 CU LARGE / 4 workers ≥662 343 ms

Candidate-pool sweep (20 public queries)

Candidates Cold First-touch Repeat Pages Top-10 overlap
40 127 ms 13 ms 5.0 ms 5,655 74.5%
100 240 ms 16 ms 7.9 ms 7,355 89%
200 387 ms 24 ms 11.7 ms 10,401 90.5%
1,000 740 ms 109 ms 35.3 ms 28,972 100% (ref)

Cache ratio → skewed hit rate (MIRACL 2M/4M, one curve)

Cache ratio Skewed hit rate
0.07 45%
0.57 86%
1.14 (index just fits) 97%

Other key figures

Metric Value Notes
LFC per CU ~1.6 GB up to 75% of compute memory
1 CU ~2 GB RAM + CPU share + local SSD
CU range 0.5–64 CU (autoscale), 112 CU fixed max autoscale ≤ 16 CU above min; scale-to-zero ≤ 32 CU
MEDIUM app ≤2 vCPU / 6 GB 2 workers → 16 conns
LARGE app ≤4 vCPU / 12 GB 4 workers → 32 conns
lakebase_ann index size 5.8 / 11.5 / 23.0 GB at 1M / 2M / 4M
Miss cost ~0.3 ms/page 252 µs vector, 316 µs hybrid
Warm vector p50 187 ms → 4 ms ratio 0.07 → 2.3
Warm hybrid p50 ~800 ms → 33 ms drops as ratio → 1
4M bulk COPY load 19 min @ 8 CU all 4M rows, binary COPY
quality index build 52 min @ 16 CU 4M passages
Embedding corpus (32.9M) 2.95 GPU-hr (1× A10G, 379 pps) vs $48.87 endpoint gte-large-en-v1.5, 1024-dim
Lakebase share of bill ~1% of Slide Hub 30-day scales to zero in prod

Caveats & unknowns

  • Vendor sizing post. All numbers are Databricks' own on their own app + a public corpus; configurations are self-selected. The point is the method (cache ratio, 0.3 ms/miss, two-stage scaling), which is testable on your own data. See concepts/benchmark-methodology-bias.
  • Recall depends on data + embedding model. 0.74 default / 0.80 quality at 4M are specific to MIRACL + gte-large-en-v1.5; the post says to measure on your own data and pick the probe count your app needs.
  • Similarity gate / RRF constants are app-specific. Slide Hub uses RRF at k=60, 200 candidates/method, a 0.70 vector similarity gate, title-tier weighting, notes weight 0.5, and scope filters — tuned for this app, not universal.
  • Numbers are from the same cloud region as the app. Cold "connection waits ~1 s for the endpoint to wake" and ~75 ms first-query planning are in-region; cross-region adds network latency on top.

Extracted entities

Source

Last updated · 854 distilled / 2,448 read