CONCEPT Cited by 1 source
Semantic-cache precision-recall threshold¶
A semantic-cache precision-recall threshold is the minimum similarity score at which a vector-retrieved cached artifact is trusted as reusable. It controls a different risk than ordinary cache eviction: it determines whether two differently worded requests have sufficiently similar intent and structure to share an executable template. (Source: sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates)
Operating trade-off¶
| Threshold posture | Retrieval result | Consequence |
|---|---|---|
| Too strict | Rejects valid paraphrases | More authoritative LLM generation, lower cache hit rate, higher cost and latency |
| Too loose | Admits a related but structurally wrong template | Incorrect query or incomplete answer can pass the fast path |
| Calibrated | Reuses only safely similar templates | Captures cache economics while preserving fallback for ambiguity |
The correct value depends on domain narrowness, template coverage, and the cost of a false match. AWS recommends recording selected templates and scores in production, examining both rejected good matches and accepted bad matches. (Source: sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates)
Retrieve broadly, decide narrowly¶
When one similarity score is not precise enough, first retrieve a wider candidate set at a looser threshold and rerank with a small LLM or specialized reranker. This preserves recall while moving the accept/reject decision to a stronger selector; it does not eliminate the need for result sufficiency checks and authoritative fallback. (Source: sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates)
Seen in¶
- sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates — template selection threshold for a production Text2SQL semantic cache.
Related¶
- concepts/vector-similarity-search — retrieval primitive.
- concepts/semantic-search-retrieval — paraphrase-resilient retrieval.
- patterns/semantic-template-cache-with-generative-fallback — low scores take the authoritative path.