PATTERN Cited by 1 source

Semantic template cache with generative fallback¶

Use an embedding-indexed cache of generalized executable templates as the fast path for repeated semantic intent, but retain expensive authoritative generation as the fallback and as the cache's source of new entries. The pattern avoids treating a cache miss as an error: misses are how the system learns new query shapes. (Source: sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates)

Flow¶

request ──> embed question ──> semantic template retrieval ──> acceptable match?
                                                             │
                              yes ───────────────────────────┤ no
                              ▼                              ▼
                       bind + execute                  authoritative generation
                              │                              │
                    result sufficient?                 successful execution?
                       │          │                         │
                     yes          no                        yes
                       ▼           └───── fallback ───────> generalize + embed + insert
                 summarize response

Structural pieces¶

Template record — generalized structure, original-question embedding, and typed placeholders.
Semantic retrieval — same embedding model for new questions and cache records, with a domain-calibrated acceptance threshold.
Safe execution — extract, validate, and prepared-statement-bind current slot values before querying live data.
Sufficiency gate — reject template results that do not answer every requested dimension.
Authoritative fallback — full-context generation supplies correctness and handles novel questions.
Reinforcement loop — successful authoritative queries expand the cache under actual user demand.

Why this differs from ordinary response caching¶

The cached artifact is neither the question string nor the answer. It is an intent-preserving SQL structure that can execute against changing data. Retrieval needs semantic similarity because paraphrased requests should share a template; execution needs live query evaluation because cached results would stale. (Source: sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates)

Trade-offs¶

Hits are cheap; misses are deliberately a little more expensive. The article's miss path includes a sufficiency check in addition to normal generation, so value is measured over blended traffic.
Coverage grows organically but can acquire bad templates. Template quality, insertion criteria, deduplication, invalidation, and rollback should be explicit operational concerns; the source only specifies successful execution and valid results as the promotion bar.
Retrieval precision controls safety. A wrong-but-similar template is more dangerous than a cache miss; threshold, reranking, and sufficiency validation form a layered defense.

Seen in¶

sources/2026-08-13-aws-reducing-text2sql-latency-with-parameterized-query-templates — AWS Text2SQL reference architecture; reported ~60% hit rate after two weeks and a roughly 80% cache-hit latency reduction.

patterns/analytical-intent-retrieval — retrieves historical query knowledge to inform generation; this pattern executes a selected template directly.
patterns/cheap-approximator-with-expensive-fallback — same fast-path/authoritative-fallback economics, but this pattern's acceptance signal is semantic-template suitability rather than calibrated prediction uncertainty.
patterns/parameterized-template-execution-with-entity-validation — safe materialization of a selected template.