Skip to content

CONCEPT Cited by 1 source

Inquiry Type classifier

Definition

An Inquiry Type classifier is a small fine-tuned model that decides whether a user question is in-scope for an LLM product's domain, and emits a label that either dispatches to the in-scope answering pipeline or gracefully declines + redirects to a more-appropriate surface. It is a pre-retrieval scope gate, a sibling of the Trust & Safety classifier but answering a different question:

Classifier Question answered
Trust & Safety "Is this question safe to answer at all?"
Inquiry Type "Is this question in scope for our system's domain?"

Canonical wiki instance: Yelp's Biz Ask Anything (2026-03-27).

Why it's load-bearing

From the post:

"In our first internal prototype, we implicitly assumed users would ask the 'right' questions — i.e., things a single business page can answer from its Yelp content. That worked in demos. In real traffic, people ask for recommendations, account help, or general knowledge — things our bot wasn't built (or allowed) to answer." (Source: sources/2026-03-27-yelp-building-biz-ask-anything-from-prototype-to-product)

Three failure modes addressed:

  1. Scope drift — "The model will cheerfully answer using its own prior knowledge instead of the business's Yelp content. This nudges users to ask more out-of-scope questions, and we lose control over answer quality because it's no longer grounded in our vetted sources."
  2. Hallucinations — "When the model lacks (or only partially has) relevant knowledge, it still tends to produce a confident answer — filling gaps with fabricated details rather than deferring or asking for clarification." See concepts/llm-hallucination.
  3. Cost & latency — "Fetching evidence and generating a full answer for out-of-scope questions wastes tokens and time."

Canonical redirects

Yelp's example dispatches on reject:

Question Redirect
"Show me good plumbers." Link to Yelp search
"How do I change my password?" Link to Yelp support
"Is there god?" Graceful decline + guidance on business-specific questions

The redirect is as important as the decline — the user should end the interaction with a clearer path, not a dead end.

Training data discipline

Yelp's Inquiry Type classifier:

  • ~7K samples — more than the T&S classifier because the scope decision has more labels and more legitimate edge cases.
  • Realistic seeds from Ask-the-Community + "early- traffic failures" + synthetic paraphrases for variation.
  • "We spent a lot of time mining logs from smaller launches to discover new inquiry types that we hadn't discovered before and re-fine tune for new classes." — label taxonomy is a live artefact, not fixed at launch.

Base model

Yelp fine-tuned GPT-4.1-nano — same small-model base as the T&S classifier. The two classifiers share the same latency + cost motivation for choosing nano.

Parallel-with-T&S architecture

Inquiry Type runs in parallel with T&S. From the post:

"We added an Inquiry Type classifier that runs in parallel with Trust & Safety. If T&S rejects, we cancel the rest of the pipeline (including inquiry typing). Otherwise, this classifier decides whether the question is in-scope for a single business, content-grounded answer."

The T&S-wins-over-Inquiry-Type short-circuit is the cost-saving variant of patterns/parallel-pre-retrieval-classifier-pipeline: even though both classifiers run concurrently, T&S reject cancels downstream work including the still-in-flight Inquiry Type classification.

Tradeoffs / gotchas

  • Label taxonomy drift — the set of inquiry types expands as user traffic surfaces new intents; requires continuous training-data curation.
  • Redirect quality matters — a clumsy redirect worsens user experience vs. a graceful one; the decline template is a product surface worth investing in.
  • Ambiguity zone — questions like "Best Italian near me?" blur in-scope (top-of-list Italian restaurant on this business page) with out-of-scope (general recommendation); the classifier's threshold and the product's scope definition have to co-evolve.
  • Cost — every request pays for Inquiry Type; small-model (nano) choice bounds this.
  • Redirect latency — the decline+redirect path still pays the T&S + Inquiry Type classifier latencies before returning; Yelp doesn't disclose the decline-path p75.

Seen in

Last updated · 476 distilled / 1,218 read