Skip to content

SYSTEM Cited by 2 sources

Zalando NER Query Builder

Identity

The NER Query Builder is the Named-Entity-Recognition query- construction middle layer in Zalando's catalog-search substrate. It sits between Catalog API and Base Search, translating the user's raw intent into a concrete Elasticsearch query.

"On top of that sits our full-text search query builder Named Entity Recognition (NER) system, which takes the raw user intent (query text, user filters). Based on recognized entities it promotes implicit filters. Filters could be applied to shrink or expand results. NER system builds an Elasticsearch query and attaches metadata indicating whether the result set looks sparse and might require expansion using our newer neural matching system." (Source: sources/2025-12-16-zalando-the-day-our-own-queries-dosed-us-inside-zalando-search.)

Responsibilities

  1. Entity recognition on user intent. Parse the raw search text + filters to identify named entities (brand names, product categories, colours, sizes, style terms) that can promote implicit filters into the query.
  2. Elasticsearch query construction. Emit a concrete ES query — lexical + structured-filter — against Base Search.
  3. Sparse-result detection. Attach metadata to the query result indicating whether the result set looks sparse and might require expansion via Zalando's newer neural matching system. The decision is made by querying Base Search separately for product counts — an independent ES round- trip to understand how many products match various filter combinations before settling on the final query.
  4. Hot-path caching. "The query builder [...] cache[s] popular queries and filter combinations."

Product-count probe as a load contributor

The product-count probe is a load-bearing mechanic: for each search, the NER query builder issues one or more extra ES queries whose sole purpose is to count matches under candidate filter combinations. At steady state, these are cheap aggregation queries that benefit from coordinator-node caching. Under load, they compound the per-request fan-out — one logical "search" turns into multiple Base Search calls from both Catalog API and NER simultaneously.

The 2025-12-16 incident's pathological caller was not the NER query builder — but understanding NER's probe behaviour matters because it is one of the several legitimate internal callers whose slow-query patterns needed per-caller attribution via X-Opaque-Id so legitimate high-volume probes could be distinguished from pathological internal faceting.

Seen in

Last updated · 507 distilled / 1,218 read