Skip to content

CONCEPT Cited by 1 source

Horizon operator

Definition

The Horizon operator is a placeholder operator in the VTGate query planner's operator tree that packages together the entire "after-the-FROM-clause" portion of a SQL query — SELECT expressions, aggregations, ORDER BY, GROUP BY, and LIMIT — into a single opaque unit that the planner can try to push down wholesale to MySQL before deciding to expand it into its constituent operators.

Andrés Taylor's canonical definition (Source: sources/2026-04-21-planetscale-optimizing-query-planning-in-vitess-a-step-by-step-approach): "A 'horizon' operator contains the SELECT expressions, aggregations, ORDER BY, GROUP BY, and LIMIT. If we can push the entire operator to MySQL, we don't need to plan this at all. If we can't delegate it to MySQL in a single piece, we have to plan these components separately."

Why it exists

In a distributed query planner, the cheapest plan is often the one that pushes the entire query to a single MySQL shard. For queries where the shard key is pinned and the underlying tables live on a single shard, there's no useful work for VTGate to do — parse, route, and get out of the way. The Horizon operator makes this case structurally explicit: keep the post-FROM query fragments bundled together so the planner's first-try rewriter can attempt pushdown on the whole thing, fast.

For queries where pushdown fails — cross-shard joins, scatter-gather, global aggregations, cross-keyspace operations — the Horizon is expanded into its parts, each of which gets its own pushdown attempt.

Lifecycle in the new-model planner

From Taylor's worked example (initial tree):

Horizon
└── ApplyJoin (u.uid = ue.uid)
   ├── Route (Scatter on user)
   │   └── Table (user.user)
   └── Route (Scatter on user)
       └── Filter (:u_uid = ue.uid)
           └── Table (user.user_extra)

Step 1 — Try wholesale pushdown. The Horizon rewriter checks whether the Horizon's SELECT / ORDER BY / GROUP BY / LIMIT / aggregations can be pushed as a single MySQL query into the Routes below. In this example there's an ApplyJoin between Horizon and the Routes — wholesale pushdown fails because the join spans two scatters and can't be expressed as a single MySQL query.

Step 2 — Expand the Horizon in place. If pushdown fails, the Horizon is replaced by its constituent operators. Same query, expanded tree:

Ordering (u.baz asc)
└── Projection (u.foo, ue.bar)
   └── ApplyJoin (u.uid = ue.uid)
       ├── Route (Scatter on user)
       │   └── Table (user.user)
       └── Route (Scatter on user)
           └── Filter (:u_uid = ue.uid)
               └── Table (user.user_extra)

Taylor: "In the next step, we decided that we can't push the Horizon and instead need to expand it into its components. The Horizon is split into an Ordering and a Projection operator." The expanded operators are each independently eligible for subsequent pushdown.

Step 3 — Push down the expanded operators. Each operator below Horizon (Ordering, Projection, Filter, Aggregator, Limit) is individually pushed as far down the tree as possible — into the LHS or RHS of joins, into Routes — by the planner's fixed-point rewriter loop.

Operators the Horizon expands into

  • Projection — SELECT u.foo, ue.bar becomes a Projection over the join output.
  • Ordering / MemorySort — ORDER BY u.baz becomes an Ordering that the planner tries to push to the LHS of the join (if its sort key lives on the LHS).
  • Aggregator (OrderedAggregate / HashAggregate) — GROUP BY + aggregate functions become Aggregator operators. If cross-shard, they get split into local + global halves via local-global decomposition.
  • Filter — HAVING clauses become post-aggregation Filters.
  • Limit — LIMIT becomes a top-level Limit.

Why Horizon-as-placeholder beats Horizon-as-struct

A natural alternative design is: don't create a Horizon operator at all; store the SELECT / ORDER BY / GROUP BY / LIMIT fragments as fields on the plan root and expand them lazily as the planner walks the tree. Vitess rejected this design. The Horizon-as-operator representation has two payoffs:

  1. Preserves the runnable-plan invariant. A Horizon over an ApplyJoin is itself a runnable plan: execute the join below, then apply the Horizon's SELECT / ORDER BY / etc. at VTGate (via evalengine). Executing side-struct fields isn't an "intermediate plan" — it's pre-planning state.
  2. Makes rewriter composition local. Rewriters that want to push the Horizon down see a node in the tree they can match against. No "rewriter
  3. global plan fields" coupling. Single operator-tree type for every planner rewriter.

Relationship to

Seen in

Last updated · 550 distilled / 1,221 read