CONCEPT Cited by 1 source

SQLancer logic bug¶

Definition¶

A SQLancer logic bug is a class of DBMS bug in which a query returns a result set that is logically incorrect — e.g. the DBMS omits rows that should have been returned, returns rows that shouldn't have been, or computes an aggregate incorrectly — relative to a mathematical oracle derived from the SQL standard's semantics.

The term comes from SQLancer (Synthesized Query Lancer), a DBMS tester that finds logic bugs via three oracle techniques:

PQS — Pivoted Query Synthesis. Generate queries that must return a specific pivot row; if the DBMS's result omits it, that's a bug.
NoREC — Non-optimising Reference Engine Construction. Rewrite the query into an equivalent form that can't trigger the same optimisation path; if results differ, one of the two plans is buggy.
TLP — Ternary Logic Partitioning. Split a query into three logically-partitioned subqueries (WHERE p, WHERE NOT p, WHERE p IS NULL); the union should equal the unfiltered query. If it doesn't, a bug.

SQLancer has been highly effective on standards-compliant DBMSes — SQLite, PostgreSQL, MariaDB, CockroachDB — where the SQL standard is the oracle. See the SQLancer README:

"SQLancer (Synthesized Query Lancer) is a tool to automatically test Database Management Systems (DBMSs) in order to find logic bugs in their implementation. We refer to logic bugs as those bugs that cause the DBMS to fetch an incorrect result set (e.g., by omitting a record)."

Why SQLancer is wrong substrate for MySQL-compatible DBMSes¶

Per Arvind Murty's 2024-04-09 intern retrospective:

"Vitess ideally should perfectly mimic MySQL, quirks included. SQLancer on the other hand compares queries to an oracle, which determines if queries are logically correct." (Source: sources/2026-04-21-planetscale-summer-2023-fuzzing-vitess-at-planetscale)

This distinction is load-bearing. A MySQL-compatible engine like Vitess must return results identical to MySQL's — quirks, edge cases, implicit coercions, and all. If MySQL happens to compute an expression in a way the SQL standard wouldn't bless, the compatible engine must compute it the same way anyway.

Consequence: SQLancer-style logic-bug testing would flag MySQL-bug-for-bug-compatible behaviour in Vitess as a bug, and would miss Vitess bugs where Vitess disagrees with MySQL on genuine MySQL-quirk behaviour. The right fuzzer for a MySQL-compatible engine is differential fuzzing against MySQL itself — same query, both engines, byte-for-byte comparison, any divergence is a Vitess bug (or, rarely, a MySQL bug worth upstreaming).

Relationship to MySQL-compatibility bugs¶

The two bug classes are disjoint:

Logic bug: DBMS result ≠ mathematically correct result (per SQL standard). Detected by SQLancer-style oracles.
Compatibility bug: target DBMS result ≠ reference DBMS (MySQL) result. Detected by differential fuzzing.

A query can be a compatibility bug without being a logic bug (MySQL has a quirk the target didn't replicate), and can be a logic bug without being a compatibility bug (both MySQL and the target agree on a wrong answer — MySQL's own quirk propagated into the clone).

The Vitess evalengine fuzzer operates at a third altitude again — it uses two in-process interpreters (AST + VM) as each other's oracles plus MySQL as a third oracle, and any disagreement between Vitess's two interpreters is a Vitess bug; any agreement between them that disagrees with MySQL is sometimes a MySQL bug (see MySQL PR #602 collation fix). See patterns/fuzz-ast-vs-vm-oracle.

VSchema also disqualifies SQLancer¶

Second reason Vitess couldn't adopt SQLancer, per Murty:

"Vitess has the added layer of the VSchema. The VSchema has many added considerations, such as sharding keys, which changes how Vitess plans the query."

SQLancer assumes a single-node DBMS. Vitess's planner makes routing and pushdown decisions based on VSchema metadata that SQLancer's query generator doesn't model. Even if the oracle problem were solved, the input-generation layer would produce queries that miss the sharding-specific planner bugs that are the bulk of Vitess's interesting bug surface.

Seen in¶

sources/2026-04-21-planetscale-summer-2023-fuzzing-vitess-at-planetscale — first canonical wiki disclosure of SQLancer's oracle-testing model and why it's the wrong substrate for MySQL-compatible DBMSes. Murty's retrospective frames the SQLancer decision as an explicit design gate at the start of his internship — the team considered SQLancer, ruled it out on two grounds (logic-oracle vs. MySQL-compatibility- oracle mismatch + VSchema not modelled), and built a bespoke differential fuzzer instead. Canonical framing: "We decided to go for the low-hanging fruit and build our own random query generator."