Skip to content

CONCEPT Cited by 1 source

NDJSON streaming format

NDJSON (Newline-Delimited JSON; also called JSON Lines or jsonl) is a wire format where each record is a single self- contained JSON object on its own line, separated by \n. Unlike a JSON document (one top-level array or object containing many records), NDJSON is a stream of independent records.

{"op":"insert","table":"sales_orders","ts":"2026-04-21T10:00:00Z","row":{...}}
{"op":"update","table":"inventory","ts":"2026-04-21T10:00:01Z","row":{...}}
{"op":"delete","table":"financial_tx","ts":"2026-04-21T10:00:02Z","row":{...}}

Why it suits streaming CDC

Three operational properties fall out of the line-delimited framing:

  • Incremental parsing without unbounded buffers. A consumer can split the byte stream on \n and parse each line as it arrives, without holding the entire payload in memory or waiting for a closing ].
  • Partial-write resilience. A TCP write that truncates mid-stream will cut off at a line boundary (leaving the last partial line to be discarded or retried), rather than producing a syntactically invalid JSON document that can't be parsed at all.
  • Append-friendly on disk. Writing a new record is a pure append (echo "$record" >> log.ndjson); no need to rewrite a closing bracket. Log files, audit trails, and CDC fan-out all benefit.

Compare with:

  • JSON array [ {...}, {...} ] — requires buffering the whole array before producing valid output; can't tail or append.
  • Binary formats (Protobuf, Avro, Cap'n Proto) — more compact + faster but require schema negotiation; NDJSON's schemaless text payload survives schema drift at the cost of size.
  • Line-delimited CSV — one record per line but flat-tabular; NDJSON natively carries nested structure (maps + arrays).

Seen in

Last updated · 476 distilled / 1,218 read