Skip to content

PATTERN Cited by 1 source

JSONL streaming child process

Intent

Embed an AI coding agent (or any long-running LLM tool) as a child process driven by three choices:

  1. Prompt over stdin (not argv) — so large prompts don't hit ARG_MAX / E2BIG.
  2. JSONL over stdout — so the parent can process events in real time and tolerate mid-stream crashes.
  3. Buffered flush on the consumer side — so disk I/O doesn't starve on line-at-a-time writes.

The result is an embedding shape that survives the operational surprises of real LLM workloads: MR descriptions 200 KB long, child processes that OOM mid-reasoning, models that pause silently for minutes.

When to reach for it

  • Embedding a CLI or server-first coding agent (OpenCode, Claude Code, Codex CLI) inside a higher-order orchestrator.
  • Orchestrating multiple concurrent LLM-calling children whose output must be processed incrementally.
  • Building CI-native AI workloads where the parent reacts to events (step_finish, error, session.idle) rather than waiting for final output.

Mechanism

The spawn call (Cloudflare's instance)

const proc = Bun.spawn(
  ["bun", opencodeScript, "--print-logs", "--log-level", logLevel,
   "--format", "json", "--agent", "review_coordinator", "run"],
  {
    stdin: Buffer.from(prompt),                    // prompt via stdin, not argv
    env: {
      ...sanitizeEnvForChildProcess(process.env),
      OPENCODE_CONFIG: process.env.OPENCODE_CONFIG_PATH ?? "",
      BUN_JSC_gcMaxHeapSize: "2684354560",         // 2.5 GB heap cap
    },
    stdout: "pipe",
    stderr: "pipe",
  },
);

The consumer loop

  1. Read JSONL from stdout. One JSON object per line; parse and dispatch.
  2. Buffer writes to disk. Cloudflare flushes every 100 lines or 50 ms — not on every line — "to save our disks from a slow but painful appendFileSync death."
  3. Pattern-match on event type:
  4. step_finish with reason: "length" → model hit max-tokens mid-sentence → retry.
  5. error events → consult the error classifier → decide failback vs. abort.
  6. session.idle → completion signal; backed by 3-second polling.
  7. Emit a thinking heartbeat every 30 s if no new event has arrived — prevents users from misreading deliberation as a hang.
  8. Stderr scanning independently — detect child-process-level retryable errors ("overloaded", "503") that may not surface as structured JSONL events.

Timeouts

Three levels, all enforced by the parent:

  • Per-task: 5 min (10 min for heavier workloads).
  • Overall: 25 min — aborts every remaining session if hit.
  • Retry budget minimum: 2 min — don't start a retry if less than that remains.

Plus 60-s inactivity kill — catches sessions that crash on startup before emitting any JSONL.

Why this shape wins

  • ARG_MAX / E2BIG avoidance is not optional. Cloudflare's own discovery path: "we learned this pretty quickly when E2BIG errors started showing up on a small percentage of our CI jobs for incredibly large merge requests." You will hit this eventually; stdin is the fix.
  • JSONL is tolerable under child-process crash. A closed-JSON-array format leaves unparseable output on disk if the child OOMs; JSONL loses at most one trailing line.
  • Buffered flush avoids pathological I/O. 100-line / 50-ms windows smooth out event bursts into disk-friendly write sizes.
  • Real-time event processing enables reactive control. Retry on reason: "length", heartbeat on silence, failback on error — all decided inside the parent loop, not after the child exits.

Environment hygiene

Two small details worth codifying from the Cloudflare instance:

  • Sanitise env before spawn. sanitizeEnvForChildProcess(process.env) strips anything the parent doesn't want inherited — credentials, auth tokens, unrelated config.
  • Cap child heap explicitly. BUN_JSC_gcMaxHeapSize: "2684354560" (2.5 GB) in Cloudflare's case — prevents a runaway child from consuming parent-process memory indirectly.

Tradeoffs

  • Stdin-only prompts are harder to debug. Can't paste the command line into a terminal to reproduce; have to capture the stdin payload separately. Worth adding a --dump-prompt mode.
  • Line-level buffering adds latency. A 50-ms buffer is a 50-ms UX budget between "event happened" and "event visible". For CI this is invisible; for interactive TUIs it may not be.
  • JSONL requires producer cooperation. If the child doesn't support --format json, the pattern doesn't apply directly.

Sibling patterns

  • vs. patterns/coordinator-sub-reviewer-orchestration — this is the transport shape; coordinator / sub-reviewer is what's transported.
  • vs. direct SDK embedding — if the agent exposes a server-first architecture (OpenCode's model), SDK embedding can replace child-process spawn entirely. Cloudflare actually uses both: orchestrator spawns coordinator as a child process; coordinator spawns sub-reviewers via the OpenCode SDK in-process.

Seen in

Last updated · 200 distilled / 1,178 read