Skip to content

CLOUDFLARE 2026-10-09 Tier 1

Read original ↗

Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash

Summary

One week after launching Clef + Clef-flash, Cloudflare's open-weight decision models, this post ships three updates: Clef-omni, a new decision model that natively ingests audio and video alongside text and image; a cheaper Clef-flash (now cheaper per input token than Jev, bought by cutting the hosted context window 64k → 24k); and a faster Clef achieved at the serving-infrastructure layer (migrating to SGLang, 1.7–2.0× median speedup, no new model weights). Clef-omni is built on a Qwen3-Omni-30B-A3B-Instruct mixture-of-experts backbone — keeping the comprehension path, discarding the text-to-speech output head — and preserves Clef's non-autoregressive, prefill-only, schema-bound decision scoring so every modality is scored in a single API call with no transcription or captioning step. This is a product/model-launch post, but the serving-infra depth (SGLang migration with before/after latency tables + an upstream PR), the multimodal inference architecture (unified sequence, two-stage attention routing over audio/video/image/text), and the explicit context-window-vs-price scaling trade-off (backed by usage data) put it decisively in scope.

Key takeaways

  1. Clef-omni ingests audio + video natively, not via a cascading pipeline. It takes audio (wav/mp3) and video (mp4/webm) alongside text and image in one call — "instead of setting up cascading pipelines of models that transcribe speech-to-text, or splitting audio and image channels from video, you can just call one model to make decisions across any modality." This is the single-call alternative to the classic transcribe-then-index multimodal path (patterns/multimodal-content-understanding). (Source: this article)

  2. The backbone is a Qwen3-Omni MoE with the TTS output head removed. Built on Qwen3-Omni-30B-A3B-Instruct, a mixture-of-experts model "already capable of processing text, imagery, audio, and video directly within a single pipeline" — Cloudflare adopts the primary comprehension backbone while discarding the text-to-speech output components. A decision model needs only to understand input, not generate speech. (Source: this article)

  3. The decision step stays non-autoregressive — one prefill pass, no output tokens. In production Clef-omni "executes a quick prefill pass across the complete payload, scoring all modalities and valid parameter options simultaneously." Because Clef models are not LLMs, they "skip output token generation," removing the overhead of transcribing/captioning incoming files — "media elements map straight into the unified sequence, where video and audio are synced with visual frames for joint processing." This is the same prefill-only, non-autoregressive mechanic as the base Clef, now extended to the audio/video modalities (corroborating source — see taxonomy note). (Source: this article)

  4. Two-stage attention routing + lexical grammar, now over every modality. Clef-omni "pulls candidate values directly from internal embeddings using our two-stage attention routing: every valid option gathers key evidence from the input (regardless if they are buried in text snippets, visual elements, or audio streams) before field vectors cross-attend across the full context to compute confidence scores," with a built-in lexical grammar preserving option semantics for "fast, schema-constrained scoring across every input type." (Source: this article)

  5. Trained with the same recipe as Clef — frozen Qwen backbone + LoRA + calibration losses. "Freezing the Qwen3 backbone, training low-rank adapters (LoRA), and applying our standard post-training approach: combining label-smoothed cross-entropy loss with Brier score calibration" — producing a model "resilient... regardless of schema variations, field ordering, and prompt structures." (Source: this article)

  6. Multimodal latency numbers (single API call): text-only decisions ~130 ms median, image inputs ~150 ms, audio clips a few hundred ms, and a full 21-second video clip with sound scored in ~1.5 s — all in one call. (Source: this article)

  7. Clef-flash got cheaper by cutting the hosted context window 64k → 24k. Clef-flash is now $0.038 per M input tokens (from $0.09), "cheaper than Jev." The trade-off: "the hosted version of Clef-flash now has a context window of 24k, instead of 64k as previously advertised" (concepts/context-window-as-token-budget). The HF weights are untouched (trained to support 256k if self-hosted). Decision backed by usage data: "only 0.24% of requests exceed 24k input tokens." Clef stays at 64k; larger-context users are steered to Clef. (Source: this article)

  8. Clef got faster purely at the serving layer — migration to SGLang. "No new model weights are released because most of the optimizations we made are at the serving infrastructure layer, not materially to the model weights or architecture." The named change: "move to SGLang for serving the model." Before→after median/p95: ~800 tok 262→152 ms (1.7×), ~3,400 tok 616→305 ms (2.0×), ~16,000 tok 2,721→1,635 ms (1.7×) — the measure-at-the-serving-layer-then-reship loop (patterns/measurement-driven-micro-optimization). (Source: this article)

  9. Cloudflare contributed Clef support upstream to SGLang (PR #42721). "We worked with the SGLang team and made pull requests to incorporate Clef (PR #42721), which will be released in SGLang 0.5.22," with self-host launch commands on the HF repo and the SGLang cookbooks — the open-source-PR + in-house-serving dual path (patterns/upstream-contribution-parallel-to-in-house-integration). (Source: this article)

  10. Pricing across the family: Clef-flash $0.038/M input tokens; Clef $0.24/M; Clef-omni (new) $0.15/M. Image and audio modalities are converted to input tokens for pricing (details in developer docs). (Source: this article)

  11. Production use cases (decision model gating/classifying at scale): Cloudflare's GitHub docs repo uses Clef to detect and auto-close spam issues; EmDash (its CMS) moderates plugin libraries for phishing; the DLP team scans for PII like government IDs; threat intelligence detects malicious domains. "The capability to do detections and classifications has moved into the model layer." (Source: this article)

  12. Benchmarks across Clef-omni / Clef / Clef-flash / Jev (Cloudflare- reported): Clef-omni leads on BANKING77 macro-F1 (94.8), CLINC150+OOS (97.7), Amazon ESCI (57.8); Clef-flash leads several case-exact workflows (Home appliances 97.73, API-Bank 93.11); Jev still leads When2Call (80.97), BRIGHT (47.52), Agent-trace observability. Clef- omni holds competitive accuracy "even with new modalities and MoE architecture." (Source: this article)

Systems / concepts / patterns extracted

  • New system? No. Clef-omni is a new member of the existing Clef decision-model family, recorded as a ## section on the Clef page per the systems gate (sub-variant of a system that already has a page). No standalone systems/clef-omni page.
  • Reused systems (gate: reuse first): SGLang — already has a page; this is its third source and the first direct Cloudflare→ SGLang production serving instance (migrated Clef to it + upstream PR #42721 + SGLang 0.5.22). Qwen — Qwen3-Omni-30B-A3B-Instruct MoE backbone (vs the Qwen3.8-27B / Qwen3.5-9B dense-ish backbones of Clef/Clef-flash). Workers AI — hosts the new model + the serving-layer optimizations. EmDash, AI Gateway — named production consumers (prose/frontmatter).
  • Mapped INTO existing concepts (taxonomy gate — no new pages): concepts/mixture-of-experts (Qwen3-Omni MoE backbone — a new deployment shape: MoE inside a non-generative decision model, scoring not token generation), concepts/structured-output-reliability (schema-constrained scoring across modalities), concepts/lora-low-rank-adaptation (frozen- backbone + LoRA recipe reused), concepts/hot-path + concepts/latency-critical-vs-latency-tolerant-workload (edge-GPU single- call decisioning), concepts/batching-latency-tradeoff / concepts/context-window-as-token-budget (the 64k→24k context cut to hit a price point, backed by the 0.24%-over-24k usage statistic).
  • Mapped INTO existing patterns: patterns/multimodal-content-understanding (Clef-omni = the single-call, no-transcription alternative to the classic per-modality cascading extraction pipeline — a decision/scoring instance, not retrieval), patterns/measurement-driven-micro-optimization (serving-layer SGLang swap with before/after median/p95 tables), patterns/upstream-contribution-parallel-to-in-house-integration (SGLang PR #42721 upstream while serving Clef on it in-house), patterns/cheap-approximator-with-expensive-fallback (the decision-model- gating framing carried from the base Clef).
  • Taxonomy gate — corroborated but NOT minted. The Clef-specific mechanics (non-autoregressive decision inference, prefill-only pass, two-stage attention routing, schema-bound scoring, lexical grammar / lexical prior) now have ≥2 sources (this + the 2026-10-01 launch). They remain Clef-implementation mechanics (one model family, one vendor), not reusable cross-article design vocabulary, so per the taxonomy discipline they stay as prose + tags on the Clef page — not promoted to concept pages. Revisit only if a distinct, non-Clef system exhibits the same named mechanic.

Operational numbers

  • Latency (Clef-omni, single call): text ~130 ms median, image ~150 ms, audio a few hundred ms, 21 s video+sound → ~1.5 s.
  • Serving-layer speedup (Clef via SGLang), median/p95 before→after:
  • ~800 tok: 262/438 → 152/351 ms (1.7× median)
  • ~3,400 tok: 616/777 → 305/531 ms (2.0× median)
  • ~16,000 tok: 2,721/3,250 → 1,635/1,805 ms (1.7× median)
  • Pricing per M input tokens: Clef-flash $0.038 (was $0.09); Clef $0.24; Clef-omni $0.15.
  • Context window: hosted Clef-flash 24k (was 64k); HF weights support 256k self-hosted; Clef stays 64k. 0.24% of requests exceed 24k.
  • Backbone: Clef-omni = Qwen3-Omni-30B-A3B-Instruct MoE (comprehension backbone kept, TTS output head discarded).
  • SGLang: PR #42721, shipping in SGLang 0.5.22.
  • Benchmarks (selected, Clef-omni): BANKING77 94.8, CLINC150+OOS 97.7, Amazon ESCI 57.8, BFCL case-exact 98.2, API-Bank 92.7.
  • Inputs: audio wav/mp3, video mp4/webm, plus image + text.
  • License / weights: open-weights on Hugging Face (Cloudflare/clef-omni), Jev-API-compatible, available via AI Gateway.

Caveats

  • Benchmarks are Cloudflare-reported (BFCL / ToolRet / API-Bank / BANKING77 / CLINC150 / BRIGHT / Amazon ESCI / PhishNChips + the Typesafe eval suite); no independent reproduction.
  • The multimodal inference mechanics (unified sequence, two-stage attention routing over audio/video/image, lexical grammar) are described narratively — weights are open, but there is no linked paper for the routing/omni architecture.
  • Qwen3-Omni-30B-A3B-Instruct, Qwen3.8-27B, Qwen3.5-9B are the post's version strings — treat as the article's naming, not necessarily canonical Alibaba release labels.
  • The Clef-flash context-window cut is to the hosted product only; it is a pricing/packaging decision, not a model-capability change (HF weights = 256k).
  • SGLang speedups are median/p95 microbenchmark-style figures from Cloudflare, not an independent production A/B.

Source

Last updated · 854 distilled / 2,448 read