Skip to content

REDPANDA 2026-03-31

Read original ↗

Redpanda — Redpanda 26.1 delivers the industry's first adaptable streaming engine

Summary

2026-03-31 Redpanda product-launch post for Redpanda 26.1 / Redpanda One (R1), whose headline is the General Availability of Cloud Topics — the per-topic object-storage substrate previewed as beta in sources/2025-11-06-redpanda-253-delivers-near-instant-disaster-recovery-and-more|25.3. Framed as "the industry's first adaptable streaming engine: a single, multi-modal platform that lets you toggle between single-digit-millisecond performance and massive-scale efficiency at the topic level." Four additional load-bearing features: Group-Based Access Control (GBAC) for IdP-federated authorization, ranked rack preferences for Leader Pinning + Cross-Region Remote Read Replicas for AWS as a paired global-data-plane shape, Schema Registry contexts + custom schema metadata, and Profile-Guided Optimization (PGO) for 10-15% efficiency improvement on small message batches. Also named: JSON improvements for Iceberg Topics (sub-schemas, nullable fields), automatic ghost-node ejection, and FIPS 140-3 compliance.

Tier-3 borderline include on vocabulary-canonicalisation grounds. Launch-voice throughout with marketing flourish ("One engine. Every workload. No more sprawl.", "Sacrificing brains for budget", "security for grown-ups"), but six primitives missing from prior wiki coverage — pass-through write to object store, disk-lite architecture, GBAC, ranked rack preferences, Schema Registry contexts, profile- guided optimization for streaming brokers — and the Cloud Topics GA graduation from the 25.3 beta disclosure. Canonical positioning against WarpStream-style diskless architectures ("diskless isn't riskless") — first wiki disclosure of Redpanda's architectural differentiation vocabulary for its own shape vs the whole-cluster-on-S3 shape.

Key takeaways

  1. Cloud Topics is now GA, with Redpanda naming its architecture "disk-lite" in explicit opposition to "diskless". Verbatim: "Cloud Topics use a pass-through write model that saves the bulk of your messages directly to object storage. We stream the heavy message payloads directly to S3 or GCS, but we keep the brains of the operation—the metadata and Raft consensus—on high-performance local NVMe." Load-bearing differentiator framing: "But diskless isn't riskless. By moving everything off of local disks in favor of object storage and external databases, these systems sacrifice data integrity, availability, and core Kafka features (like transactions) on the altar of cost." New wiki concept: concepts/pass-through-write-to-object-store. New wiki pattern: patterns/diskless-disk-lite-hybrid-streaming. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  2. Cost claim canonicalised: >90% lower networking costs + no broken transactions + no metadata lag + no external control plane dependencies. Verbatim: "You get over 90% lower networking costs and the economics of a diskless system, while keeping the battle-hardened reliability of a local-disk broker that can safely run low-latency workloads alongside these topics. No broken transactions. No metadata lag. No external control plane dependencies. Just efficient, Raft-consistent streaming." The 90% figure is new vs the 25.3 preview's unquantified "virtually eliminates the cross-AZ network traffic" framing — concrete percentage disclosure that extends concepts/cross-az-replication-bandwidth-cost. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  3. Group-Based Access Control (GBAC) replaces per-user ACL micromanagement with IdP-federated group-to-role mapping. Verbatim: "With 26.1, we're introducing Group-Based Access Control (GBAC). Instead of micromanaging permissions for every single 'Bob' and 'Alice' in your organization in each cluster, you can now map roles or permissions to groups provided centrally by an Identity Provider (IdP). Access control that automatically absorbs your organization's changes delivers a 'set it and forget it' model for authorization that finally brings sanity to enterprise-scale streaming security." New wiki concept: concepts/group-based-access-control. New wiki pattern: patterns/idp-group-mapped-authorization. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  4. Ranked rack preferences turn Leader Pinning from a single- location hint into an ordered fallback list. Verbatim: "Redpanda 26.1 introduces ranked rack preferences for Leader Pinning, allowing you to deterministically list which regions and AZs should host your partition leaders. It turns leader placement from a game of chance into a strategic advantage. You can actively shape traffic by pinning leaders to the specific locations where your producers live, eliminating much of all of the ingress costs for these topics." Extends the leader pinning primitive canonicalised in sources/2025-02-11-redpanda-high-availability-deployment-multi-region-stretch-clusters|25.2 from "pin to region X" to "prefer region X, then Y, then Z on failover." New wiki concept: concepts/ranked-rack-preferences. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  5. Cross-Region Remote Read Replicas for AWS via S3 decouple global read fan-out from the production cluster. Verbatim: "Cross-Region Remote Read Replicas for AWS allow you to decouple your global read traffic from your production cluster. By accessing topics through S3 instead of the network, you can serve reads in AWS regions worldwide without putting any workload on the production cluster, at a much lower cost than using a 'stretch' cluster. It's the easiest way to scale reads globally without the operational overhead stretch or multi-cluster replication." Extends Remote Read Replica (same-region read-decoupling via object store) to cross-region fan-out. Positioned against stretch clusters and multi-cluster replication for read scaling. AWS-only at launch. New wiki pattern: patterns/cross-region-read-via-object-storage. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  6. Schema Registry contexts add namespace boundaries to schemas. Verbatim: "Contexts allow you to namespace your schemas, making it easy to isolate environments, perform complex migrations, and manage multi-team registries." New wiki concept: concepts/schema-registry-context. Extends concepts/schema-registry which previously covered the single-namespace default shape. Three named use cases — environment isolation (dev/staging/prod), complex migrations (old-schema-context + new-schema-context co-existing), multi-team registries (one physical registry, many logical namespaces). (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  7. Custom schema metadata turns Schema Registry into a first-class data-governance substrate. Verbatim: "You can now attach arbitrary metadata properties to your schemas, turning Redpanda Schema Registry into a first-class citizen in your data governance and observability stack." Extends concepts/schema-registry from pure schema-version-tracking to a key-value governance annotation layer (owner, SLA, sensitivity classification, lineage tags). (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  8. Profile-Guided Optimization (PGO) delivers 10-15% efficiency improvement on small message batches. Verbatim: "We've squeezed even more performance out of the Redpanda Streaming engine by optimizing the binary based on real-world code execution paths, resulting in a 10-15% efficiency improvement for small message batches." First wiki disclosure of PGO applied to a streaming broker binary. Links to Redpanda's earlier deep-dive post "Supercharging streaming: Profile- guided optimization" (not itself ingested). Small-batch specificity matters: small-batch workloads are CPU-bound on per-message overhead rather than throughput-bound on batch assembly, so PGO's branch-prediction-improvement target is well-matched. New wiki concept: concepts/profile-guided-optimization. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  9. Iceberg Topics get JSON sub-schemas, nullable fields, and other enhancements for translating complex JSON structures into a clean bronze layer in Apache Iceberg. Verbatim: "Iceberg Topics now support JSON sub-schemas, nullable fields, and other enhancements for translating complex JSON structures into a clean bronze layer in Apache Iceberg." Extends Iceberg Topics — 26.1's third round of enhancements after the 25.1 GA and the 25.3 BigLake integration. Nullable-field support is the schema-evolution-friendly variant that lets JSON-origin schemas add fields without rewriting existing Parquet files. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  10. Automatic ghost-node ejection cleans up cluster state after node departures. Verbatim: "Redpanda now automatically cleans up after 'ghost' nodes that have left the cluster, keeping your cluster state pristine." First wiki disclosure of the phenomenon — a node that has left the cluster but whose metadata references persist in cluster state, causing drift between the node-presence ground truth and the cluster's internal view. Automatic cleanup is the reliability-substrate equivalent of concepts/explicit-teardown-on-completion at the cluster- membership altitude. New wiki concept: concepts/ghost-node-ejection. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

  11. FIPS 140-3 compliance lands as an incremental upgrade from 25.2's FIPS 140-2 work. Verbatim: "For those of you in high-compliance environments, we're keeping things locked down and government-grade." Extends the FIPS 140-2 cryptographic- boundary work canonicalised in the 2025-05-20 FIPS post to the NIST FIPS 140-3 standard (the successor to 140-2, aligned with ISO/IEC 19790:2012). Extends concepts/fips-140-validated-cryptographic-module from 140-2 coverage to 140-3. (Source: sources/2026-03-31-redpanda-261-delivers-the-industrys-first-adaptable-streaming-engine)

Architecture: the "disk-lite" vocabulary

The post introduces a named architectural category — "disk-lite" — positioned between Redpanda's traditional NVMe-heavy topics and the WarpStream-style fully-diskless shape:

"Cloud Topics use a pass-through write model that saves the bulk of your messages directly to object storage. We stream the heavy message payloads directly to S3 or GCS, but we keep the brains of the operation—the metadata and Raft consensus—on high-performance local NVMe."

The design split:

Layer Substrate Why
Message payload (bulk bytes) Object storage (S3/GCS) Cheap, multi-AZ durable, no cross-AZ replication billing
Metadata + Raft consensus Local NVMe Single-digit-ms leader elections, transaction state, Kafka feature compatibility

This is structurally distinct from WarpStream's shape, which moves metadata to an external metadata store / coordination service as well as the data. The post's framing: "no broken transactions, no metadata lag, no external control plane dependencies." Each of those three phrases is a jab at whole-cluster diskless architectures.

Confluent positioning (repeated from 25.3)

"Most vendors solve the problem of high-volume, latency- tolerant data by telling you to spin up a specialized diskless cluster. That sounds fine until you're managing three different technologies, three different security models, and three different ways to do observability."

"It's one engine, one API, and zero unnecessary clusters. R1, ftw!"

Reiterates the 25.3-era framing ( per-topic workload class) against Confluent's Kora + WarpStream multi-cluster shape. "One engine. Every workload. No more sprawl."

Global data plane: two-feature shape

The post frames ranked rack preferences + Cross-Region RRR as a paired write-path + read-path solution:

  • Write-path: ranked rack preferences → leaders live where producers live → zero ingress cost from producer region.
  • Read-path: Cross-Region RRR → reads served from S3 in consumer regions → zero read load on production cluster + no stretch-cluster operational overhead.

"Your data travels for business, not pleasure. Stop letting it run up expenses on the scenic route and fly direct."

Together they replace two older shapes: stretch clusters (expensive cross-region Raft quorum) and multi-cluster replication (MM2 / Shadowing operational cost). Cross-Region RRR is AWS-only at launch — the post discloses no timeline for GCP / Azure.

Release-note roundup

The post explicitly separates headline features from the "everything else" laundry list:

"We couldn't fit everything into one post without it becoming a novella, but 26.1 is packed with updates that make life easier for the on-call engineer:"

  • Schema Registry contexts
  • Custom schema metadata
  • JSON improvements for Iceberg Topics (sub-schemas, nullable fields)
  • Automatic ghost node ejection
  • Profile-guided optimization (10-15% small-batch efficiency)
  • FIPS 140-3 compliance

"For the full list of nitty-gritty details, head over to our release notes."

Release-note link points to docs.redpanda.com/current/get-started/release-notes/redpanda/ — not itself ingested.

Cross-source continuity

  • Direct sequel to [[sources/2025-11-06-redpanda-253-delivers-near-instant-disaster-recovery-and-more|25.3 release preview]] — that post previewed Cloud Topics as beta; this post GAs it, adds the 90% cost figure, introduces the "disk-lite" vocabulary, and adds the WarpStream-displacing differentiator framing.
  • Extension of [[sources/2025-02-11-redpanda-high-availability-deployment-multi-region-stretch-clusters|2025-02-11 multi-region stretch clusters]] — that post introduced leader pinning + Remote Read Replicas as same-region primitives; 26.1 extends both to cross-region (ranked rack preferences + Cross-Region RRR).
  • Extension of [[sources/2025-05-20-redpanda-implementing-fips-compliance-in-redpanda|2025-05-20 FIPS post]] — that post covered FIPS 140-2; 26.1 ships FIPS 140-3.
  • Extension of the Iceberg Topics corpus — [[sources/2025-05-13-redpanda-getting-started-with-iceberg-topics-on-redpanda-byoc|2025-05-13 BYOC beta]] → 2025-04-07 GA → 25.3 BigLake integration → 26.1 JSON sub-schemas + nullable fields.

Caveats

  • Launch-voice throughout. Marketing-flourish density high ("One engine. Every workload. No more sprawl.", "security for grown-ups", "Sacrificing brains for budget", "fly direct").
  • No production numbers. The 90% cost claim is vendor framing, not a production measurement.
  • Zero mechanism depth on GBAC. IdP protocol (SAML? OIDC? SCIM?), token format, group-claim mapping, policy evaluation engine — all undisclosed. "Tired of micromanaging Bob from accounting?" framing gives zero architecture.
  • Ranked rack preferences mechanism undisclosed. Failover semantics when the top-ranked region is unavailable, cost of leader re-election on rank change, compatibility with Raft quorum placement constraints — all unstated.
  • Cross-Region RRR is AWS-only at launch. No disclosure of the underlying mechanism — presumably cross-region S3 bucket-level replication (customer-configured) + region-local Redpanda clusters reading from the replicated bucket, but the post doesn't say.
  • Schema Registry contexts — no mechanism depth. Context- scoping wire-level semantics, backward-compatibility with context-unaware clients, and cross-context evolution constraints all undisclosed.
  • Ghost-node ejection detection mechanism undisclosed. Heartbeat-timeout threshold, metadata-gossip convergence, interaction with Raft membership changes — all unstated.
  • PGO 10-15% figure is small-batch-specific. Broader throughput-workload improvement unstated; PGO-induced binary size bloat unstated.
  • Iceberg Topics JSON enhancements — no format spec. JSON sub-schema registration API, nullable-field Iceberg type mapping, interaction with existing Parquet projections — all undisclosed.
  • Cloud Topics cost figure not workload-typed. "Over 90% lower networking costs" is an aggregate claim; per-workload sensitivity (produce rate, consumer fan-out, cross-AZ consumer distribution) unstated.
  • Redpanda One / R1 branding is new. "R1, ftw!" — the Redpanda One packaging/vision framing first named in the linked Redpanda One multi-modal streaming data engine post (not ingested). Relationship between R1 as marketing-bundle name and 26.1 as release number is not elaborated.
  • Cross-Region RRR latency floor undisclosed. S3 cross-region access latency (40-200 ms depending on region pair) sets a hard floor on the RRR shape's freshness; not addressed.
  • Enterprise license / GA scope ambiguous. Cloud Topics GA'd, but the license-gating disclosure (was it enterprise- licensed as beta? is it enterprise-only at GA?) is unstated.
  • Unsigned (Redpanda default attribution).

Scope disposition

Tier-3 borderline include on vocabulary-canonicalisation grounds. Redpanda is Tier-3 (stricter content filter); post is classic release-announcement voice with marketing flourish ("One engine. Every workload. No more sprawl."). Would fail the generic "product launches are usually skipped" test in isolation.

Passes because:

  1. Cloud Topics GA is a scheduled follow-up to the 25.3 beta preview already ingested — the wiki already has prior art and this post is the GA-disclosure event.
  2. Six new vocabulary primitives missing from prior wiki coverage (pass-through-write-to-object-store, group-based-access-control, ranked-rack-preferences, schema-registry-context, ghost-node-ejection, profile-guided-optimization).
  3. "Disk-lite" vs "diskless" framing first-named — canonical differentiation vocabulary Redpanda uses against its architectural rivals.
  4. Concrete 90% cost figure extends the unquantified 25.3 cost-claim to a vendor-disclosed percentage.
  5. Architecture content ~50% of ~1,300-word body concentrated in the Cloud Topics-architecture section + the GBAC-mechanism framing + the global-data-plane section + the release-note roundup.

Would skip if first Redpanda ingest — vendor-PR density high. Passes because the release-framings ride on top of the already- canonicalised 25.3 Cloud Topics pair + 25.2 leader-pinning corpus + 25.2 FIPS post + 25.1 Iceberg Topics GA.

Source

Last updated · 470 distilled / 1,213 read