Skip to content

SYSTEM Cited by 8 sources

Redpanda BYOC (Bring Your Own Cloud)

Redpanda BYOC (Bring Your Own Cloud) is Redpanda's deployment model where the data plane — the Redpanda broker cluster itself — runs inside the customer's own cloud account / VPC, while Redpanda operates the control plane as a managed service. Launched ~2019 after a year-long rewrite of Redpanda's first cloud offering; the founder-voice retrospective positions BYOC as the direct precursor of Redpanda's agent-infrastructure thesis because it keeps private data inside the firewall while still providing a fully managed service.

Product page: redpanda.com/product/bring-your-own-cloud-byoc.

Load-bearing design tenet: Data Plane Atomicity

BYOC's central architectural invariant is Data Plane Atomicity: no deployment can bring down any other deployment, including under a control-plane outage, and the data path at every customer runs with zero runtime dependency on externalised services — no external consensus, secret managers, databases, offset-recording services, or metadata lookups as preconditions to writing durably to disk.

Gallego's canonical statement (Source: Gallego 2025-04-03):

"All failures on all data planes are independent for the data path while still providing a fully managed service. No externalized consensus algorithm, secret managers, no external databases, no external offset recording service, or metadata look up as you are trying to write your data durably to disk. At worst, you wouldn't be able to click through an upgrade process, but your data path would always be up — hooray! (And without getting paged at 3 AM.)"

The mechanism-depth disclosure is in Redpanda's 2021 Data Plane Atomicity post (not yet in the wiki corpus; future ingest candidate).

Origin story (founder-voice)

From the 2025-04-03 autonomy post (Source: Gallego 2025-04-03):

"Six years ago, we spent a year writing the first version of our cloud, then threw it away and wrote it again, which led to the birth of what we call BYOC. At the time, this decision was controversial because we were not charging incremental dollars to resell Amazon infrastructure, simply the value of the software we delivered. In a small way, we helped change how infrastructure is delivered and how we wish to consume it."

"The goal was to have a single deployment model that we could build on top for years to come that would be compatible with the strictest security and compliance requirements."

The commercial-model departure is load-bearing in the origin: BYOC skipped the "resell-AWS-with-markup" SaaS playbook; Redpanda charges for the software only. The customer pays their cloud provider directly for the infrastructure.

Redpanda SQL on BYOC (2026-05-27 GA)

Redpanda SQL reached GA on 2026-05-27 for Redpanda BYOC on AWS, consumption-based plans only (Source: 2026-05-27 Redpanda SQL is GA). The Oxla MPP query engine deploys onto the existing BYOC cluster; activation is three steps with no cluster restart from the Redpanda Console overview page. "No new cluster. No broker restart."

This extends BYOC's load-bearing data-residency story from the storage tier to the analytical-compute tier:

"Redpanda SQL runs inside your existing BYOC cluster, on the same infrastructure as your streaming data brokers and Iceberg storage. […] Redpanda SQL runs on the same infrastructure as your brokers, inside your VPC, and every query accesses data in-place, in both the hot (stream) and cold (Iceberg table) tiers. Nothing is sent to a third-party compute service, which is critical if you have compliance requirements (and just as important for strong cybersecurity hygiene), working within your existing infosec-approved environment."

"Regulated data that cannot egress to an external SaaS provider can now be queried directly within your VPC, without procuring a separate query engine or moving data across providers, regions, or network zones. The data stays in your environment. Your data doesn't need to travel to be queryable."

Pre-2026-05-27, BYOC kept the broker + storage in the customer's VPC, but analytical queries had to egress to an external SaaS warehouse (Snowflake / Databricks / BigQuery) — a structural gap in the data-residency story for regulated workloads. Redpanda SQL on BYOC closes that gap. The compliance / no-egress / cybersecurity- hygiene argument now covers the full query path, not just the storage path.

Canonical wiki framings of this property:

Combined with the Iceberg Topics on BYOC beta (2025-05-13) and Redpanda's existing in-VPC streaming substrate, the 2026-05-27 GA delivers a complete in-VPC data platform substrate: brokers + Iceberg storage + analytical SQL + connectors all running inside the customer's own cloud account, managed remotely by Redpanda but with no data egress for query.

GA scope is initially narrow — AWS BYOC consumption-plan only; GCP BYOC + BYOVPC are "coming soon"; Self-Managed targets 2H FY27.

Iceberg Topics on BYOC (2025-05-13 beta)

Five weeks after the Iceberg Topics GA on Redpanda Cloud Dedicated (2025-04-07), Redpanda extended the feature to BYOC as a beta (Source: sources/2025-05-13-redpanda-getting-started-with-iceberg-topics-on-redpanda-byoc).

The combination is architecturally significant because BYOC's customer-VPC data plane and Iceberg Topics' object-storage data output compose into a data-ownership compound property canonicalised as concepts/byoc-data-ownership-for-iceberg: both the Parquet data files and the Iceberg metadata land in the customer's own bucket, under the customer's IAM, KMS keys, and lifecycle rules. Verbatim framing:

"For BYOC customers who already control their own object storage buckets, this means full control of your Iceberg data with zero compromises." (Source: sources/2025-05-13-redpanda-getting-started-with-iceberg-topics-on-redpanda-byoc)

The BYOC beta introduces three named capabilities vs the GA:

  • Self-service Iceberg config at the cluster level via rpk CLI or Cloud HTTP API.
  • File-based catalog as a first-class alternative to REST catalog sync (concepts/iceberg-file-based-catalog) for engines like BigQuery that read Iceberg via metadata-pointer DDL. Reframes what the GA post called the "built-in object-store catalog fallback" from last-resort to primary integration option.
  • Secure credential handling (Iceberg REST catalog secrets).

2× partition density in 25.1

Adjacent BYOC substrate improvement disclosed in the same 2025-05-13 post: supported partition count per cluster tier roughly doubled in 25.1 thanks to per-partition memory-efficiency improvements.

  • Tier 1: 1,000 → 2,000 partitions
  • Tier 5: 22,800 → 45,600 partitions

Verbatim framing: "Redpanda BYOC now supports double the partition limits across most tiers, thanks to improved partition memory efficiency." With the caveat: "existing clusters may not yet support these partition counts if they haven't been upgraded to 25.1." Canonicalised as concepts/broker-partition-density.

Operator implications (marketing-voice from the source, load- bearing as capability statements):

  • Scale further on smaller clusters → cost efficiency.
  • More producers / consumers per topic → parallelism headroom.
  • Capacity headroom for rising data volumes.

No disclosure of the underlying memory-efficiency mechanism.

Relationship to other Redpanda deployment shapes

  • Redpanda Self-Managed — the customer runs everything, no Redpanda control plane. No managed-upgrades, no managed-observability. BYOC adds the managed control plane while preserving the data-plane- in-customer-VPC topology.
  • Redpanda Cloud Dedicated — Redpanda runs both control plane and data plane in its own account. Classic managed-SaaS shape. Cross-region stretch clusters are supported here (Source: sources/2025-02-11-redpanda-high-availability-deployment-multi-region-stretch-clusters).
  • Redpanda Serverless — fully managed, multi-tenant, pay-per- use. See redpanda.com/product/serverless.

BYOC is the hybrid: customer owns data-plane infra, Redpanda owns control-plane operations. The control-plane-outage tolerance from Data Plane Atomicity is what makes the hybrid clean.

Why BYOC matters for agent infrastructure

Gallego's 2025-04-03 reframing: BYOC's data-plane-in-customer-VPC topology is the substrate for sending models to the data:

"Our insight was that since the dataplane is already deployed within the customer's network, it is used to connect the internal applications already with Redpanda Connect's portfolio."

If the Redpanda Agents SDK is deployed alongside the BYOC broker, the entire enterprise-agent pipeline — durable log, Redpanda Connect connectors, MCP server, Python SDK, small- model inference — runs inside the customer's firewall. Only the frontier-model orchestration calls (which see plans, not bulk private data) cross the boundary. See concepts/frontier-model-minion-delegation.

Production scale

Single data point disclosed in Kinley's 2024-11 batch-tuning case study: a 2024 Redpanda Cloud BYOC customer ran 1.75–2 GB/sec, several million msg/sec split across two Tier-7 clusters (later consolidated to one at ~50% CPU after three rounds of linger tuning). This is the canonical wiki BYOC production-scale data point.

Other content

Redpanda's three-cloud BYOC availability (AWS / GCP / Azure) is the deployment surface for the 2025-04-03 Redpanda Agents SDK preview.

Caveats

  • Mechanism depth deferred. The 2025-04-03 source post is a founder-voice retrospective framing — the implementation-level disclosure lives in the 2021 Data Plane Atomicity post (not yet ingested).
  • Kubernetes deployment restriction. Per the 2025-02-11 HA post, Self-Managed on K8s is multi-AZ only; multi-region stretch clusters require VMs / bare metal / cloud compute / Redpanda Cloud Dedicated + BYOC. (BYOC supports the stretch topology.)
  • Control-plane operational scope opaque. What Redpanda's managed control plane covers (upgrades, observability, support) vs what the customer retains is not enumerated in the source posts.
  • Single production customer datum. The 1.75-2 GB/sec BYOC number from the 2024-11-26 case study is one customer; fleet distribution of BYOC workloads is not disclosed.

Seen in

Last updated · 542 distilled / 1,571 read