Skip to content

CONCEPT Cited by 1 source

Virtual Kubernetes cluster

Definition

A virtual Kubernetes cluster is a Kubernetes-cluster-shaped isolation unit — its own API server, controller manager, namespaces, RBAC, CRDs, and DNS — that runs inside another Kubernetes cluster (the host cluster) rather than on its own set of physical nodes. Users of a virtual cluster interact with it exactly as if it were an independent K8s cluster (kubectl → virtual API server, own kubeconfig), but workload pods created inside the virtual cluster are synced down to the host cluster and run on the host's real nodes.

This is the inverse of the classic concepts/control-plane-data-plane-separation split: instead of "shared control plane, per-tenant data plane" (managed K8s where AWS runs the control plane and customers bring nodes), a virtual Kubernetes cluster is control-plane-per-tenant, shared data plane — each tenant gets their own API server, but the real worker nodes are pooled.

Why the primitive matters

  • Sub-5-minute environment provisioning. Spinning up a new virtual cluster is creating a set of pods in the host cluster; no new EC2 instances, no managed-control-plane provisioning (the EKS control plane takes ~15 min to create), no new VPC wiring. Deloitte measured 45 min → <5 min.
  • Eliminates per-environment controller / ingress / monitoring duplication. Essential platform services (Load Balancer Controller, CSI drivers, monitoring agents) run once on the host and are shared across all virtual clusters via the syncer. Dedicated clusters would duplicate all of this per environment.
  • Dissolves the platform-team bottleneck for test environments. QA engineers can create their own virtual cluster via a self-service UI (or CLI) without a ticket to the platform team. Provisioning moves from "human serialisation point" to "platform-exposed contract".
  • Density. Deloitte runs 50+ virtual clusters on a single EKS host cluster — one-physical-cluster-per-fifty-logical- environments.
  • Version independence. A virtual cluster can run a different K8s minor version from the host (within compatibility bounds), so feature-regression testing on a new K8s version doesn't require standing up a separate physical cluster.

What isolation you do and don't get

You do get:

  • Namespace / RBAC / CRD isolation. Each virtual cluster's namespaces, roles, and CRDs are separate. A default namespace in vcluster-A is not the same as default in vcluster-B.
  • DNS isolation. Each virtual cluster runs its own CoreDNS; service discovery inside the virtual cluster doesn't collide with sibling virtual clusters.
  • Config isolation. API-server feature gates, admission controllers, quotas are per-virtual-cluster.
  • API-group isolation. Installing CRD foo/v1 in vcluster-A does not make it available in vcluster-B.

You don't get:

  • Kernel isolation. All virtual clusters share the host kernel. A kernel escape from any pod in any virtual cluster compromises sibling virtual clusters sharing the same host node.
  • Hard resource isolation by default. Without explicit ResourceQuota + LimitRange + PodDisruptionBudget policies on the host, a pod in vcluster-A can starve the host node and impact vcluster-B's pods.
  • Network-policy isolation by default. Unless the host CNI supports and enforces network policies that map to virtual- cluster identity, workloads across virtual clusters can reach each other at L3.
  • Host-cluster blast radius. Host cluster failure takes down all virtual clusters simultaneously.

The practical upshot: virtual Kubernetes clusters are appropriate for QA / pre-prod / ephemeral testing environments (the Deloitte use case), inappropriate for hard-multi-tenant production where tenants are mutually distrusting. For production hard isolation, dedicated clusters or Firecracker-style microVM pod sandboxing remain the reference options.

Implementation — how the sync works

vCluster (Loft Labs) is the reference implementation. At creation, a virtual cluster is:

  1. Control-plane pods scheduled onto the host cluster: one pod runs the virtual K8s API server, one runs the controller manager, one runs the data store (default SQLite), one runs CoreDNS.
  2. A dedicated host namespace is allocated; all workload pods created inside the virtual cluster are synced down to this host namespace as real pods (with name-prefixed for uniqueness).
  3. Sync configuration controls which resources flow which way: sync.fromHost surfaces host IngressClasses / StorageClasses up to the virtual cluster; sync.toHost pushes virtual-cluster Ingresses / PVCs down to the host where host-side controllers materialise them.

The syncer is the linchpin: without it, a virtual cluster would be a disconnected control plane with nowhere to run its pods.

Contrast with alternatives

  • Namespace-based multi-tenancy — lighter weight, but all tenants share an API server, so no RBAC / CRD / quota isolation at the API-surface level. Widely used but not an "independent K8s cluster" abstraction.
  • Dedicated physical clusters — strongest isolation, longest provisioning time (30–45 min for EKS per Deloitte), highest cost from duplicated controllers.
  • Firecracker-isolated pods — hard kernel isolation per pod via lightweight microVMs; stronger than vcluster but different abstraction (isolates workloads, not whole cluster surfaces).
  • Kamaji / Cluster API nested control planes — similar primitive to vcluster; Kamaji is another open-source project that externalises the K8s control plane as pods in a host cluster.

Seen in

  • sources/2026-04-27-aws-deloitte-optimizes-eks-environment-provisioning-with-vcluster — Deloitte's QA-testing substrate: 50+ virtual clusters on one EKS host cluster, provisioned via vCluster platform. The load-bearing canonical quote: "vCluster enables the creation of lightweight, fully functional virtual clusters that act like independent Kubernetes environments. This gives QA teams dedicated spaces for their work without the overhead of managing dozens of separate Amazon EKS clusters." Sub-5-minute provisioning, ~500 engineer- hours / year reclaimed, >50 vCPU + >200 GB RAM saved from shared controllers across the fleet. The shared-data-plane property is explicit: "Essential platform services […] are deployed once on the host cluster and shared across all virtual clusters. This approach reduces resource duplication and streamlines management."
Last updated · 427 distilled / 1,229 read