Skip to content

CONCEPT Cited by 1 source

IP address fragmentation

IP address fragmentation is the phenomenon where a cluster's available IP address space is distributed across many small subnets such that no single subnet has enough contiguous addresses to launch a new node or pod, even though the total free-IP count across all subnets is ample.

Analogous to memory fragmentation: total free bytes might be fine, but there's no single contiguous block big enough for the allocation.

How it arises in Kubernetes on AWS

EKS deployments carve VPC subnets per AZ, and each node typically draws its pod-network IPs from one subnet:

  • Subnet-per-AZ planning — three small subnets (one per AZ) each with a /24 (~256 addresses). Each subnet serves nodes provisioned in that AZ.
  • Node-group-per-workload-shape — each node group is pinned to a fixed set of subnets. If one shape dominates, its subnets fill up while others sit unused.
  • Pod-per-node density with VPC CNI — each pod can consume one VPC IP, so every node allocates a chunk of IPs per its instance type.

Over time: subnets fill unevenly, and new node provisioning fails with InsufficientFreeAddressesInSubnet even though other subnets have plenty of room. The whole cluster is "out of IPs" despite the cluster-wide IP count being fine.

How Karpenter mitigates it

Salesforce's 2026-01-12 post names IP efficiency as a Karpenter win:

"Karpenter further improved IP efficiency by decoupling node provisioning from specific subnets, helping reduce IP fragmentation and exhaustion across the platform." (Source: sources/2026-01-12-aws-salesforce-karpenter-migration-1000-eks-clusters)

The mechanism: one NodePool / EC2NodeClass can be allowed to span many subnets (across AZs, across multiple same-AZ subnets) so Karpenter picks the subnet at provisioning time based on current availability. The scheduler's view is "what subnet has IPs right now?" not "this node group is pinned to these subnets." Free addresses anywhere in the allowed set can be used, which restores the total-free-IP count as the usable ceiling.

Broader context

The same concept shows up elsewhere:

  • Container network planning. Per-pod IP density on large nodes (p4d.24xlarge can hold 400+ pods with VPC CNI) makes subnet sizing a first-class capacity problem.
  • Service mesh sidecars double IP consumption per pod.
  • Multi-cluster fleets — cross-cluster VPC-peering plans have to avoid CIDR collisions, forcing smaller subnets.

Fixes outside of Karpenter:

  • Secondary CIDR ranges on the VPC — EKS supports attaching secondary CIDR (e.g. 100.64.0.0/16) dedicated to pod networking.
  • IPv6 dual-stack — eliminates the 32-bit address scarcity at the cost of IPv6 operational maturity.
  • VPC CNI Custom Networking — pods draw from a separate pool than nodes.

Seen in

Last updated · 200 distilled / 1,178 read