Skip to content

AWS 2026-10-29 Tier 1

Read original ↗

AWS: How BMW Group accelerated connected vehicle workloads on AWS Graviton

Summary

AWS Architecture Blog customer case study, co-written with BMW Group and Critical TechWorks (CTW). BMW Connected Company runs the Connected Vehicle platform (internal name Orbit) for a fleet of more than 24.5 million vehicles, on more than 400 Amazon EKS clusters, with a CI/CD platform that runs more than 130,000 builds and deployments per day on self-hosted GitHub Actions runners. In 2024 the team began to migrate Orbit from x86 to Arm-based AWS Graviton compute. The post is a migration methodology, not a chip benchmark. The core idea: keep both CPU architectures viable at the same time so each application team can move on its own schedule and fall back without a rebuild. Four layers make this work: (1) Karpenter NodePools instead of fixed Managed Node Groups; (2) multi-arch container images (an OCI image index / manifest list behind one tag), built either natively on Graviton runners or emulated with Docker Buildx + QEMU; (3) tainted, shared Graviton NodePools that workloads must explicitly opt into with a node selector and a toleration, plus a soft node affinity that prefers arm64; (4) a platform-first rollout (Karpenter, Prometheus, operational tooling first) and ArgoCD GitOps for full rollback. Results: about $600,000/year saved, about 40% better price-performance, about 57% less CPU on the NGINX-based ingress controller, and more than 65% of the platform on Graviton in production.

Key takeaways

  1. Fix the provisioning layer before the CPU architecture. The team first replaced fixed Managed Node Groups with Karpenter's dynamic, policy-driven NodePools. Only then could the cluster pick Arm or x86 instance types automatically per workload. A separate BMW post covers that Karpenter migration. (Source: this page)

  2. Validate on self-contained services, then build two tracks. Graviton CI runners first ran beside the x86 fleet on a small set of self-contained services. Cost per build minute dropped and builds got faster, which made the business case. The wider rollout then ran as two tracks: dual build pipelines (ship both architectures in parallel) and container hosting (Graviton NodePools in the central platform). The figure shows six stages in two phases: build foundation, then workload migration. (Source: this page) — a concrete patterns/staged-rollout.

  3. One tag, two architectures. Every image is published as an architecture-specific image per platform plus a manifest list (OCI image index). The container runtime reads the index and pulls the variant for the node's architecture. So one tag serves amd64 and arm64 nodes, and the deployment manifest does not change when a pod lands on a different architecture. (Source: this page)

  4. Native vs emulated builds is a real trade-off — offer both.

  5. Native per architecture: one runner per architecture builds and pushes its image, then a final job creates the image index. More reliable; needs a dedicated runner per architecture.

  6. Emulated from one machine: Docker Buildx + QEMU builds every architecture on one x86 or Arm runner. Simpler pipeline, but emulation overhead makes compute-heavy builds "significantly slower". Teams choose through GitHub Actions reusable workflows: simple Dockerfiles use QEMU (no extra runner); complex builds, large dependency downloads, or long build times use native Graviton runners. Native arm64 builds also surface architecture-specific bugs earlier. Graviton runner labels mirror the x86 labels, so adoption is a one-word change (x64 → arm64). (Source: this page) — the low-friction default with an escape hatch is patterns/golden-path-with-escapes.

  7. Mirror base images without losing multi-arch. Public images are mirrored into private Amazon ECR for availability, immutability, and CVE compliance. Google's Crane copies images registry-to-registry and filters the manifest to only the required platforms, so unused architectures do not waste storage. Most application Dockerfiles needed no change because base images and dependencies were already multi-arch; a few needed an Arm-capable alternative dependency. (Source: this page)

  8. Opt-in scheduling with taints, selectors, and tolerations. Graviton NodePools are shared and general-purpose, not tuned per workload, so the infrastructure model stays uniform. They carry a NoSchedule taint (bmw.de/stateless-arm=supported). A workload must add the matching node selector and toleration to land there. Unready workloads keep running on amd64 with no change. This is the same Karpenter taint primitive that other teams use for tenant isolation, used here as a migration gate. (Source: this page)

  9. Prefer Arm, do not force it. A shared Helm helper adds a preferredDuringSchedulingIgnoredDuringExecution node affinity (weight 100, kubernetes.io/arch In [arm64]) plus the toleration. Opted-in workloads gravitate to Graviton, but the scheduler can still place them on x86 when Arm capacity is short. (Source: this page)

  10. Platform-first adoption. Before application teams were invited, the platform team moved its own shared services — Karpenter, Prometheus, and operational tooling — to Graviton. This found compatibility gaps early, tuned the NodePool configuration, and set rollback patterns, so teams got a proven reference model. (Source: this page)

  11. Rollback is designed in, at two levels. Because every app has dual-arch images and the x86 pool stays in place, a workload with an Arm problem falls back to x86 without an image rebuild or a pipeline change (drop the opt-in). For a full rollback, the team reverts the previous version in Git and lets ArgoCD reconcile the cluster. (Source: this page) — patterns/fast-rollback: keeping the old path warm bounds the concepts/blast-radius of the migration.

  12. Adoption is driven by removing friction, not by policy. "Getting teams onto Graviton depended less on policy than on removing friction." The platform supplied reusable workflows, mirrored labels, shared pools, and a Helm helper; teams opted in themselves — a concepts/self-service-infrastructure framing of a platform migration. (Source: this page)

Architecture

            GitHub Actions (130k+ builds/deploys/day)
   ┌────────────────────────────┬──────────────────────────────┐
   │ reusable workflow: native  │ reusable workflow: emulated  │
   │ x64 runner → amd64 image   │ one runner + Buildx + QEMU   │
   │ arm64 runner → arm64 image │ → amd64 + arm64 images       │
   │ final job → image index    │ → image index                │
   └──────────────┬─────────────┴───────────────┬──────────────┘
                  ▼                             ▼
     Amazon ECR (private; public bases mirrored by Crane,
                 manifest filtered to needed platforms)
                  │  one tag → OCI image index
                  ▼
   ArgoCD (GitOps) ──► EKS cluster (400+ clusters fleet-wide)
                        ├─ Karpenter NodePool: x86 (default)
                        └─ Karpenter NodePool: Graviton, tainted NoSchedule
                             ↑ only pods with nodeSelector + toleration;
                               soft affinity prefers kubernetes.io/arch=arm64
   On-premises systems connect into the cluster (Figure 2).

Systems / concepts / patterns extracted

Operational numbers

Metric Value
Connected vehicles served > 24.5 million
EKS clusters (platform lead's scope) > 400
CI/CD builds + deployments per day > 130,000
Annualized infrastructure savings ~ $600,000
Price-performance improvement ~ 40%
NGINX ingress CPU consumption change ~ −57% (same ingress profile)
Share of platform on Graviton (production) > 65%
Migration start 2024

Caveats

  • Vendor co-authored customer case study. The numbers are headline figures with no baseline instance types, Graviton generation, or measurement window.
  • The 57% CPU and 40% price-performance numbers come from one workload (the NGINX ingress controller). Do not read them as fleet averages.
  • No data on which workloads could not move (the remaining ~35%), on Arm-specific bugs found, or on how often the x86 fallback was used.
  • The soft affinity means "opted-in" pods can still land on x86. The post does not say how the team measured actual Arm placement.
  • No detail on how on-premises connectivity or stateful workloads were handled; the taint key stateless-arm suggests the pools target stateless services. The two code samples also use different taint keys (bmw.de/stateless-arm vs bmw.de/arm).

Source

Last updated · 854 distilled / 2,448 read