Skip to content

SYSTEM Cited by 7 sources

Envoy

Envoy is an open-source L7 proxy originally developed at Lyft, now a CNCF graduated project. It's the canonical data-plane component of sidecar-based service meshes (Istio, Consul Connect, AWS App Mesh) and a common ingress/edge proxy in its own right. Envoy's dynamic configuration API — xDS — has become the de facto standard interface between control planes and L7 proxies, and is now used beyond sidecars as a general service-discovery contract.

Minimum viable definition for this wiki; expand on future ingests.

Core features relevant to system design

  • L7 load balancing. Per-request decisions (vs. per-connection at L4); required for HTTP/2, gRPC, and other long-lived-connection protocols.
  • xDS-driven dynamic configuration. Control planes stream cluster / endpoint / route / listener updates without reloads.
  • Health-aware routing. Outlier detection, active health checks, priority levels for failover.
  • Advanced load-balancing algorithms. Round-robin, random, least-request, ring hash, Maglev, weighted — and Power of Two Choices on least_request.
  • Observability. Per-request metrics, access logs, tracing hooks.

Roles in distributed architectures

  • Sidecar-mesh data plane (Istio, Consul): one Envoy per pod, intercepts all service traffic.
  • Ingress / edge proxy: single Envoy tier at the cluster edge; frequent pattern for HTTP/gRPC gateway.
  • Standalone L7 LB: Envoy as a modern replacement for HAProxy / NGINX when dynamic topology matters.
  • Outbound egress filter / SSRF guard: Envoy fronting a user-URL-touching service (webhook sender, URL preview fetcher, image proxy) with rules blocking internal IPs + non-HTTPS ports. Rules are re-evaluated at send time against the actual resolved IP, closing the DNS-rebinding check-to-send gap that submission-time URL validation alone cannot close. See patterns/isolated-egress-proxy-for-user-urls.
  • ML-inference routing data plane: Envoy as the header-based routing tier in a multi-tenant ML-serving platform. A metadata resolver (Lightbulb-class) attaches a routingKey header; Envoy maps the key to a cluster VIP via dynamic routing rules and forwards the untouched payload. Avoids the serialization tax of an in-path proxy that would need to deserialize the body. See patterns/separate-routing-from-model-selection.
  • Telemetry-ingress L7 tier: Envoy as a dedicated ingress tier for observability traffic, running independent of the general-purpose service mesh that carries business traffic. Eighth canonical Envoy role on this wiki. Resolves three concerns that a shared mesh fails at: (1) circular dependency between mesh and mesh-metrics, (2) telemetry-traffic congestion-induced blindness (concepts/observability-traffic-volume-asymmetry), (3) telemetry spikes as noisy-neighbour on business traffic. Airbnb's 2026-05-05 instance uses header-based tenant routing mapping ~1,000 services → cluster backends. See patterns/custom-l7-proxy-for-telemetry-over-service-mesh.

Seen in

  • sources/2026-05-05-airbnb-monitoring-reliably-at-scale — Envoy as a custom L7 telemetry-ingress tier, independent of Airbnb's Istio mesh that carries business traffic. Verbatim: "We built a custom Layer 7 network ingress layer based on Envoy that load-balances traffic and routes read and write requests to the right backends. Running this proxy independent of the shared compute layer added fault tolerance and shielded our ingest path from service-mesh failures." Tenant model: ~1,000 Airbnb services mapped 1:1 to tenants in a single global user space; every request must include a tenant header; Envoy maps service-name → backend cluster via that header. Extensibility hooks the team called out: "we can mirror metrics to alternate destinations for testing or enforce fine-grained access controls, which is critical when working with external vendors or specialized use cases." Canonicalises an eighth Envoy role on this wiki alongside sidecar / edge-proxy / standalone-LB / egress-SSRF-guard / JWT-validator / µVM-orchestrator-ingress / ML-serving-routing. First canonical wiki instance of Envoy in a telemetry-ingress / observability-networking role, distinct from carrying business traffic via Istio. Composes with patterns/custom-l7-proxy-for-telemetry-over-service-mesh and concepts/tenant-header-routing.
  • sources/2026-05-01-netflix-state-of-routing-in-model-serving — Envoy as the ML-inference routing data plane in Netflix's 2026 model-serving architecture. Netflix's prior zero-configuration service mesh with on-demand cluster discovery already had Envoy on all egress paths; the 2026-05-01 post extends Envoy's role to ML-serving cluster VIP routing. A new Lightbulb service produces a routingKey header + ObjectiveConfig body for each request; Envoy consumes the routingKey to route at minimum overhead without parsing the body. Canonicalises a seventh Envoy role on this wiki alongside sidecar / edge-proxy / standalone-LB / egress-SSRF-guard / JWT-validator / µVM-orchestrator-ingress. Load-bearing for Netflix's evolution away from the in-path Switchboard proxy (10–20ms serialization tax, SPOF across 30+ client services) at 1M req/sec. First canonical wiki instance of Envoy in the ML-serving-routing role; pairs with systems/netflix-lightbulb as the metadata-resolver complement.
  • — Envoy as the outbound egress filter for user-supplied URLs in PlanetScale's webhook-sender service (Mike Coutermarsh 2023-11-21, re-fetched 2026-04-21). First canonical wiki disclosure of Envoy in the egress-SSRF-guard role — adds a fifth Envoy-role entry alongside ingress gateway / EDS-client / sidecar / JWT-validator. PlanetScale runs webhooks as a dedicated Kubernetes service whose outbound HTTP flows exclusively through Envoy; two rules enforced at the proxy tier verbatim: "Block any connections to internal/private IPs. Limit traffic to HTTPS ports." Near-identical to the application-tier URL-validation rules but executed at send time against the actual resolved connect-IP, defeating DNS rebinding. Canonical pattern: patterns/isolated-egress-proxy-for-user-urls; composes into patterns/defense-in-depth-webhook-abuse-mitigation with Sidekiq uniqueness + isolated infrastructure + strict send timeouts + per-DB quota.
  • sources/2025-10-01-databricks-intelligent-kubernetes-load-balancing — Databricks uses Envoy as its ingress gateway, configured via the same EDS control plane that feeds internal Armeria clients. This gives internal and external traffic a single source of truth for endpoint topology. The post also discusses Envoy-as-sidecar (Istio/Ambient Mesh) as the evaluated-and-rejected alternative for intra-cluster routing.
  • sources/2024-10-28-dropbox-robinhood-in-house-load-balancing — Dropbox's Robinhood LBS writes PID-computed endpoint weights into a ZooKeeper/etcd routing DB; Envoy clients consume them natively via EDS and do weighted round-robin per request. Same one-control-plane-many-data-planes pattern as Databricks, but applied to PID-driven weights rather than topology-only EDS.
  • sources/2025-01-18-aws-app-mesh-discontinuation-service-connect-migration — Envoy is the data plane for both AWS App Mesh (customer-managed sidecar) and ECS Service Connect (AWS-managed sidecar). Same proxy, different operational contract — canonical write-up of the managed sidecar tradeoff vs customer-run Envoy. App Mesh exposes the full Envoy config surface (retry/outlier tuning, mTLS, version-weighted routing); Service Connect narrows to timeouts-only in exchange for AWS-managed lifecycle + free CloudWatch metrics.
  • sources/2026-03-19-pinterest-building-an-mcp-ecosystem-at-pinterest — Envoy as the JWT validation + identity-header-mapping point for Pinterest's hosted MCP ecosystem. "Envoy validates the JWT, maps it to X-Forwarded-User, X-Forwarded-Groups, and related headers, and enforces coarse-grained security policies (for example, 'AI chat webapp in prod may talk to the Presto MCP server, but not to experimental MCP servers in dev namespaces')." Canonical wiki instance of Envoy-as-AI-agent-auth-enforcement — the mesh-level leg of Pinterest's layered JWT + mesh auth. Pairs with in-process @authorize_tool decorators (patterns/per-tool-authorization-decorator) for fine-grained per-tool authorization. First wiki datum of Envoy's JWT-plus-header-mapping role in an MCP / agent-traffic context.
  • sources/2026-04-24-atlassian-rovo-dev-driven-development — Envoy as the ingress tier of a Firecracker-microVM orchestrator on Kubernetes. Atlassian's Fireworks platform names "envoy ingress layers" verbatim as one of the core components the team built alongside the scheduler / autoscaler / node agents / Raft persistence. Canonicalises a sixth Envoy role on this wiki: agent-platform-ingress / µVM-orchestrator-ingress — the L7 entry point in front of a µVM control plane, distinct from the existing sidecar / edge-LB / EDS-client / JWT-validator / egress-filter roles. First wiki disclosure of this shape; structurally adjacent to the Firecracker-on-K8s composition.
Last updated · 542 distilled / 1,571 read