Skip to content

CONCEPT Cited by 1 source

Escrow VPC for LLM serving

Definition

An escrow VPC for LLM serving is a deployment shape in which a third-party model provider's proprietary model weights are hosted in a customer-controlled cloud environment under conditions that make both directions of access opaque: the customer's data remains private to the customer, and the provider's model weights remain inaccessible to the customer. The VPC acts as the trust boundary that satisfies both parties' contractual requirements simultaneously.

Slack's 2026-05-28 retrospective canonicalises the framing verbatim (Source: sources/2026-05-28-slack-slack-ai-the-path-to-multi-cloud):

"We were able to leverage a sophisticated escrow virtual private cloud (VPC) strategy to establish a strict zero-knowledge environment: our data remained private to Slack, and the provider's proprietary model weights remained inaccessible to us."

This was Slack's Phase 1 (early 2023) solution for hosting Anthropic models inside their AWS SageMaker substrate while maintaining FedRAMP compliance and the enterprise data privacy contract Slack offers customers.

Structural properties

The escrow-VPC pattern has three load-bearing properties:

  1. Customer-controlled VPC — the network, IAM, audit, compliance posture, and data-plane traffic are all customer-owned. The provider cannot exfiltrate customer prompts or activations.
  2. Provider-protected weights — model weight files are deployed into the VPC under access controls that prevent the customer's engineers, ops, or security teams from reading them directly. SageMaker's container-isolated inference endpoints with provider-managed deployment pipelines is the canonical realisation.
  3. Bidirectional opacity — both directions of trust are enforced by the VPC's structure rather than by contractual honour. The customer can be certain the provider doesn't see their data; the provider can be certain the customer doesn't see their weights.

Why escrow VPC was necessary in 2023

In early 2023, third-party LLM providers (Anthropic in Slack's case) had two configuration options for enterprise customers with strict data privacy requirements:

  • Provider-hosted API — customer prompts leave the customer's network into the provider's environment. Did not meet Slack's enterprise data-privacy bar.
  • Customer-hosted weights — customer downloads model weights and runs inference internally. Did not meet provider's IP-protection bar.

The escrow VPC pattern resolves this asymmetry by putting the weights in the customer's environment under deployment-time controls that prevent customer-side weight access while keeping data inside the customer's trust boundary.

Trade-offs

The escrow-VPC posture is structurally expensive:

  • Operational overhead — customer manages cross-region IAM roles, balanced routing across model endpoints, proactive capacity planning, and auto-scaling logic. "This required our teams to manage the operational lifecycle" (Slack verbatim).
  • Model feature lag — provider's model iterations and optimisations debut on the provider-managed cloud surface (e.g. Amazon Bedrock for Anthropic) "weeks or months before SageMaker availability."
  • GPU scarcity exposure — customer holds the hardware reservation problem; "Enterprise-grade Nvidia GPUs, such as the A100 (Ampere architecture) and the emerging H100 (Hopper architecture) instances, were often unavailable."
  • Over-provisioning for peak SLAs — "Maintaining idle resources to meet peak SLAs" — the customer pays for the peak capacity, not the average.

These three taxes are why Slack moved from escrow-VPC SageMaker (Phase 1) to fully managed Bedrock (Phase 2) once Bedrock achieved equivalent compliance posture in mid-2024.

Composition with neighbouring concepts

Concept Relationship
concepts/zero-knowledge-environment (general) The escrow VPC is the LLM-serving-specialised realisation of zero-knowledge between two principals.
concepts/llm-model-feature-lag The structural cost of escrow VPC: the provider can't ship updates as fast through customer-hosted weights as through their own primary launchpad.
concepts/data-sovereignty Escrow VPC is one mechanism for satisfying data-sovereignty requirements when the workload involves third-party model IP.
Multi-tenant SaaS LLM endpoint The opposite end of the spectrum: prompts leave customer environment; weights stay in provider environment; only contractual privacy guarantees.

When escrow VPC is the right shape

  • Strict data-privacy contracts with enterprise customers (e.g. FedRAMP Moderate or higher, regulated industries, large customer-data perimeter).
  • Provider-protected weights that the provider will not release outside their control under any other configuration.
  • Provider's primary-launchpad cloud surface doesn't yet meet compliance requirements — escrow VPC is the bridge until a fully-managed compliant alternative ships.

When to migrate off

  • Provider's managed-service surface (e.g. Amazon Bedrock for Anthropic) reaches the required compliance posture and ships models faster than escrow-VPC.
  • Operational overhead of self-managed serving exceeds the cost of moving to managed serving.
  • GPU scarcity and capacity-planning friction make predictable SLAs harder than the alternative.

All three triggered Slack's Phase 2 migration in mid-2024.

Seen in

  • sources/2026-05-28-slack-slack-ai-the-path-to-multi-cloud — canonical wiki disclosure of escrow VPC as the Phase 1 (early 2023) solution for hosting Anthropic models inside AWS SageMaker with FedRAMP-compliant zero-knowledge enterprise data privacy. Slack named the structural costs (scaling latency, GPU scarcity, over-provisioning, model feature lag) that drove the eventual Phase 2 migration to Amazon Bedrock.
Last updated · 542 distilled / 1,571 read