SYSTEM Cited by 1 source

Instacart AI Gateway

AI Gateway is Instacart's internal proxy layer for communicating with external LLM providers. It acts as a centralised abstraction over multiple LLM vendors and integrates with the Cost Tracker to log usage and spend per job and team.

Referenced in the 2025-08-27 Maple post:

"Maple proxies requests through our AI Gateway, another internal Instacart service which acts as a centralized abstraction layer for communicating with multiple LLM providers. The AI Gateway is also responsible for integrating with the Cost Tracker, logging detailed usage and spend per job and team." (Source: sources/2025-08-27-instacart-simplifying-large-scale-llm-processing-with-maple)

Stub page — post does not disclose AI Gateway internals (fleet shape, caching behaviour, provider fallback logic, region strategy, key- management posture). Expand on future Instacart sources that describe it.

Role in the Instacart platform cluster

The AI Gateway is the provider-abstraction tier shared by Instacart's LLM-consuming internal platforms:

• Maple — batch LLM processing (directly cites AI Gateway as the proxy target).
• PIXEL — image generation (inherits the pattern; 2025-07-17 post references "existing Instacart infra" without naming AI Gateway but the architectural shape matches).
• PARSE — multi-modal LLM attribute extraction (2025-08-01 post; architectural shape matches).

This is a canonical patterns/ai-gateway-provider-abstraction instance — the text-LLM pattern that Cloudflare AI Gateway and Databricks Unity AI Gateway productise externally, Instacart runs internally. Each concern at its natural layer:

• Inference = LLM provider.
• Provider routing + cost tracking = AI Gateway.
• Workflow / batching / retry = Maple (batch) or PIXEL/PARSE (synchronous).

Seen in

• sources/2025-08-27-instacart-simplifying-large-scale-llm-processing-with-maple — first wiki disclosure of AI Gateway as a named internal Instacart service with provider-abstraction + cost-tracking responsibilities.

Related

• systems/maple-instacart — direct caller in the 2025-08-27 post.
• systems/instacart-cost-tracker — downstream integration.
• systems/instacart-pixel — sibling platform above the same provider-abstraction tier.
• systems/instacart-parse — sibling platform above the same tier.
• systems/cloudflare-ai-gateway — external product canonicalising the same pattern.
• systems/unity-ai-gateway — Databricks equivalent.
• patterns/ai-gateway-provider-abstraction — the pattern.
• concepts/cost-tracking-per-team — the accounting primitive.
• companies/instacart — operator.