Skip to content

SYSTEM Cited by 1 source

Piqama (Pinterest Quota Management Ecosystem)

Piqama is Pinterest's generic quota management ecosystem — a single platform with pluggable hooks that handles both capacity-based quotas (memory / vcore / concurrent-app limits for the Moka Big Data Processing Platform) and rate-limit-based quotas (QPS / bandwidth for online storage services like TiDB and Key-Value stores).

Architecture

Piqama is a control-plane / data-plane platform:

  • Control plane — REST + Thrift management portal. Owns the entire quota lifecycle: schema management (unique identifiers + hierarchical relationships, e.g. workloads within an organisation's project), validation (pluggable framework including remote-service hooks for cluster-capacity sum-checks), authorization (owner-based on creation; owners can be individuals or groups), update dispatch, and default enforcement / punishment strategies.
  • Data plane — application services. Each application either uses Piqama's default enforcement via an integrated Piqama client, or plugs in its own schema / validation / dispatch / enforcement logic. Rate-limit decisions are made locally in the data path for speed and flexibility.
  • Governance + optimisation — usage statistics flow back into Apache Iceberg on S3 (pre-aggregated for storage efficiency). A separate auto-rightsizing service consumes this data from Presto, Iceberg, or user-defined sources and writes recomputed quota values back via the control-plane API.

Integrations disclosed

Moka (capacity-based quota)

Moka is Pinterest's next-gen Big Data Processing platform built atop Apache Yunikorn. Piqama stores per-project:

  • Guaranteed resources — memory + vcore floor.
  • Maximum resources — memory + vcore ceiling.
  • Max concurrent applications — simultaneous-app cap.

A Yunikorn Config Updater polls Piqama for updated quota values and rewrites Yunikorn's scheduling-framework configuration. Each application runs inside its project's dedicated Yunikorn queue. On completion, Yunikorn emits a usage summary to S3 → resource database → feeds both future quota calculations and budget enforcement: when a project exceeds its allocated budget in a defined time window, Piqama dynamically lowers that project's maximum-resource quota. The over-budget haircut is tier-weighted (X% depending on the project's tier).

Quota values are generated via two paths — Auto-Rightsizing based on sliding-window historical usage (Moka's current "legacy" mode, with a budget-based approach under development) and Manual Adjustment for firefighting.

Canonical wiki instance of budget-enforced quota throttling.

TiDB + KV Stores (rate-limit quota)

Pinterest's online-storage rate-limit framework needed to be upgraded: existing rules were non-declarative, manual to adjust, and had static thresholds. Piqama redesign:

  • Rule creation — via UI (humans) or API (online services / automated pipelines). Centrally managed via Piqama with authorization + auditing.
  • Rule delivery — via PinConf (same substrate as feature flags and dynamic service config). Canonical config-distribution-for-quota-rules instance.
  • Rule adjustment — UI/API ad-hoc + continuous via the rightsizing service aggregating request stats.
  • Rule enforcement — local, via an in-house library called the Service-Protection Framework (SPF). Local decisions enable "fast rate limiting decisions (in contrast to relying on a global rate limiting service)" and let the host factor in local service-health for graceful rejection.

Canonical wiki instance of async-centralized quota + local enforcement.

Quota vs Budget vs Entitlement

Piqama frames a three-part hierarchy:

  • Budgets allocate dollars to orgs / teams / projects.
  • Entitlements (Pinterest's Entitlement system; integration is future work) translate dollars into resource rights.
  • Quotas enforce those rights at the scheduler / data-plane layer.

A chargeback system translates usage to dollars, draws from budget, and triggers the quota-haircut loop on exceedance.

Canonical wiki instance of entitlement-budget-quota integration.

Roadmap (disclosed)

  1. Entitlement integration — deeper link between resource quotas and Pinterest's Entitlement system.
  2. Advanced auto-rightsizing — custom strategies, tighter resource-utilisation bounds.
  3. Distributed quota management — cross-instance quota coordination for complex environments. Implementation not disclosed.
  4. Unified client — one-stop SDK for Piqama integration across services.
  5. New named adopters — PinCompute (Pinterest's Kubernetes-backed general-purpose compute platform), the ML Training Platform, and LLM Serving Services.

Seen in

Last updated · 319 distilled / 1,201 read