Skip to content

PATTERN Cited by 1 source

Merchant-of-Record Aggregation

Merchant-of-Record Aggregation is the design pattern where a single intermediary (the Merchant of Record or MoR) sits between N content producers and M consumers, acts as the legal counter-party for all transactions, and internally aggregates the flows into two bulk operations: charge each consumer once across all producers they consumed from, and pay each producer once across all consumers that consumed them. Cloudflare uses this shape to make per-request content billing (pay-per-crawl, 2025-07-01) economically feasible for publishers who individually could not support the bilateral-deal overhead.

Why the N×M bilateral version doesn't work

Historically, monetizing crawler access required "knowing the right individual and striking a one-off deal" (Cloudflare's launch-post phrasing) — per-publisher-per-crawler legal contract, per-pair billing infrastructure, per-pair KYC/tax/dispute handling. Any given publisher might want to charge ten crawlers; any given crawler might want to buy from hundreds of publishers; the pairwise overhead is O(N×M) in legal/tax/billing work. Small publishers without scale or legal capacity are effectively locked out.

How MoR collapses it

  • Consumer side: pay-per-crawl emits a per-request billing event (signed crawler + payment intent + HTTP-200 response with crawler-charged). Events from all publishers a given crawler consumed roll up into one bill Cloudflare sends to the crawler. The crawler sees one invoice from one counter-party.
  • Producer side: events for all crawlers a given publisher received roll up into one payout Cloudflare sends to the publisher. The publisher sees one deposit from one counter-party.
  • MoR carries the legal counter-party risk: tax collection and remittance, dispute handling, fraud, chargebacks. Publishers don't need to KYC crawlers; crawlers don't need to contract per publisher.

Complexity collapses from O(N×M) contracts to O(N+M) enrollments.

Required pre-conditions for the pattern to work

  1. Trusted intermediary acceptable to both sides. All parties trust the MoR to accurately measure / fairly distribute. A well-positioned infrastructure vendor (Cloudflare, Stripe) is a natural fit; a new-entrant MoR struggles.
  2. Verifiable metering. You can only aggregate what you can measure cleanly. Pay-per-crawl requires a signed (patterns/signed-bot-request) request + a payment-intent header + an HTTP-200 response — the billing event emerges from three verifiable signals, not from a publisher's self-report.
  3. Common protocol surface. Bilateral contracts permit publisher-specific flows; MoR aggregation requires a single shared protocol (here: HTTP 402 + crawler-price / crawler-exact-price / crawler-max-price / crawler-charged headers) so metering is uniform.

Trade-offs

  • Rake / fee: the MoR charges a cut of every transaction; that's the price of having someone else do the legal, tax, billing, dispute work.
  • Platform risk: when the MoR is also the CDN/edge provider (Cloudflare), participants become doubly coupled to that vendor — losing the CDN also loses the monetization channel.
  • Price-discovery opacity: publishers don't see per-crawler market-clearing prices because pricing is configured upfront, not bid — the MoR pattern doesn't automatically produce an auction / market.

Generalizes beyond pay-per-crawl

Merchant-of-Record aggregation is the generic shape of app stores (Apple/Google act as MoR for every app purchase), payment processors like Stripe Connect / Paddle, ad networks (Google, Meta) aggregating between advertisers and publishers, and cloud marketplaces (AWS Marketplace, Azure Marketplace) aggregating between ISVs and customers. Pay-per-crawl is the content-access-per-request instance applied to the agentic-paywall use case.

Seen in

Last updated · 200 distilled / 1,178 read