Skip to content

SYSTEM Cited by 1 source

MERJ Web Rendering Monitor (WRM)

MERJ's Web Rendering Monitor is an open-source lightweight JavaScript library (published at github.com/merj/wrm-research-vercel) that measures whether and when a search-engine / AI crawler finishes rendering a page. Injected into the HTML response for bot requests via edge middleware, the library fires on page- render completion and POSTs a payload to a long-running beacon server for later pairing with the original server access-log entry.

The WRM + Vercel Edge Middleware + long-running beacon server is the three-part measurement architecture Vercel and MERJ used to produce the 2024-08-01 Google-rendering study's 37,000+ server- beacon-pair dataset.

Payload shape (per beacon)

On render completion, WRM POSTs to the beacon server:

  • Page URL — what the bot was rendering.
  • Unique request identifier — assigned by the edge middleware on injection; used to join with server access logs.
  • Rendering-completion timestamp — computed on the beacon server as "time of WRM library's HTTP POST arrival". Done server-side so a misconfigured / clock-skewed bot can't skew the measurement.

Pairing loop

   Googlebot                Vercel Edge Middleware          Beacon Server
   ---------                ----------------------          -------------
   1. GET /page             ------> log access (req-id=X, t=T_crawl)
                            -----> inject WRM JS into HTML response
                                    with embedded req-id=X
   2. [WRS renders page, runs WRM]
   3. POST /beacon          -----------------------------> log beacon (req-id=X, t=T_render)

   Join (server access log) ⋈ (beacon log) on req-id
   Render delay = T_render - T_crawl

The pairing works even though the rendering client (Googlebot + WRS) is outside the measurer's control — the only assumption is that the bot's sandbox permits outbound fetch() / XHR from the injected script. See patterns/server-beacon-pairing-for-render-measurement.

What WRM does not measure directly

  • Rendering failure. If WRS fails to render the page (rare per the study — 100 % success on nextjs.org's 100,000+ fetches), WRM never fires, and the entry shows up as a crawl without a render. The study's "100 % rendered" claim is defined as "every 200/304 indexable HTML page produced a beacon" — rendering failures that also suppress beacons are an unobservable residual (small by independent indication, not zero by measurement).
  • Rendering attempts before success. WRS may have internal retries; WRM only sees the final successful render.
  • Non-Googlebot crawlers with sandboxes that block outbound HTTP. The study was explicit about still gathering data for OpenAI / Anthropic / Perplexity bots — which may or may not run the beacon, depending on whether their JS sandbox permits outbound fetch.

Positioning

  • Research instrument, not production observability. WRM is explicitly a research / study library (repository name includes research). The tooling is open-sourced to make the Vercel + MERJ methodology reproducible, not as a production SEO tool.
  • Complementary to Google Search Console's URL Inspection Tool, which lets site operators see a single page's Google- rendered output. WRM measures when across many pages; URL Inspection Tool measures what on one page.

Seen in

Last updated · 476 distilled / 1,218 read