Skip to content

CONCEPT Cited by 1 source

Runtime nullability telemetry

Definition

Runtime nullability telemetry is the practice of instrumenting the runtime — typically via a compiler plugin that emits recording shims — to capture the actual null behaviour of Java parameters, return types, and fields that have no static annotation. The collected data then drives codemods that backfill correct @Nullable / @NonNull annotations into the source. "When static analysis runs out of road, instrument production."

Why it matters

Static null-safety analysers are load-bearing but coverage- bounded: "static analysis is only 100% effective for 100% code coverage, which is simply not viable in any large mobile codebase that interacts with the server and third- party libraries." In a large Java codebase at the tail of a migration, the remaining non-annotated parameters and return types are exactly the ones the team has been unable to resolve statically — by definition, the hardest cases. Runtime telemetry turns them into empirical data: whatever values actually flow through the parameter or return type in production determine the correct annotation.

Four consequences that make it worth the cost:

  1. Ground truth, not modelled truth. Production null flows are the authoritative behaviour. Disagreement with static inference points at either a missing annotation or a genuine correctness issue; both are worth fixing.
  2. Drives codemod-based backfill. Collected data is structured (class × method × parameter × bucket), so codemods can mechanically update annotations at scale.
  3. Improves even untranslated Java. Backfilled @Nullable annotations benefit the existing Java code forever, independent of the Java-to-Kotlin migration's eventual completion. "More accurate nullability across the codebase — even in Java files that we may never translate — is not only safer, it's also conducive to more successful conversions."
  4. Unblocks the long tail of conversions. Annotations that used to gate translation because "no one knows if this parameter can be null" become decidable.

Canonical wiki instance — Meta's javac plugin

The 2024-12-18 Meta post canonicalises the pattern at Meta scale:

"Previous attempts to resolve them relied mostly on static analysis, so we decided to borrow an idea from the Kotlin compiler and create a Java compiler plugin that helps us collect runtime nullability data. This plugin allows us to collect data on all return types and parameters that are receiving/returning a null value and are not annotated as such. Whether these are from Java/ Kotlin interop or classes that were annotated incorrectly at a local level, we can determine ultimate sources of truth and use codemods to finally fix the annotations."

The plugin is modelled on Kotlin's own runtime checkNotNull at the interlanguage boundary (see concepts/kotlin-platform-type). Kotlin's compiler emits that check for its own correctness; Meta's plugin emits the same check as a data-collection shim, not just a fail-fast guard — the value is recorded, not only the assertion.

Design considerations

  • Sampling vs logging. Plugin output at fleet scale is high-cardinality; either rate-limit at the shim or aggregate in-process and flush per interval (the aggregating- buffered-logger shape).
  • Privacy discipline. Record only null / non-null observations and callsite locations, never the argument values themselves; parameters may be PII-bearing.
  • Data lifetime. Telemetry drives a codemod (one-off) not a dashboard (forever) — once annotations are backfilled, the plugin can be retired from the affected classes.

Contrast with static analysers

  • Static analysers prove nullability in the subset of code they cover. Runtime telemetry measures it in the subset of code that runs.
  • Static is cheap per unit of code, free per request; telemetry is cheap per request, pays a fleet-scale aggregation cost.
  • Static fails open at coverage boundaries. Telemetry fails open at cold-code boundaries (code that rarely runs observes no nulls, which is not the same as "is non- null"). Use both.

Seen in

Last updated · 517 distilled / 1,221 read