Skip to content

META 2024-12-18 Tier 1

Read original ↗

Meta — Translating 10M lines of Java to Kotlin

Summary

Meta Engineering post (2024-12-18) on the multi-year effort to translate the entire Android codebase at Meta from Java to Kotlin — "roughly ten million lines of perfectly good Java code" — driven end-to-end by an internal pipeline called the Kotlinator. The post is a rare public look at what it takes to run a language migration of this size on a monorepo: not an LLM/agent rewrite, but a deterministic AST-transformation pipeline built around JetBrains' J2K IDE inspection extended with ~200 custom pre/post-processing steps, a build-error-driven fix loop, and a parallel multi-year push to retrofit null safety on Java before translation. At the time of writing Meta had shipped "over 40,000 conversions" and was "well past the halfway point" on the ~10M-line target. The Android codebase has been Kotlin-first since 2020.

The architecturally load-bearing content is in four areas:

  1. Kotlinator pipeline — 6 phases (deep build → preprocessing → headless J2K → postprocessing → linters → build-error-based fixes), ~30 min per file, runs on a remote fleet, with the daily diffs produced by an internal cron system functionally identical to a GitHub-PR bot.
  2. Metaprogramming on broken code — Meta's internal AST tooling built on JetBrains PSI is "very much not a compiler plugin" so it can analyse Java + Kotlin across thousands of unbuildable files simultaneously. Trade-off: it bails early when type information is unresolvable (e.g. third-party symbols), leaving obvious human fixes.
  3. Null safety as a prerequisite — Kotlin's runtime checkNotNull at the interlanguage boundary makes nullability annotations load-bearing in production in a way Java's static-analysis-only @Nullsafe / NullAway are not. Meta runs >12 complementary codemods plus a new Java compiler plugin that collects runtime nullability data at Java/Kotlin interop boundaries, so @Nullable annotations reflect reality, not hope.
  4. Bot-safer-than-human principle — Meta deliberately automates "delicate" transformations that aren't strictly necessary (e.g. condensing long chains of null checks) because human reviewers "accidentally dropping a negation" is a worse failure mode than a slightly-less-idiomatic auto-generated diff.

Key takeaways

  1. Scale frame: 10M lines, ~100K files, 40K+ conversions shipped, "well past the halfway point." The naive per-file IDE-button workflow was a non-starter: "We would have to click that button — and then wait the couple of minutes it takes to run — almost 100,000 times."
  2. Kotlinator has 6 phases — deep build (symbol resolution), preprocessing (~50 Editus steps for nullability + J2K workarounds + custom-DI accommodations), headless J2K (the biggest JetBrains-collab win), postprocessing (~150 steps for Android + nullability + idiomatic-Kotlin tweaks), linters with autofixes, build-error-based fixes (parse compiler errors, apply targeted fixes like "insert missing import" or "add !!").
  3. Headless J2K was the unlock for parallelism. JetBrains' J2K was tightly coupled to IntelliJ; Meta built an IntelliJ plugin whose class extends ApplicationStarter and calls JavaToKotlinConverter directly. "The headless approach allowed us to translate multiple files at once, and it unblocked all sorts of helpful but time-consuming steps, like the 'build and fix errors' process." (Source: Meta engineering post, 2024-12-18.)
  4. ~30 min per remote conversion, 0 developer seconds. The trade-off was total-time up, developer-time down: "Overall conversion time grew longer (a typical remote conversion now takes about 30 minutes to run), but time spent by the developers decreased substantially."
  5. Daily diff cron = PR bot. "Meta has an internal system that allows developers to set up what is essentially a cron job that produces a daily batch of diffs based on user-defined selection criteria. This system also helps choose relevant reviewers, ensures that tests and other validations pass, and ships the diff once it's approved by a human." Plus a web UI for on-demand single-file or module conversion. See patterns/daily-diff-cron-for-automated-migration.
  6. Metaprogramming on broken code is the secret sauce. "Unlike most metaprogramming tools, it is very much not a compiler plugin, so it can analyze broken code across both languages, and does so very quickly." Required because postprocessing runs on Kotlin that has just been machine- translated and frequently does not yet compile. One named transform: scan a Java interface's Kotlin implementers across "several thousand unbuildable Java and Kotlin files" and rewrite override fun getName() → override val name when the interface's property was translated from a getter.
  7. 200+ custom steps exist because J2K alone doesn't build. "Due to the size of our codebase and the custom frameworks we use, the vast majority of conversion diffs produced by the vanilla J2K would not build." That's a blunt framing of why scale-out migrations need the "pipeline with open-ended passes" shape: there is no fixed transform that hits the long tail; each custom step closes one corner case the blog-post-author-of-J2K never saw.
  8. Upstream collaboration unblocked the remaining long tail. In early 2024 JetBrains began adapting J2K to the new K2 compiler; Meta used the opening to fix years-old J2K bugs (e.g. "disappearing override keywords") and insert client hooks into J2K so custom preprocessing / postprocessing can run inside the IDE with J2K's better symbol resolution. Named benefits: improved symbol resolution on third-party libraries, easier open-sourcing of Android-specific steps.
  9. Null safety is the real blocker — not syntax. Kotlin-side dereferences are safe at the type level only if the corresponding Java side is truly @Nullable/non-null. Meta's Java is @Nullsafe (Infer) and some is not, and even @Nullsafe Java throws NPEs via untagged dependents. Translation risk = "someone has to take that initial risk of effectively inserting a nonnull assertion." Canonical example from the post: someMethodDefinedInJava(foo!!) — a missing @Nullable on a Java parameter combined with !! insertion creates "a very unnecessary NPE."
  10. Runtime nullability telemetry closes the long tail. When static analysis can't resolve nullability (Java/Kotlin interop, unannotated dependencies), Meta built a Java compiler plugin that "allows us to collect data on all return types and parameters that are receiving/returning a null value and are not annotated as such." Codemods then backfill @Nullable from the runtime telemetry, so "Java files that we may never translate" become safer too. Canonical wiki datum for "when static analysis stops, instrument production."
  11. Bot-safer-than-human on delicate transformations. "Contrary to popular belief, we've found it's often safer to leave the most delicate transformations to bots. There are certain fixes we've automated as part of postprocessing, even though they aren't strictly necessary, because we want to minimize the temptation for human (i.e., error-prone) intervention." Canonical named example: condensing long chains of null checks — correctness equivalent, but "less susceptible to a well-meaning developer accidentally dropping a negation."
  12. Why translate at all (not just write new code in Kotlin). Remaining non-null-safe Java in the central dependency graph is "an agent of nullability chaos" for the Kotlin above it. Parallel toolchains (Java + Kotlin linters, build configs) are the other ongoing tax, plus: "Compiling Kotlin is slower than compiling Java, but compiling both together is the slowest of all." That mixed-language build-speed cliff is the load-bearing cost argument against "just stop writing new Java."

Operational numbers

Dimension Number (Meta, 2024-12)
Starting Java codebase (approx.) ~10,000,000 lines
Files to convert (approx.) ~100,000
Conversions shipped >40,000
Per-file remote conversion time ~30 minutes
Kotlinator phases 6
Preprocessing steps ~50
Postprocessing steps ~150
Total custom steps (pre+post+build-error) >200
Complementary nullability codemods >12
Kotlin-first Android at Meta since 2020

Systems extracted

  • systems/kotlinator — Meta's internal end-to-end pipeline for Java→Kotlin translation at monorepo scale. 6-phase architecture (deep build → preprocessing → headless J2K → postprocessing → linters → build-error-based fixes).
  • systems/j2k-converter — JetBrains' IntelliJ Java-to-Kotlin Converter. Open-source. The core translator that Kotlinator wraps. URL: github.com/JetBrains/intellij-community/tree/master/plugins/kotlin/j2k.
  • systems/intellij-platform — the JetBrains IDE platform. Kotlinator's first step ("going headless") requires extending IntelliJ's ApplicationStarter and instantiating JavaToKotlinConverter server-side.
  • systems/psi-libraries — JetBrains' Program Structure Interface libraries. Meta's custom metaprogramming tool is built on PSI for both Java and Kotlin. Enables parsing + analysing partially-broken code without a working build.
  • systems/kotlin-ast-tools — Meta's open-source subset of postprocessing transformations. URL: github.com/fbsamples/kotlin_ast_tools.
  • systems/nullsafe — Meta's Java null-safety static analyser (part of Infer-adjacent annotations). URL: github.com/facebook/infer/blob/main/infer/annotations/.
  • systems/nullaway — Uber's Java null-safety static analyser. Named alongside Nullsafe as the incumbent "null-safe Java" tool family this migration is built on top of. URL: github.com/uber/NullAway.
  • systems/javac-plugin — the Java compiler plugin mechanism (not a Meta system). Meta built "a Java compiler plugin that helps us collect runtime nullability data" to close the static-analysis long tail.

Concepts extracted

  • concepts/headless-ide-inspection — take an IDE inspection that was designed to run interactively and drive it from a server process with no UI. Enables parallelism, CI integration, and time-expensive pre/post steps.
  • concepts/metaprogramming-on-broken-code — AST-level tooling that parses + analyses + rewrites code that does not compile, by building on PSI-style libraries rather than compiler plugins. Load-bearing when the intermediate state of a migration is unbuildable by construction.
  • concepts/build-error-driven-fix-loop — use the compiler's error messages as the spec for a final round of fixes: try to build, parse the error list, apply targeted edits (add missing import, insert !!, etc.), rebuild. Closes the long tail without re-implementing compiler logic.
  • concepts/interlanguage-null-safety — Kotlin inserts implicit checkNotNull calls at the Java/Kotlin boundary in bytecode, which makes nullability operational, not just static. Java static analysers (@Nullsafe, NullAway) are 100%-effective only at 100% coverage, which is infeasible on any real codebase.
  • concepts/runtime-nullability-telemetry — instrument the Java runtime (via a javac plugin) to record actual null flows through parameters + return types that aren't annotated. Use the collected data to drive codemods that backfill correct @Nullable / non-null annotations.
  • concepts/bot-safer-than-human — deliberately automate transformations that aren't strictly required, precisely because they're delicate enough that human reviewers will get them wrong. Counter-intuitive framing: scope up the bot to de-scope the human.

Patterns extracted

  • patterns/pipeline-with-open-ended-passes — structure a migration as a sequence of transformation phases where any phase may accumulate arbitrary custom steps over time. No single transform hits the long tail; the pipeline is the long-tail solution.
  • patterns/automated-migration-at-monorepo-scale — the wrapping architectural pattern: scale-out IDE tooling, headless execution, per-file fan-out, daily-diff cron, human-in-the-loop review only for the final diff.
  • patterns/upstream-collaboration-as-migration-unblock — when the migration hits the ceiling of what can be done outside the core tool (here: J2K), invest in upstream contribution + collaboration to add extension points (client hooks, pre/post hooks, symbol-resolution APIs) so the rest of the migration unblocks itself.
  • patterns/daily-diff-cron-for-automated-migration — an internal cron system produces a daily batch of migration diffs against user-defined selection criteria, auto-routes them to reviewers, runs tests, ships on approval. The web-UI on-demand path uses the same pipeline. Functionally equivalent to a PR bot but embedded in Meta's diff-review infra.

Caveats + explicit non-statements

  • No LLM / no agent. The post is explicit that the pipeline is deterministic AST transformations + build-error heuristics. This is notable for a 2024 migration post — the case for rules-based tooling over LLMs at this scale is implicit but strong. Compare +contrast with Instacart's Jetpack Compose migration (AI-skill-based, smaller scope), Cloudflare's vinext (AI-driven framework rewrite, not a translation).
  • No per-phase cost numbers — the 30-minute total is named but the breakdown across the 6 phases is not disclosed.
  • No false-positive / regression rate. The post names

    40,000 conversions shipped but does not disclose the revert / NPE / regression rate after ship. "Over the course of shipping over 40,000 conversions, we've learned about many of these the hard way" is the only window.

  • No build-speed numbers. The mixed-language Java+Kotlin build is "the slowest of all" but no percentages, regressions, or recovery metrics are given.
  • "Well past the halfway point" — the stated progress is directional; the post does not disclose an exact ratio or projected completion date.
  • Only a subset of custom steps is open-sourced. The fbsamples/kotlin_ast_tools repo is named but the full 200+-step Editus-based toolchain is not public; the J2K-client-hook-based migration is mentioned as a future vehicle for open-sourcing.
  • Nullability bias is explicit: when the Kotlinator can't prove nullability, it errs toward String?. That's correct for correctness but adds noise to reviewers. The post calls out reviewer scrutiny on unexpected !! as the counterweight.

Contradictions + extensions to existing wiki

  • Contradicts "AI will do all refactoring" framing. The AI-refactoring economics thesis (Instacart, Cloudflare) centres on LLM agents. Meta's Kotlinator is the deterministic-tooling counterexample at ~50× the code scale: 10M lines translated without LLMs. The two positions are complementary, not opposed — Meta's problem is a translation (source + target languages both syntactically well-defined, million-file monorepo) whereas Instacart / Cloudflare are doing migrations (framework-API shape changes, fewer files, more judgement). Wiki captures both thesis and counter-thesis under the umbrella economics concept.
  • Extends concepts/monorepo with a fifth-kind-of-tax. The monorepo page already notes atomic-change wins and build-system / tooling costs. Meta's post adds an explicit language-migration cost: "Compiling Kotlin is slower than compiling Java, but compiling both together is the slowest of all." Mixed-language monorepos pay a compile-speed tax that polyrepos can avoid by language-per-repo partitioning.
  • Companion Meta migration post — Meta's earlier 2022-10-24 post by Omer Strulovich is referenced but not ingested into this wiki. It covers the adoption-decision rationale (this 2024 post covers the execution infrastructure). Flagged as ingest candidate.

Source

Last updated · 319 distilled / 1,201 read