CONCEPT Cited by 1 source

ML technical debt

Definition

Technical debt specific to machine learning systems, arising from the unique properties of ML components: entangled inputs, unstable data dependencies, feedback loops, and the absence of formal specifications for component behavior. First formalized by Sculley et al. in "Hidden Technical Debt in Machine Learning Systems" (NeurIPS 2015).

The CACE principle

Changing Anything Changes Everything — in ML pipelines, components are entangled in ways that make isolated validation insufficient. Unlike traditional software where module boundaries provide containment, ML systems have:

• No formal specifications for what a model "should" do
• Implicit dependencies through shared training data
• Feedback loops where outputs influence future training inputs
• Calibration drift over time without any code change

Key insight from production

The debt accumulates at the seams, not in the components. Individual models, judges, and preprocessing steps may pass their own tests, but their interactions create failures that only surface under end-to-end execution. Component-level confidence creates false assurance at integration boundaries (Kästner et al., 2021).

LLM-specific manifestations

• Judge drift — LLM evaluators produce different scores on identical inputs across runs
• Reference instability — LLM-generated references regenerate as different strings
• Adapter stack interaction — stacked LoRA patches interact unpredictably
• Evaluation noise masking real regressions — small signals drowned by measurement noise

Mitigations

• patterns/deterministic-evaluation-cache — eliminate measurement noise at source
• patterns/end-to-end-seam-validation — test the combined path, not just components
• patterns/adapter-lifecycle-management — prevent adapter stack CACE
• patterns/micro-adapter-hotfix — scoped changes that limit interaction surface

Seen in

• sources/2026-07-14-airbnb-llm-evaluation-infrastructure — Airbnb frames their four-layer stack as applying "boring, well-understood patterns of systems engineering" to where ML's new failure modes actually live; cites Sculley et al. as directly applicable to LLM pipelines