Skip to content

PATTERN Cited by 1 source

Adapter lifecycle management

Intent

Prevent accumulated LoRA adapter patches from degrading model quality through uncontrolled interaction, using three lifecycle rules that parallel software dependency management.

Problem

Stacked LoRA patches interact — a direct case of CACE ("changing anything changes everything") from Sculley et al. (2015). Each patch is self-contained in isolation, but the combined stack can drift in unpredictable ways as patches accumulate.

Three lifecycle rules

1. Fuse co-triggering patches

Patches firing on overlapping inputs share relational neighborhoods. Naive stacking creates subspace interference. Solution: learnable fusion resolves the interference (per Gao et al., 2024 on LoRA adapter composition).

2. Retrain on accumulation

When the number of patches against one category approaches the empirical ceiling (~hundreds of examples per Pletenev et al., 2025), fold the patches into a clean full retrain. This prevents the adapter stack from crossing the threshold where reasoning degrades and the model becomes overconfident in hard-to-detect ways.

3. Unload unused patches

Every loaded patch needs revalidation when upstream changes. Patches not triggered within a defined window are automatically unloaded, reducing: - Revalidation overhead - Interaction surface area - Inference-time compute cost

Analogy to software

Adapter lifecycle rule Software equivalent
Fuse co-triggering Merge related PRs / consolidate patches
Retrain on accumulation Major version release (clean slate)
Unload unused Deprecate and remove dead code

Seen in

Last updated · 571 distilled / 1,747 read