CONCEPT Cited by 1 source

Modality Masking During Training

Modality masking during training is the discipline of randomly removing one or more modality inputs from each training example so the model learns to predict without them. It is the multimodal-ML analogue of dropout: a regularisation technique whose purpose is to simulate deployment-time modality absence and force the model to avoid over-reliance on any single modality.

The framing that earns it a page

"Modality masking during training: remove inputs during development to simulate deployment reality." (Source: sources/2026-04-22-databricks-multimodal-data-integration-production-architectures-for-healthcare-ai)

Named by Databricks as the first of three production-design responses to the missing-modality problem. The core claim is that a model trained only on modality-complete examples learns latent dependencies it cannot actually count on in production; masking during training makes those dependencies optional.

Relationship to dropout and other regularisers

• Dropout drops random neurons; modality masking drops whole modalities' inputs — a structured, semantically meaningful dropout.
• Feature dropout is a closer cousin — drop whole input features rather than neurons — but modality masking operates at a coarser, deployment-aligned boundary.
• Adversarial training fights against worst-case input perturbation; modality masking fights against worst-case input absence.

Implementation shapes

• Per-example uniform masking — each training example drops each modality with some probability p (typically 0.1–0.3).
• Curriculum masking — start with low masking rate, increase over training to gradually expose the model to sparser inputs.
• Deployment-distribution masking — calibrate the masking probabilities to the observed modality-missingness rates in the target deployment population.

Why it pairs with late fusion

Late fusion is structurally graceful under missingness — drop the per-modality model whose input is absent. Modality masking during training is the extra insurance that the per-modality combiner is calibrated for the sparse case: without masking, the combiner might weight modalities based on co-occurrence statistics that don't hold at deployment.

See concepts/missing-modality-problem for the failure-mode framing and patterns/fusion-strategy-selection-by-deployment-reality for where modality masking fits in the production playbook.

Seen in

• sources/2026-04-22-databricks-multimodal-data-integration-production-architectures-for-healthcare-ai — Databricks lists modality masking first among the three production-design responses to missing modalities ("remove inputs during development to simulate deployment reality"), pairing it with sparse-attention / modality-aware models and transfer-learning-from-richer-cohorts. The post's broader framing — "architectures designed for sparsity generalize" — makes modality masking a training-time prerequisite for that generalisation.