Operational substitutability, declared applicability domain, and out-of-domain refusal: what an executive committee actually carries before a regulator
A digital twin in healthcare is not validated by measuring its resemblance to the real. It is validated by the guarantees it brings to the decisions it replaces. The sentence looks trivial once written; it nonetheless contradicts almost every criterion mobilized, in the boardroom, to defend a synthetic cohort before an authority.
The scene is familiar. A generator produces data a discriminator struggles to separate from the real, an expert validates by eye, and one concludes the twin is “good”. That conclusion conflates two objects that everything separates: the quality of an imitation and the robustness of a substitution. Realism is not a bad indicator. It is a specialized one, relevant only when the level it captures matches the task being claimed.
The statistical fidelity of a synthetic cohort does not measure its usefulness for a task. The methodological community has, for the most part, already settled this point. The displacement persists elsewhere, at the layer that actually decides an industrial dossier: deployment, governance, regulatory accountability. That is where a twin is still judged by its apparent realism, and that is precisely where the criterion is wrong.
The relevant question is never “does the twin resemble the real?”. It is “for what use does one wish to replace the real, and under what guarantees?”. A twin is never deployed in general: it trains a model, builds a virtual control arm, estimates a recruitment strategy, calibrates a threshold, tests a clinical policy. Each use carries its own notion of fidelity, and global distributional resemblance captures only one of them — rarely the one that governs the decision.
All twins begin as statistical portraits. Some become decision instruments. The difference is structuring. A portrait seeks to represent a population. An instrument seeks to produce a decision faithful enough to replace, in a given context, the one that would have been made from real data. The moment a twin enters a clinical, regulatory, or industrial loop, what is evaluated is no longer the imitation: it is the decision loop in which it intervenes.
A dossier is not filed as a resemblance score. It is filed as a chain of guarantees. The PDF note unfolds the full demonstration; the skeleton is retained here. First, the absence of leakage: the real evaluation cohort must never have contributed to generating the synthetic data, failing which the measured performance reflects only contamination. Then operational substitutability, for which the Train-on-Synthetic / Test-on-Real methodology — train exclusively on the synthetic, evaluate on an independent real cohort, formalized by Esteban, Hyland and Rätsch (2017) for medical time series — provides the frame, provided it preserves not discrimination alone but also calibration and decision benefit when those dimensions govern the act. Finally, the declared applicability domain: not a guarantee, but a refutable hypothesis that surveillance must be able to disprove.
Beyond its declared domain, a governable twin does not produce a confident answer. It produces a refusal. That refusal does not fall out of the generator as a spontaneous property: it presupposes an explicit mechanism for detecting out-of-domain situations, and as such it is an architectural decision. The consequence is uncomfortable and must be held: the system will sometimes decline to answer precisely where it was hoped for most. This is not a failure; it is the honest form of governability, set against the silent self-assurance of a system that answers everywhere without ever knowing where it ceases to be valid.
This line separates two regimes that the scarcity objection forces apart. In the anchored regime, a real validation cohort exists and substitutability is demonstrated in the strong sense. In the extrapolated regime — underrepresented populations, orphan diseases, never-observed events — no ground truth is available: substitutability is no longer demonstrated, it is bounded and monitored, under the sole framing of the declared domain that out-of-domain detection must be able to invalidate in real time.
A twin’s deployability cannot be deduced from its generator’s performance alone. An excellent generator can leak confidentiality, collapse rare modes, poorly preserve the relevant dependencies, or fail to extrapolate. Conversely, data that remain easily distinguishable from the real may allow excellent substitutability for a precise task. ToxTwin, an implementation ground and not a universal proof, illustrates exactly this decoupling: global distributional resemblance fails, substitution for the task considered succeeds. Indistinguishability is neither a necessary condition nor a sufficient proof of deployability. PREDICARE sheds light on the symmetric problem: a triage twin must not only decide, it must recognize the situations where its decision is no longer guaranteed.
This governability stops being a formula the moment it is anchored to the instrument that already encodes it. The predetermined change plans regulators are beginning to recognize for learning-based devices — the FDA’s Predetermined Change Control Plan being the most accomplished expression — authorize in advance a bounded envelope of changes, under surveillance, rather than freezing a model. The doctrine of the promotion port, developed elsewhere in this series, offers its architectural primitive.
A digital twin is therefore not an autonomous object. It is a component of a decision architecture, and like any critical architecture, it is not validated in the mirror. It is validated by the guarantees it brings to the decisions it replaces.
A twin that resembles is shown. A twin that replaces is governed.
[Series: Digital Twin in Healthcare — 9/12 · Sunday closing article. The full demonstration, the ToxTwin protocol, and the distinction between the two regimes are in the PDF note above.]
Doctrinal notes and explorations on AI in regulated systems. Once or twice a month. One-click unsubscribe.