What evaluation evidence can be reused, for which uses, under which assumptions, and what makes it obsolete.
Specialized healthcare models, those that predict a toxicity, generate a synthetic cohort or estimate survival, will never have the cross-cutting leaderboard that general-purpose language models enjoy. They live by dossiers, and every dossier asks the same prior question: why should the instrument that produced these numbers be believed? This article argues that the credibility of a model cannot exceed the credibility of the test bench that evaluates it, and that the bench’s own credibility is built from demonstrations that do not age at the same rate. Part of today’s cost of evidence comes from not knowing how to organize their conditional reuse.
The scope is deliberately narrow. Two objects are concerned: AI-enabled medical devices that the AI Act classifies as high-risk through its Article 6 and Annex I, and computational tools whose outputs support a regulatory decision in drug development. A general-purpose model supporting a defined clinical decision shares the methodological requirements, representativeness, independent evaluation, relevant thresholds, without inheriting the same regulatory qualification.
A leaderboard ranks systems on a given protocol. A dossier asks that a result be accepted for a given decision. The two are not competitors; they answer different questions. The 2026 Nature Medicine comparison by Vishwanath et al., in which three general-purpose models outperformed two commercial clinical tools, and the Matters Arising by Beaulieu-Jones and Nemati that followed, are the clearest recent illustration, already discussed in Same Benchmark Performance, Opposite Care: the ranking is informative, and stretching it toward procurement, reimbursement or regulation is where it stops carrying weight. What produced the number is a separate question, treated in A Benchmark Measures Relative Performance, a Decision-Maker Must Govern Its Provenance; this article asks what makes the producing instrument credible.
The word benchmark is overloaded. The article speaks instead of a test bench: evaluation choices, test and reference data, protocol, execution tooling, and the applicability domain outside which the bench says nothing. Uberti-Bona Marin and co-authors identified in 2026 the four normative choices that accuracy under the AI Act presupposes: metrics, trade-offs between them, representative data, thresholds.
The first cut separates integrity from validity. Sealing, versioning and logging tell us what ran, on what, in which order. They say nothing about whether the bench measures the property that matters. Reproducibility is a property of the procedure; validity is a property of the relation between the procedure and the world. A bench can be impeccable on the first and wrong on the second, indefinitely and reproducibly.
Extending the verification and validation distinction of ASME V&V 40, and the V3 framework of Goldsack et al. for digital health measurements, the article requires five separate demonstrations. Computational correctness: the software computes the announced metric. Measurement validity: the metric informs the claimed property. Reference quality: labels, events and comparators are reliable enough to serve as a standard. Discriminating capacity: the bench catches relevant defects, including deliberately injected ones. Decisional relevance: the result supports the decision at hand, not a neighbouring one.
Discriminating capacity is singled out as the most neglected and the most enforceable. A bench that has never faced negative controls, known reference cases, injected defects or naive baselines has not shown that it can reject a defective model. For synthetic cohorts the test is concrete: the bench must catch a generator that preserves marginal distributions while destroying the relationships between variables, and a generator that memorizes real patients. Catching injected defects establishes capacity on those defects, not exhaustive coverage, which is why the bench must document its own limits, as a digital twin cannot be validated in the mirror of its own training distribution.
The thermometer metaphor has a ceiling, and the text names it. A bench mixes fidelity, utility, privacy and acceptability; part of that is measurement, part is a choice about what matters and what risk is acceptable. That choice is not demonstrated. It is declared and justified.
The harder cut concerns systems that generate their own models. On the TweenMe implementation ground, several dozen models span heterogeneous task families, and some are produced by an optimization agent that hybridizes architectures and proposes trade-offs between criteria. There, bench scores flow back into model construction, and the bench becomes an optimization component rather than a final judge.
A dataset can stay sealed and unchanged while being progressively overfitted by the decisions taken on its scores, the selection bias Cawley and Talbot documented in 2010. Exhaustive logging makes the adaptation visible; it does not remove it. Three functions must therefore be kept apart: training, architecture selection and trade-off arbitration, and confirmatory evaluation shielded from both. On small cohorts this is not a three-way split of patients but a control of information flows, through nested validation, external data or prospective data. The operative criterion is stated as a compression: informational independence takes precedence over chronological precedence.
Statistical power belongs to qualification as well. In the 184-patient BRAF V600E metastatic NSCLC cohort used in work accepted at ISPOR Europe 2026, raw sample size says little; events and censoring structure drive precision, which collapses in subgroups. Generating more synthetic patients does not create more independent clinical information. For external control arms, fidelity is not enough either: population comparability, time zero, event measurement, censoring and the assumptions that license a causal reading all have to be examined, because a generator can reproduce an observational database together with its biases.
The absence of a universal bench does not mean rebuilding everything per dossier. The article proposes three levels. Infrastructure (execution tooling, sealing and versioning of test data, traceability, access control) is demonstrated once and maintained. The task family (survival protocols, censoring, calibration analyses, privacy tests for synthetic data, batteries of negative controls and injected defects) is demonstrated per family and reused within it. The specific use (target population, comparator, thresholds, error consequences, deployment conditions) is demonstrated each time, and should be, since that is where the decision lives. The debate shifts from one thermometer versus many to a more useful question: which part of the evidence survives into the next dossier.
Tool qualification exists and is designed to pool effort. The EMA’s March 2025 opinion on AIM-NASH qualified an AI as a measurement instrument for liver biopsy reading, not as an evaluator of other AI. Article 15 of the AI Act requires declared accuracy levels and metrics and tasks the Commission with encouraging benchmarks with metrology authorities; its real common instruments are harmonised standards and, failing those, common specifications (Articles 40 and 41). As of September 2026, the CEN-CENELEC parts on accuracy and robustness had not reached public enquiry, and the Digital Omnibus (Regulation (EU) 2026/1744) postponed the deadline for AI embedded in medical devices to 2 August 2028, the outcome of the Annex I arbitration discussed in Annex I Trilogue: the track signs the dossier, not the system.
On the US side, the figures measure throughput rather than capacity: 7 clinical outcome assessments qualified out of 86 listed, about six years each (Yang et al., 2025), and 21 MDDT tools qualified since 2017, none presented as AI. The FDA’s Regulatory Science Tools catalogue, unqualified but reusable, already offers task-family building blocks for AI evaluation, and its January 2025 draft guidance on AI in drug regulatory decision-making scales requirements to model influence and decision consequence. A dated benchmark of practice: 126 of 130 FDA-cleared AI devices between 2015 and 2020 were evaluated on retrospective data only (Wu et al., 2021).
A notified body would rightly object that ISO 13485, clause 7.6, already requires validation of monitoring and measurement software. That covers computational correctness for the infrastructure level, not measurement validity or reference quality; when such tooling is revalidated per dossier, it is often because vendors have not placed it where the quality system expects it. The vendor experience behind the article suggests that evidence cost does not fall appreciably between dossiers with different intended purposes, and that a non-negligible share sits at the infrastructure level: a potentially poolable perimeter, explicitly not a demonstrated saving.
Other industries grow confidence in tools rather than rebuilding it: ISO 26262 part 8 (confidence from use), DO-178C with DO-330 (qualification scaled to what the tool’s output allows one to stop verifying), the FDA’s 2023 computational modelling credibility guidance built on ASME V&V 40. These are methodological analogies, not transposable obligations. The use-specific level does not pool automatically, and should not be expected to. The article compresses the asymmetry: the cost of use-specific evidence follows the diversity of decisions; the infrastructure debt follows only the profession’s delay.
A heterogeneous ecosystem of models has already reached regulatory acceptability without a common leaderboard. The OECD adopted five principles for (Q)SAR validation in November 2004: defined endpoint, unambiguous algorithm, defined domain of applicability, appropriate measures of goodness-of-fit, robustness and predictivity, mechanistic interpretation where possible. REACH accepted such results under conditions in 2006; ICH M7, in 2014, let two complementary methodologies, expert rule-based and statistical, conclude on mutagenic impurities without further testing. Producers kept their models. What was pooled were acceptance criteria. A common bench, the Japanese NIHS blind challenge on more than 12,000 substances, came later and served comparative improvement, not qualification.
The transposition has limits, which the article states: mutagenicity is a narrow, quasi-binary endpoint, and timelines ran from two years to ten. What transposes is a form, shared qualification requirements per task family, a method rule, expert review. And since vendors producing models at scale have an interest in seeing their criteria become the norm, those proposals should be published, contestable and examined by a third party of known funding with access to failed trials. Publication does not prevent capture by dominant players; it makes capture visible.
Pooling has a price: it concentrates failures. A computational error or a flawed reference in a shared bench can invalidate dozens of models at once. One must therefore know which models depend on which bench version, which evidence a discovered defect invalidates, which changes require re-evaluation, and when a test set has become too well known. Regulators are equipping themselves: AIM-NASH qualifies a locked tool, and the FDA’s predetermined change control plans, final guidance December 2024 updated August 2025, specify in advance which modifications remain covered. Reusable evidence is not frozen evidence; it is evidence whose dependencies are known and which can be revoked and recomputed. What becomes obsolete is not always the result, sometimes only its interpretation or its right of reuse. Versioning protects integrity; only re-examining assumptions tells whether validity still holds.
Evidence also ages in months of operation. Under the AI Act the responsibility is shared: the provider makes the system loggable (Article 12), declares accuracy, metrics and log-interpretation mechanisms in the instructions for use (Article 13) and runs post-market monitoring (Article 72); the deployer monitors, reports and suspends use in the situations provided for (Article 26). The texts neither require handing a full bench to the operator nor allow monitoring to be transferred by handing over a method. The defensible industrial translation is an in-service bench shipped with the system: properties to monitor and the data they need, indicators with references and limits, analysis frequencies and sample sizes, triggers for investigation, re-evaluation or suspension, and allocation of roles.
Logging reconstructs events, telemetry collects signals, evaluation gives them meaning. Three blind spots follow. Continuous monitoring is not continuous measurement of clinical performance: survival calibration or rare-event performance need observed outcomes, sometimes months later, and biased follow-up yields a biased picture. The in-service bench must itself be evaluated on what it detects, how fast, with how many false alarms, and what it misses, in hands and on populations its author does not control. And monitoring is a legitimate recurring cost, not a debt: it lasts as long as use does. The delivered product therefore includes the initial evidence, the monitoring instrumentation and the rules for acting on its signals. An evaluation that never leaves the factory proves only the factory exit.
The main ground is a single vendor, and the data behind the economic argument are not published: it should be read as a reasoned hypothesis. A cost that does not fall between models with different purposes does not, by itself, prove an absence of learning. The industrial standards cited are paywalled and paraphrased from summaries by parties with an interest in tool qualification. A harmonised standard or common specification on accuracy, if adopted, would shift part of the argument toward its fitness for the models concerned.
Separate selection from confirmatory evaluation. Demonstrate the validity of the bench, level by level. Make every reusable piece of evidence revocable. Ship with the system the means to know when its conclusions are no longer supported. A leaderboard says who comes first on a given protocol; a conclusion must state what makes it obsolete.
Full argument, regulatory references, the QSAR precedent and the bibliography in the PDF below (13 pages).
See also: Same Benchmark Performance, Opposite Care · A Clinical Benchmark Is a Perimeter Decision · Public Benchmarks Have Lost the Right to Decide Alone · When Evaluation Becomes the Infrastructure of Concentration · An Applicability Domain Measures a Proximity, Not a Capacity · Evidence Is a Conditional Promise · From Certified Product to Continuous Authority · The Contractualised Promotion Port · Benchmark Performance Is Not Deployability
Doctrinal notes and explorations on AI in regulated systems. Once or twice a month. One-click unsubscribe.