Six dimensions to evaluate what almost nobody measures
The predominant question in literature, conferences and calls for proposals remains: how to improve model performance? This focus, legitimate at an initial technological stage, becomes problematic when it delays examination of architectural deployment conditions. Model performance is not an objective invariant — it depends on the chosen benchmark, often monocentric, retrospective, cleaned of real clinical noise. The change in unit of analysis is decisive: the starting point is no longer the algorithm seeking its insertion point, but the real socio-technical system identifying where an algorithmic capability can reduce a specific uncertainty without generating complexity greater than the gain. The same model, inserted in two different architectures, will produce radically different effects — not because the algorithm changes, but because the system in which it operates changes.
AI does not insert itself into “medical decision-making” in general. It inserts itself into a specific decision topology. Four minimal configurations: individual synchronous decision (emergency), individual asynchronous (scheduled consultation), collegial synchronous (tumor board, staff meeting), distributed asynchronous (chronic pathway with multiple actors). Each context is described by a minimal triplet — D (time per case), N (number of actors), L (latency tolerance). In an oncological tumor board (D≈5 min, N≈6-10, L≈0), a tool requiring three minutes of explanation within a five-minute slot will not be adopted. The incompatibility is not algorithmic — it is topological. The same recommendation, unacceptable in a tumor board, may integrate without friction in a general practice consultation (D≈15-20 min, N=1, moderate L).
The clinician implicitly calculates: T_net = T_returned − (T_learning + T_integration + T_evaluation + T_coordination). If T_net is durably negative, the tool will not be integrated regardless of its statistical performance. This criterion is falsifiable and measurable under real-world conditions, interruptions included. In hospitals, ROI depends on the observation scale — clinician, department, institution, territory — and these levels do not necessarily converge. The investor is not always the beneficiary. Strong coupling increases maintenance and DRP costs, contractual dependency limits negotiation capacity, absence of reversibility transforms an initial investment into a structural constraint.
A surgeon hesitates between two operative strategies; an oncologist faces an atypical profile in second-line treatment. In these situations, clinical value is not measured in minutes saved but in uncertainty reduced. The uncertainty delta ΔI = I_before − I_after can be approximated by inter-clinician variance, tumor board discordance rate, diagnostic reclassification frequency. A tool may slightly increase decision time yet be accepted if the uncertainty reduction is substantial. The framework becomes bidimensional: time and decisional entropy. An AI is relevant if it improves one or the other without excessively degrading the other.
Coupling describes the degree of structural interdependence between the AI component and the existing information system. Three typical configurations: encapsulated (weak coupling — autonomous component, API-first, deactivatable without major alteration, such as TweenMe), integrated (moderate coupling — AI embedded within an existing application, constrained by the vendor’s lifecycle), diffuse (strong coupling — AI functionally distributed across multiple application layers, such as PREDICARE’s agentic architecture). Coupling must be evaluated across three planes: technical, contractual and regulatory. A technically encapsulated component can remain strongly coupled contractually. Design rule: coupling degree must be proportional to the model’s proof maturity and to the criticality of the supported decision.
The classic error: evaluating impact solely from the end user’s perspective. AI generates a distributed load — clinician (cognitive, temporal), IT department (integration, maintenance, monitoring, cybersecurity, multi-vendor coordination), governance (regulatory compliance, technical documentation, post-market surveillance), organizational (training, change management), sovereignty (infrastructure, data location, strategic dependency). The rarely calculated upstream TCO includes integration, continuous maintenance, regulatory and reversibility costs. A tool may have a locally positive T_net yet remain globally non-viable if IT or regulatory load exceeds organizational capacity. The deeper error is failing to think a priori about the means of measuring medico-economic effectiveness.
A clinical system is a critical system. Introducing an AI component must never transform an assistance into a single point of failure. Three requirements: functional reversibility (the system must operate without the AI component; withdrawal plan formalized and tested in simulation), redundancy and business continuity (multi-node, DRP and BCP explicitly integrating the AI component, failover times compatible with clinical criticality), graceful degradation (continuous monitoring, explicit alert thresholds, progressive scope reduction). An unavailable model is visible; a silently degraded model is dangerous. Graceful degradation must be tested periodically like a fire drill.
These six dimensions form a system of coupled constraints. Structural fragility zone: negative T_net in target topology, strong coupling with low-maturity model, absent DRP integrating AI component, untested reversibility, uncompensated IT load. Acceptable experimentation zone: weak or encapsulated coupling, individual or asynchronous topology, active drift monitoring, periodically tested reversibility. A severe deficit on a single dimension suffices to compromise the whole. This framework is not an anti-technology tool — it is an anti-naive-deployment tool. The index card does not require a six-dimension framework: it works, it is available, it does not crash, it cannot be ransomed. It will survive as long as digital systems do not offer an equivalent level of robustness.