Axis III · D9
Human and organizational transformation
Thesis. AI systems rarely fail by technical defect in regulated environments. They fail because the organization was not prepared to absorb them. Human transformation is not change accompaniment; it is its condition of existence.
The distinction that cuts
Adoption (the user clicks) vs absorption (the organization transforms its processes). The first produces flattering usage metrics; the second produces a verifiable change of practice in the patient record or incident report.
Typical market error
Importing change management mechanisms calibrated for classical IT (flash training, communication, key users), when AI introduces a new epistemic asymmetry. The operator must contest a system that statistically « knows » better on the average case but can fail seriously on the singular case. The competence to train is less use than informed contestation.
Failure signals
No explicit training on the known limits of the system, though this is the critical competence. No documented user-side override protocol. Automation bias unmeasured: override rate decreases with use, without distinguishing whether by justified confidence or cognitive exhaustion. No mechanism to surface cases where the human corrected the machine, hence no organizational learning. Additional cognitive load ignored. AI adds a validation step, hence time, never accounted for.
References
Parasuraman & Riley, Humans and Automation: Use, Misuse, Disuse, Abuse (Human Factors, 1997), the founding reference on automation bias; Bainbridge, Ironies of Automation (Automatica, 1983), more current than at publication; Cabitza et al. on clinical workflow disruption; AI HLEG Ethics Guidelines for Trustworthy AI (EU, 2019) for the ethical frame; HAS recommendations on decision support devices (2020-2024).
Ground of implementation
PREDICARE places the clinician validation phase explicitly before production release, with training centred on cases where the score is wrong (rather than on interface use). The override protocol is a contractual deliverable, not an option. The instance illustrates an approach where absorption is treated as organizational architecture, not accompaniment; it does not prove this approach transposes outside a setting where the clinician retains stable final decision authority.
Articulation
Loops with D5, since without absorption, human oversight remains theoretical. Loops with D8, which fixes its often underestimated cost.