Original editorial illustration of a concept, not a study figure or measured result. Source discussed: Sebin and Karaman Sebin, 2026.
An artificial intelligence system can produce an answer that sounds convincing and fits the question. Does that establish that its use in healthcare is safe? No. Between the output and the patient’s experience sit people, working arrangements, decisions and actions. A new article proposes examining that entire pathway. It offers a starting point for an organisational discussion—not proof that the safeguards it describes have already been validated in clinical practice.
What kind of article is this?
Burhan Sebin and Irem Karaman Sebin published From model failure to system harm: operationalizing a sociotechnical pathway for healthcare AI safety in Frontiers in Artificial Intelligence on 14 September 2026. It is a Perspective: the authors present a way of organising a problem and thinking about possible responses. It is not a randomised clinical trial, validation of a particular product, or professional-society guideline. The full article is available from the publisher.
The authors draw on deliberately selected publications to connect technical and organisational issues. They did not conduct a systematic search or a formal risk-of-bias assessment of the cited studies. There are no new patient data, and the proposed framework has not been prospectively validated as a means of preventing harm. These limitations do not make the discussion worthless. They define the boundary between a useful idea and an established benefit.
Why model quality and patient safety are different questions
Assessing an answer concerns what the system produced. Assessing safety also requires examining how that answer was used. An error may be noticed, challenged or stopped before anyone acts on it. Conversely, a favourable model-testing result cannot describe every condition in a particular service or identify who takes responsibility for the next step.
This does not make measurement of model quality unnecessary. It means that measurement should not end the assessment. Output, decision and consequence are related but different subjects. The article encourages readers to trace their connections rather than treat safety as a single property of software. It proposes a way to examine a working system, not a new score that can automatically label a product safe.
The five stages proposed by the authors
The proposed pathway moves from vulnerability to AI behaviour, then through people and workflow to a decision or system action, and finally to potential harm. This is the authors’ conceptual map. It does not mean every incident follows an identical sequence, or that the probability of moving between stages is known. The explanations below clarify the proposal; they do not validate it.
1. Vulnerability: where could a problem begin?
The first stage concerns conditions that could contribute to unwanted behaviour. Discussion need not wait until an incorrect answer appears. Defining the system’s intended task and the circumstances of its use helps identify areas requiring scrutiny. Naming a potential weakness, however, does not demonstrate that it has been removed. A list of vulnerabilities is not a measurement of a patient’s actual risk.
2. AI behaviour: what did the system actually do?
Next comes the tool’s behaviour. Observing its answer must be separated from interpreting its consequences. Saying that software made an error describes a different part of an event from saying that a patient was harmed. Connecting the two requires examining subsequent steps. The article supplies no universal error rate below which every application would be safe, regardless of its task or setting.
3. People and workflow: what happens to the answer?
The answer reaches a particular person within a particular process. Relevant questions include who reads it, how that person understands their role, and whether concerns can be raised. This stage concerns actual working arrangements, not simply a statement that a human remains involved. A promise of supervision does not explain what the supervisor sees, checks or does when a problem arises.
4. Decision or action: what was done next?
The next question is whether, and how, the output influenced an action. Influence should not be assumed merely because the tool was available. Information displayed by a model is different from a decision made within care. Without that distinction, both benefits and harms can be attributed to AI without properly assessing their connection to its use.
5. Potential harm: what is known about the consequences?
The final stage concerns consequences, not just answer correctness. The article provides no single harm frequency transferable to every clinic. Examples drawn from different publications do not create a shared denominator. The number of reported incidents cannot therefore be converted into an individual patient’s risk, nor can a proportion of flawed answers be treated as a proportion of patients harmed.
Using the idea without presenting it as a standard
The cautious approach is to treat this Perspective as an invitation to discussion between those responsible for the tool and those responsible for care. Instead of beginning with a general assurance of safety, a team can describe what follows the model’s response. Organising these questions is not certification, regulatory endorsement or evidence that the proposed safeguards work.
The following questions are an editorial discussion aid, not a validated checklist or a scored test. Answering them does not produce a verdict that deployment is safe. Their purpose is to identify what a team still needs to understand about its own process:
- What specific task is AI being used for, and which tasks are excluded?
- Who receives the answer, and who reviews the decision made after it?
- What prompts a concern to be raised, and who receives that concern?
- What happens afterwards, and who is responsible for responding?
- How are model-output problems distinguished from workflow problems?
- What information would be needed to assess consequences for patients?
The purpose is not to tick boxes and close the discussion. “We have a procedure” starts an enquiry; it does not describe how the procedure functions. Naming a responsible person similarly does not establish that they have the information they need. These are organisational questions inspired by the article, not experimental findings or advice about treating an individual patient.
What the article does not establish
It does not show how much the framework reduces harm, whether adopting it changes clinical decisions, or what implementation costs. There is no single original model and test dataset against which to assess representativeness, calibration or subgroup performance. Those remain appropriate questions for particular tools and applications, but this Perspective does not supply their answers.
Evidence cited by the authors must also be distinguished from evidence generated by them. Results from a study of one tool do not automatically validate the proposed oversight architecture. Any numerical claim borrowed from that study would require its own source review. This discussion deliberately avoids importing error or harm percentages from the examples.
Reading safety claims carefully
It helps to distinguish intention, arrangement and outcome: “we aim to improve safety”, “we introduced supervision”, and “we examined the consequences”. This is an editorial explanation, not another research model. It illustrates why an ambition or a newly introduced procedure should not be presented as evidence of a patient benefit.
Likewise, “the authors propose” differs from “the authors demonstrate”. Discussing multiple examples does not mean all relevant literature was systematically reviewed. Lack of validation does not prove the proposal is useless, either. It means its potential organisational value must be separated from demonstrated effectiveness in preventing harm. This avoids both excessive promises and premature dismissal of a worthwhile discussion.
What can patients take from this?
Questions about AI need not stop at its name or advertised accuracy. Patients can also ask what role the tool plays, who reviews the decision, and how concerns are handled. This is not an invitation to independently evaluate an algorithm or change treatment. It is a way to have a more concrete conversation about responsibility for care.
The opposite conclusion—that this article proves all healthcare AI ineffective—would be equally unjustified. That was not its purpose. The authors offer a language for examining safety, not a verdict on every product. Potential usefulness and the quality of safeguards must be assessed for a defined application, rather than inferred from general enthusiasm or fear.
Transparency and the main takeaway
The authors declared no financial support for the work and publication. Burhan Sebin disclosed employment at eMerge Americas; the other author declared no conflicting relationships. They also disclosed generative-AI assistance with ideas, language editing and figure development. Reporting these declarations provides transparency, not an independent audit of interests or an automatic reason to accept or reject the arguments.
The central point is straightforward: a good model result does not end the safety questions. The path from answer to decision and consequence still needs examination. This Perspective helps name that path, but does not demonstrate the effectiveness of its own proposal. Its appropriate use is to raise precise questions—not announce that healthcare AI safety has been solved.
Source: Sebin B, Karaman Sebin I. From model failure to system harm: operationalizing a sociotechnical pathway for healthcare AI safety. Frontiers in Artificial Intelligence. 2026;9:1956533. DOI: 10.3389/frai.2026.1956533. Article type: Perspective.