The development and validation of software in healthcare is a complex process that is becoming increasingly important, especially with the introduction of the MDR (Medical Device Regulation) and the ‘Convenant for the Safe Application of Medical Technology’. Recently, the ‘Guideline for Validation of Software as a Medical Device’ was published to help physicians validate software from purchase to implementation. Together with Paul, my colleague from the working group that drafted this guideline, I look back on the process and where we stand now.
At the time, I assembled a panel of physicians and, together with them, delved into the topic using literature and expert input on digital strategy. This not only yielded valuable insights for the working group but also raised open questions. For example: how important is clinical expertise in software development?
Our outdated digital infrastructures often fail to provide sustainable, safe, or reliable data for validation, making data curation intensive and costly. How do we ensure continuous updates of prediction models using prospectively collected data? How do we safeguard interests in the collaboration between physicians and developers? And how do we maintain oversight of clinical risks? There remains much to explore outside the scope of this guideline.
Validation Vacation for Physicians?
According to Paul, clinical work will increasingly need to be interrupted for software validation—a ‘validation vacation’. The guideline helps with this, but the pace of new software and updates makes it more complex. Even minor changes in the EHR can take hours to validate.
We expect the developer to guarantee sound and safe products. The same applies to physicians: they must identify where risks lie and when warnings are necessary.
Exploring Beyond the Scope is Unavoidable!
The guideline emphasizes the importance of a broader perspective on the entire software lifecycle. Physicians must be integrally involved to contribute their expertise—something developers, policymakers, lawyers, and other domain experts cannot provide! Physicians must also ensure that legislation, frameworks, standards, or norms continue to add value to health.
Physicians will increasingly immerse themselves in relevant domains such as ICT, law, and policy to provide crucial input. Think of it as an LLM prompt: the more specific the formulation, the more useful the answer. But also knowing when to release the LLM: an art, because it provides insights you would never have discovered on your own! And that should be a breeze, since making that connection with the patient is core business for the physician!
Paul recalls how he used to program every piece of code himself, so he knew exactly how everything worked. Nowadays, it’s different. Instead of toiling for hours, we now pull tools from digital libraries. You call up those tools and let them do the work until it meets your needs. This ‘vibe-coding’ offers convenience: the AI agent quickly solves problems with access to global solutions. But this convenience raises questions about control and reliability.
‘Crank-Software’ Validation?
Paul compares trust in software to a ‘conspiracy message’: it looks logical, but somewhere there’s a flaw in the reasoning. With software, it’s difficult to find such a hidden defect. Validation checks whether software works as expected, but more importantly, whether the software warns you with an error message when something goes wrong. For example, Paul once discovered an error in cytostatic dosing that the software failed to detect. Tools from online libraries raise additional questions: have they been validated, by whom, and under what conditions?
False Sense of Control
The guideline suggests control over software validation, but in reality, risks, structure, and flaws are often unknown. This can lead to health risks without physicians being aware. Or worse: intentionally deployed software risks in the interest of a malicious individual or group!
According to Paul, it’s no different than before: you still have to expect the developer to guarantee sound and safe products. And the same goes for physicians: they must indicate where risks lie and when warnings are necessary.
Paul de Wolf is a hospital pharmacist and Chief Pharmacy Information Officer at Haga Teaching Hospital, the Hague, The Netherlands.
