Verifying and validating AI-based medical devices
Itoro Udofia, director, Medical Health Service at TÜV SÜD, a global product testing and certification organisation, outlines the challenges for medical device manufacturers with AI when it comes to regulation.
Regulatory requirements in the European Union (EU) and other major medical markets do not currently address the unique and complex characteristics of medical devices incorporating artificial intelligence (AI) and machine-learning (ML) technologies. This gap between advanced technologies that are currently available and existing regulations poses a myriad of challenges to medical device manufacturers that are seeking device approval.
By leveraging advanced algorithms and vast amounts of data generated through their routine use, AI-enabled medical devices and software as a medical device (SaMD) can quickly adapt to new information and changing conditions, as well as optimise their performance in real-time. These advantages can lead to improved treatment outcomes for patients, resulting in reduced costs and substantial gains in the overall quality of healthcare everywhere.
Although medical technologies with integrated AI capabilities offer significant potential for improving the quality of healthcare, they also present some important challenges when it comes to assessing safety. The greatest challenge comes from the singular advantage that AI-enabled technologies offer - their ability to adapt their prediction to reflect accumulated data.
Traditionally, the assessment of the safety of medical devices has been based on predetermined and clearly defined risk assessment principles and practices, and ISO 14971 provides medical device developers with a detailed roadmap.
However, many algorithms and data models used in certain AI-enabled medical technologies are not “locked” but instead continuously learn and adapt their functionality in real-time to optimise performance. These technologies may well present one risk profile during the initial product development process and a different risk profile after the device has been deployed for use with patients. Unlike static program code which can be evaluated line by line for its suitability, assessing AI functionality is a less transparent process.
A thorough evaluation of AI functionality is largely dependent on an assessment of both the quality and quantity of data since these factors directly impact how well AI algorithms and models perform. Key aspects affecting data quality can include hidden biases in the selection and collection of data, or errors in data labelling.
A more problematic issue affecting data quality involves the overfitting or underfitting of data, in which data sets either align too closely or not closely enough with the data models being used. Factors to be considered in assessing the quantity of the data required to validate AI models include both the complexity of the AI algorithm and the complexity of the problem that the model is tasked with solving.
Another challenge in the development and assessment of AI algorithms is the difficulty in explaining specifically how the algorithm drives the functionality of the medical technologies. AI models are generally based on a highly nested and non-linear structure, making it difficult to determine which specific input data has determined device function. Given this lack of transparency, it is often difficult to validate the basis or the appropriateness of the model’s process.
There are at present no harmonised standards that specifically address the unique performance aspects of AI technologies. At most, current regulations in major jurisdictions around the world address only specific aspects regarding the assessment of software.
Organisations developing medical technologies with integrated AI capabilities should strongly consider taking a more expansive approach in assessing the safety of their products. Such a holistic approach would address every aspect of the product planning and development process and extend beyond the initial product release date to include rigorous post-market surveillance activities.
To assist developers and manufacturers in evaluating these processes, the Association of Notified Bodies for Medical Devices in Germany (IG-NB) has issued a comprehensive “Requirements Checklist” for assessing the safety of AI-enabled medical technologies. Compiled with the assistance of TÜV SÜD, the IG-NB checklist does not prescribe requirements for AI-enabled medical technologies. Instead, it details a process-oriented approach that considers all the relevant processes and phases of the product development life cycle. The criteria presented in the IG-NB checklist provide a comprehensive assessment of the risks associated with the application of AI in medical technologiesthroughout the entire product lifecycle.
It is important to note that, in situations where a manufacturer outsources key aspects of any of these processes, the specific recommendations of the IG-NB checklist are applicable to these outsourced activities as well. Until appropriate standards and regulations can be developed and implemented, the IG-NB criteria provide a critical intermediate pathway for the approval of AI-enabled medical technologies.