Critical Challenges in ML-Based SaMDs prompting FDA Action Plan

Author: Navid Toosi Saidy, Quality and Technology Translation Lead at Max Kelsen

At Max Kelsen, we’re developing a range of advanced machine learning-based Digital Health solutions for a range of pressing issues across the healthcare and life sciences industry. We envision this next generation of software tools will be vital to improving healthcare services, diagnostics and disease or injury management and assist healthcare workers to provide the best quality of care to patients.

Software used for diagnostics, prevention, treatment or monitoring of a disease fall into a category called ‘software as a medical device’ (SaMD) and are closely monitored and regulated by authorities such as the Food and Drug Administration (FDA) in the USA, as well as Australia’s Therapeutic Goods Administration (TGA).

While recent endeavours by the International Medical Device Regulators Forum (IMDRF) have provided better clarity around quality management and translation of SaMDs, regulation authorities are under mounting pressure to consider the implications of the fast-paced development of artificial intelligence (AI) and machine learning (ML)-based SaMDs.

Just a couple of weeks ago, the US FDA published an action plan for safe and patient-centric regulation of AI and ML-based SaMD. This action plan has been developed with significant engagement from medical device stakeholders on FDA’s discussion paper released in early 2019 called “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based SaMD”. Overall, the FDA has proposed a “multi-pronged approach” which will be dedicated to providing better oversight of AI/ML-based SaMDs through a five-part initiative that aims to ensure patient safety and algorithm transparency.

Here, we briefly summarise some of the critical challenges in ML-based SaMDs, which have prompted each initiative within the action plan, and discuss the next steps in regulating these exciting new software tools.

1. Tailoring Regulatory Framework for AI/ML-based SaMD

Traditionally, and in the current regulatory environment, modifications to regulatory approved and cleared SaMDs may require a premarket submission depending on the potential risk of those changes to users/patients. In the case of AI/ML-based SaMDs, a premarket submission would be required to the FDA when the modifications resulting from AI/ML driven suggestions impact the device’s intended use, performance, safety and/or introduce changes to the SaMD algorithm. Therefore, given the current framework, updates to ML-based SaMDs could be substantially delayed, leading to a ML dataset shift which can compromise the functionality of the SaMD and hinder the quality of care provided to the patient.

To address this issue, the FDA’s discussion paper proposed a ‘Total Product Lifecycle Management’ regulatory approach to allow for rapid integration of AI/ML-based updates to SaMDs. FDA’s proposal suggested submission of predetermined change control plans, comprising SaMD pre-specifications and algorithm change protocols which described “what” and “how” changes will be incorporated respectively. This will enable SaMD developers to release adaptive software as opposed to locked algorithms and unleash the true potential of AI/ML in device-based diagnosis and treatment without the need for those devices to be completely reassessed. This week’s action plan indicates that the FDA will be expanding on this initiative and are receiving submissions from stakeholders to inform the implementation of this initiative.

2. Good Machine Learning Practice (GMLP)

The establishment of good machine learning practice (GMLP) principles and procedures has also been widely encouraged, and the FDA has committed to working with the International Organization for Standardization (ISO), International Electrotechnical Commission (IEC), Institute of Electrical and Electronics Engineers (IEEE) and British Standards Institution (BSI) to develop consensus standards comprising a robust GMLP. The standardisation of SaMD development procedures will be vital to the quality management of new software and allow more rapid technology translation.

3. Patient-Centred Approach Incorporating Transparency to Users

The third initiative aims to address the substantial lack of public understanding of how ML-based software operates, which can lead to mistrust, poor acceptance and uptake in the community, particularly in healthcare settings. Here, the FDA is proposing to run a series of public engagement workshops to better understand how ML-based software can appear more transparent, with a focus on providing clarity to users on the input data used to train the SaMD and evidence of its performance. The information gathered from these workshops will ideally be used to improve the FDA’s guidance on product labelling, which will ultimately lead to wider market uptake and improved public perception.

4. Regulatory Science Methods Related to Algorithm Bias & Robustness

Algorithm bias is a well-known challenge in ML-based SaMD, where data from skewed, incomplete or non-representative cross-sections of the community can lead to poor performance or limited accuracy of the SaMD in applications extending beyond the data on which it was initially trained. This can be a critical challenge for software developers whose access to historical datasets may be limited by historical bias towards Caucasian populations, young participants, or other non-representative participant groups.

To address these challenges, a research collaboration between the FDA and leading research teams from Stanford, University of California San Francisco (UCSF) and Johns Hopkins Universities has been established to identify methods for correcting, mitigating or accounting for training data bias to reduce the impact on SaMD performance in the market.

5. Real-World Performance (RWP)

The final initiative builds on point 1, inviting SaMD developers to consider implementing real-time monitoring of software performance after market translation to ensure continued accuracy, precision, specificity, and sensitivity.


Overall, Max Kelsen strongly supports the FDA’s proposal to develop best practice guidance. With further development, we hope clear functional requirements will be established to bring the SaMD development community to a consensus on regulatory requirements which ultimately improve the safety and performance of the products alongside public perception and market uptake. There are still several challenges which need to be addressed by the FDA, and this recent action plan only outlines what needs to be undertaken — but it’s a great start and keep watching this space as the field develops.

We are an Artificial Intelligence and Machine Learning consultancy that delivers competitive advantage for government and enterprise.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store