Critical Challenges in ML-Based SaMDs prompting FDA Action Plan

Author: Navid Toosi Saidy, Quality and Technology Translation Lead at Max Kelsen

1. Tailoring Regulatory Framework for AI/ML-based SaMD

Traditionally, and in the current regulatory environment, modifications to regulatory approved and cleared SaMDs may require a premarket submission depending on the potential risk of those changes to users/patients. In the case of AI/ML-based SaMDs, a premarket submission would be required to the FDA when the modifications resulting from AI/ML driven suggestions impact the device’s intended use, performance, safety and/or introduce changes to the SaMD algorithm. Therefore, given the current framework, updates to ML-based SaMDs could be substantially delayed, leading to a ML dataset shift which can compromise the functionality of the SaMD and hinder the quality of care provided to the patient.

2. Good Machine Learning Practice (GMLP)

The establishment of good machine learning practice (GMLP) principles and procedures has also been widely encouraged, and the FDA has committed to working with the International Organization for Standardization (ISO), International Electrotechnical Commission (IEC), Institute of Electrical and Electronics Engineers (IEEE) and British Standards Institution (BSI) to develop consensus standards comprising a robust GMLP. The standardisation of SaMD development procedures will be vital to the quality management of new software and allow more rapid technology translation.

3. Patient-Centred Approach Incorporating Transparency to Users

The third initiative aims to address the substantial lack of public understanding of how ML-based software operates, which can lead to mistrust, poor acceptance and uptake in the community, particularly in healthcare settings. Here, the FDA is proposing to run a series of public engagement workshops to better understand how ML-based software can appear more transparent, with a focus on providing clarity to users on the input data used to train the SaMD and evidence of its performance. The information gathered from these workshops will ideally be used to improve the FDA’s guidance on product labelling, which will ultimately lead to wider market uptake and improved public perception.

4. Regulatory Science Methods Related to Algorithm Bias & Robustness

Algorithm bias is a well-known challenge in ML-based SaMD, where data from skewed, incomplete or non-representative cross-sections of the community can lead to poor performance or limited accuracy of the SaMD in applications extending beyond the data on which it was initially trained. This can be a critical challenge for software developers whose access to historical datasets may be limited by historical bias towards Caucasian populations, young participants, or other non-representative participant groups.

5. Real-World Performance (RWP)

The final initiative builds on point 1, inviting SaMD developers to consider implementing real-time monitoring of software performance after market translation to ensure continued accuracy, precision, specificity, and sensitivity.

Conclusion

Overall, Max Kelsen strongly supports the FDA’s proposal to develop best practice guidance. With further development, we hope clear functional requirements will be established to bring the SaMD development community to a consensus on regulatory requirements which ultimately improve the safety and performance of the products alongside public perception and market uptake. There are still several challenges which need to be addressed by the FDA, and this recent action plan only outlines what needs to be undertaken — but it’s a great start and keep watching this space as the field develops.

We are an Artificial Intelligence and Machine Learning consultancy that delivers competitive advantage for government and enterprise. https://maxkelsen.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store