FDA's Evolving Cybersecurity Expectations for AI/ML submissions
Following FDA's January 2025 draft guidance on AI-Enabled Device Software Functions and FDA recognized AAMI report CR515:2025 Cybersecurity considerations unique to machine learning-enabled medical devices, we have noticed a dramatic increase in the amount of cybersecurity questions from FDA specific to AI/ML devices (e.g., data management and control, data poisoning and/or access to personal data). Companies without traditional "cyber" devices are also often surprised by the depth of FDA's questions, which cover products that do not connect to the internet (e.g., have a single USB port) or for which FDA has not asked for cybersecurity documentation in the past.
The FDA guidance and AAMI standard together establish a comprehensive framework for addressing the unique cybersecurity risks posed by machine learning-enabled medical devices. These documents recognize that AI/ML systems introduce novel attack surfaces beyond traditional software threats, requiring targeted security controls throughout the product lifecycle. This is also against the backdrop of federal and state privacy, security, and AI laws and increased scrutiny, particularly by state Attorneys General, on the use of AI in health care and use of health data to train AI models.
The FDA guidance positions cybersecurity as a core component of post market management alongside device performance monitoring. For devices meeting the new "cyber device" definition under Section 524B of the FDCA, manufacturers must address security objectives, including authenticity, integrity, authorization, availability, confidentiality, and secure updatability. The guidance explicitly links AI-specific risks, such as data poisoning, model inversion/stealing, model evasion, data leakage, overfitting exploitation, model bias manipulation, and performance drift, to potential cybersecurity threats.
AAMI CR515:2025 provides detailed threat modeling guidance for MLMDs, enumerating 15 distinct threat types spanning the ML algorithm lifecycle. These include:
- Data and model poisoning: Manipulation of training data or model parameters to degrade performance or introduce backdoors.
- Adversarial evasion: Crafted inputs designed to cause misclassifications at runtime.
- Model inversion and membership inference: Attacks that reverse-engineer models to extract sensitive training data.
- Output integrity attacks: Manipulation of model predictions during inference.
- AI supply chain attacks: Compromised ML libraries or third-party components.
Both documents emphasize proactive mitigations. FDA recommends adversarial training, data validation and authentication, cryptographic integrity checks (e.g., hashes on training data), differential privacy, secure multi-party computation, access controls, anomaly detection, input verification, watermarking, and continuous model performance monitoring. AAMI CR515 supplements AAMI SW96's security risk management process with ML-specific guidance on threat modeling, risk assessment tied to patient safety impacts, and lifecycle-stage mitigations.
In alignment with the 2025 FDA Guidance: Cybersecurity in Medical Devices, marketing submissions for AI-enabled devices should include cybersecurity risk management reports, threat models addressing AI-specific considerations, Security Use Case Views, fuzz testing and penetration testing results, and descriptions of controls preventing data leakage. This documentation must demonstrate how AI cybersecurity risks are identified and controlled across premarket and post market phases.


