From Medical Research Model towards Secure Clinical AI New!

Artificial intelligence is increasingly being used to support medical imaging and clinical decision-making. In safety-critical environments, however, model accuracy alone is not sufficient. When AI output may influence a clinical decision, the integrity of the model, its data, runtime behaviour, surrounding application and interaction with the end user all become part of the security problem.

In 2019, a U-Net convolutional neural network was evaluated for the segmentation of Third Harmonic Generation (THG) images of human brain tissue. The objective was to rapidly reproduce segmentation masks identifying structures such as dark holes, nuclei and fibres, with the broader goal of supporting intraoperative tissue assessment (Stooker, 2019). The research demonstrated promising segmentation results, while also identifying limitations in some model outputs and downstream tissue classification.