Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jan;108(1):198-214.
doi: 10.1109/JPROC.2019.2946993. Epub 2019 Oct 23.

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Affiliations
Free PMC article

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Tom Vercauteren et al. Proc IEEE Inst Electr Electron Eng. .
Free PMC article

Abstract

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

Keywords: Artificial intelligence; computer assisted interventions; context-aware user interface; data fusion; interventional workflow; intra-operative imaging; machine and deep learning; surgical data science; surgical planning; surgical scene understanding.

Figures

Fig. 1
Fig. 1
Interactive algorithms are required to deliver context-aware artificial intelligence. In this example, using the algorithm presented in [8], brain tumour segmentation is initially performed automatically using a pre-trained algorithm. As part of the surgical planning, the user may want to refine the segmentation by providing scribbles to denote areas that should be excluded (green) or included (pink) irrespective of the initial segmentation. The algorithm then adapts its output to respect the user input.
Fig. 2
Fig. 2
Realistic simulation of X-ray image formation from pre-operative CT is one possibility to create large quantities of well annotated images. Pipeline represents the simulation approach described in [57].
Fig. 3
Fig. 3
Endoscopic video (top), monocular depth estimate (middle), and rendering of a photorealistic reconstruction (bottom). Results were achieved using the self-supervised method described in [112].
Fig. 4
Fig. 4
Capturing the 3D context of the operating room is necessary for providing AI-based decision support and monitoring risk. In this example, the staff radiation exposure during a X-ray based procedure is computed in-situ via simulation and displayed with augmented reality in a training scenario [134].

Similar articles

See all similar articles

LinkOut - more resources

Feedback