Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; a novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking

PLoS One. 2020 Jan 16;15(1):e0227312. doi: 10.1371/journal.pone.0227312. eCollection 2020.


Objective: Surgical navigation is a well-established tool in endoscopic skull base surgery. However, navigational and endoscopic views are usually displayed on separate monitors, forcing the surgeon to focus on one or the other. Aiming to provide real-time integration of endoscopic and diagnostic imaging information, we present a new navigation technique based on augmented reality with fusion of intraoperative cone beam computed tomography (CBCT) on the endoscopic view. The aim of this study was to evaluate the accuracy of the method.

Material and methods: An augmented reality surgical navigation system (ARSN) with 3D CBCT capability was used. The navigation system incorporates an optical tracking system (OTS) with four video cameras embedded in the flat detector of the motorized C-arm. Intra-operative CBCT images were fused with the view of the surgical field obtained by the endoscope's camera. Accuracy of CBCT image co-registration was tested using a custom-made grid with incorporated 3D spheres.

Results: Co-registration of the CBCT image on the endoscopic view was performed. Accuracy of the overlay, measured as mean target registration error (TRE), was 0.55 mm with a standard deviation of 0.24 mm and with a median value of 0.51mm and interquartile range of 0.39--0.68 mm.

Conclusion: We present a novel augmented reality surgical navigation system, with fusion of intraoperative CBCT on the endoscopic view. The system shows sub-millimeter accuracy.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Augmented Reality*
  • Cone-Beam Computed Tomography / instrumentation
  • Cone-Beam Computed Tomography / methods*
  • Humans
  • Imaging, Three-Dimensional / methods*
  • Neuroendoscopy / methods*
  • Neuronavigation / methods*
  • Phantoms, Imaging
  • Skull Base / surgery*
  • Surgery, Computer-Assisted / instrumentation
  • Surgery, Computer-Assisted / methods*

Grants and funding

The author Marco Lai (M Lai) has received funding from the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No.: 721766 (FBI). website: https://ec.europa.eu/programmes/horizon2020/en. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.