A Patient-Specific Mixed-Reality Visualization Tool for Thoracic Surgical Planning

Ann Thorac Surg. 2020 Jul;110(1):290-295. doi: 10.1016/j.athoracsur.2020.01.060. Epub 2020 Mar 5.


Purpose: Identifying small lung lesions during minimally invasive thoracic surgery can be challenging. We describe 3-dimensional mixed-reality visualization technology that may facilitate noninvasive nodule localization.

Description: A software application and medical image processing pipeline were developed for the Microsoft HoloLens to incorporate patient-specific data and provide a mixed-reality tool to explore and manipulate chest anatomy with a custom-designed user interface featuring gesture and voice recognition.

Evaluation: A needs assessment between engineering and clinical disciplines identified the potential utility of mixed-reality technology in facilitating safe and effective resection of small lung nodules. Through an iterative process, we developed a prototype employing a wearable headset that allows the user to (1) view a patient's original preoperative imaging; (2) manipulate a 3-dimensional rendering of that patient's chest anatomy including the bronchial, osseus, and vascular structures; and (3) simulate lung deflation and surgical instrument placement.

Conclusions: Mixed-reality visualization during surgical planning may facilitate accurate and rapid identification of small lung lesions during minimally invasive surgeries and reduce the need for additional invasive preoperative localization procedures.

Publication types

  • Case Reports
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adenocarcinoma / diagnostic imaging*
  • Adenocarcinoma / surgery
  • Augmented Reality*
  • Humans
  • Image Processing, Computer-Assisted*
  • Lung Neoplasms / diagnostic imaging*
  • Lung Neoplasms / surgery
  • Software
  • Thoracic Surgical Procedures
  • Tomography, X-Ray Computed*