The Reconstructive Metaverse - Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction

Surg Innov. 2024 Oct;31(5):563-566. doi: 10.1177/15533506241262946. Epub 2024 Jun 21.

Abstract

Plastic surgeons routinely use 3D-models in their clinical practice, from 3D-photography and surface imaging to 3D-segmentations from radiological scans. However, these models continue to be viewed on flattened 2D screens that do not enable an intuitive understanding of 3D-relationships and cause challenges regarding collaboration with colleagues. The Metaverse has been proposed as a new age of applications building on modern Mixed Reality headset technology that allows remote collaboration on virtual 3D-models in a shared physical-virtual space in real-time. We demonstrate the first use of the Metaverse in the context of reconstructive surgery, focusing on preoperative planning discussions and trainee education. Using a HoloLens headset with the Microsoft Mesh application, we performed planning sessions for 4 DIEP-flaps in our reconstructive metaverse on virtual patient-models segmented from routine CT angiography. In these sessions, surgeons discuss perforator anatomy and perforator selection strategies whilst comprehensively assessing the respective models. We demonstrate the workflow for a one-on-one interaction between an attending surgeon and a trainee in a video featuring both viewpoints as seen through the headset. We believe the Metaverse will provide novel opportunities to use the 3D-models that are already created in everyday plastic surgery practice in a more collaborative, immersive, accessible, and educational manner.

Keywords: abdominal flaps; breast reconstruction; metaverse; microsurgery; mixed reality; preoperative planning; resident education.

MeSH terms

  • Augmented Reality
  • Humans
  • Imaging, Three-Dimensional*
  • Microsurgery* / education
  • Microsurgery* / methods
  • Models, Anatomic
  • Plastic Surgery Procedures* / education
  • Plastic Surgery Procedures* / methods
  • Virtual Reality