Major hurdles for Microsoft's HoloLens as a tool in medicine have been accessing tracking data, as well as a relatively high-localisation error of the displayed information; cumulatively resulting in its limited use and minimal quantification. The following work investigates the augmentation of HoloLens with the proprietary image processing SDK Vuforia, allowing integration of data from its front-facing RGB camera to provide more spatially stable holograms for neuronavigational use. Continuous camera tracking was able to maintain hologram registration with a mean perceived drift of 1.41 mm, as well as a mean sub 2-mm surface point localisation accuracy of 53%, all while allowing the researcher to walk about a test area. This represents a 68% improvement for the later and a 34% improvement for the former compared with a typical HoloLens deployment used as a control. Both represent a significant improvement on hologram stability given the current state-of-the-art, and to the best of the authors knowledge are the first example of quantified measurements when augmenting hologram stability using data from the RGB sensor.
Keywords: Microsoft HoloLens augmentation; augmenting hologram stability; cameras; continuous camera tracking; front-facing RGB camera; high-localisation error; hologram registration; holography; image registration; mean perceived drift; medical image processing; minimal quantification; neuronavigation; object tracking; proprietary image processing SDK Vuforia; size 1.41 mm; spatially stable holograms; surface point localisation accuracy; tracking data; typical HoloLens deployment; vuforia tracking.