Chromatic differential confocal matrix-based 3D topography and hyperspectral imaging with deep learning for osteoarthritis grading

Abstract

Accurate assessment of knee osteoarthritis (KOA) is limited by the current arthroscopy's reliance on subjective visual inspection. We propose a 3D spatio-spectral automatic grading framework combining deep learning with chromatic differential confocal matrix-based 3D topography and hyperspectral imaging (CDCM-THI). Our digital micromirror device-based system simultaneously captures high-resolution 3D micro-topography and hyperspectral cartilage signatures (lateral: 3.91 µm, axial: 1.96 µm, spectral: ∼15 nm). To analyze this multi-modal data, we developed an improved ResNet50 late fusion network optimized with a consistency-enhanced loss function. Evaluated on ex vivo clinical samples, our framework achieved an average cross-validation accuracy of 94.83%, significantly outperforming single-modal baselines. This non-destructive approach bridges the gap between topographical assessment and broadband optical analysis, providing a powerful quantitative tool for future deep learning-assisted arthroscopic integration.