Beyond Performance Metrics: Automatic Deep Learning Retinal OCT Analysis Reproduces Clinical Trial Outcome

Abstract

Purpose: To validate the efficacy of a fully automatic, deep learning-based segmentation algorithm beyond conventional performance metrics by measuring the primary outcome of a clinical trial for macular telangiectasia type 2 (MacTel2).

Design: Evaluation of diagnostic test or technology.

Participants: A total of 92 eyes from 62 participants with MacTel2 from a phase 2 clinical trial (NCT01949324) randomized to 1 of 2 treatment groups METHODS: The ellipsoid zone (EZ) defect areas were measured on spectral domain OCT images of each eye at 2 time points (baseline and month 24) by a fully automatic, deep learning-based segmentation algorithm. The change in EZ defect area from baseline to month 24 was calculated and analyzed according to the clinical trial protocol.

Main outcome measure: Difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups.

Results: The difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups measured by the fully automatic segmentation algorithm was 0.072±0.035 mm² (P = 0.021). This was comparable to the outcome of the clinical trial using semiautomatic measurements by expert readers, 0.065±0.033 mm² (P = 0.025).

Conclusions: The fully automatic segmentation algorithm was as accurate as semiautomatic expert segmentation to assess EZ defect areas and was able to reliably reproduce the statistically significant primary outcome measure of the clinical trial. This approach, to validate the performance of an automatic segmentation algorithm on the primary clinical trial end point, provides a robust gauge of its clinical applicability.