A novel benchmark model for intelligent annotation of spectral-domain optical coherence tomography scans using the example of cyst annotation

Abstract

Background and objectives: The lack of benchmark data in computational ophthalmology contributes to the challenging task of applying disease assessment and evaluate performance of machine learning based methods on retinal spectral domain optical coherence tomography (SD-OCT) scans. Presented here is a general framework for constructing a benchmark dataset for retinal image processing tasks such as cyst, vessel, and subretinal fluid segmentation and as a result, a benchmark dataset for cyst segmentation has been developed.

Method: First, a dataset captured by different SD-OCT vendors with different numbers of scans and pathology qualities are selected. Then a robust and intelligent method is used to evaluate performance of readers, partitioning the dataset into subsets. Subsets are then assigned to complementary readers for annotation with respect to a novel confidence based annotation protocol. Finally, reader annotations are combined based on their performance to generate final annotations.

Result: The generated benchmark dataset for cyst segmentation comprises 26 SD-OCT scans with differing cyst qualities, collected from 4 different SD-OCT vendors to cover a wide variety of data. The dataset is partitioned into three subsets which are annotated by complementary readers based on a confidence based annotation protocol. Experimental results show annotations of complementary readers are combined efficiently with respect to their performance, generating accurate annotations.

Conclusion: Our results facilitate the process of generating benchmark datasets. Moreover the generated benchmark data set for cyst segmentation can be used reliably to train and test machine learning based methods.

Keywords: Benchmark dataset; Cyst segmentation; SD-OCT.