A human-in-the-loop deep learning paradigm for synergic visual evaluation in children

Abstract

Visual development during early childhood is a vital process. Examining the visual acuity of children is essential for early detection of visual abnormalities, but performing visual examination in children is challenging. Here, we developed a human-in-the-loop deep learning (DL) paradigm that combines traditional vision examination and DL with integration of software and hardware, thus facilitating the execution of vision examinations, offsetting the shortcomings of human doctors, and improving the abilities of both DL and doctors to evaluate the vision of children. Because this paradigm contains two rounds (a human round and DL round), doctors can learn from DL and the two can mutually supervise each other such that the precision of the DL system in evaluating the visual acuity of children is improved. Based on DL-based object localization and image identification, the experiences of doctors and the videos captured in the first round, the DL system in the second round can simulate doctors in evaluating the visual acuity of children with a final accuracy of 75.54%. For comparison, we also assessed an automatic deep learning method that did not consider the experiences of doctors, but its performance was not satisfactory. This entire paradigm can evaluate the visual acuity of children more accurately than humans alone. Furthermore, the paradigm facilitates automatic evaluation of the vision of children with a wearable device.

Keywords: Deep learning; Evaluating the visual acuity of children; Human-in-the-loop; Image identification; Integration of software and hardware; Object localization.