Simulation of unequal-variance binormal multireader ROC decision data: an extension of the Roe and Metz simulation model

Abstract

Rationale and objectives: Roe and Metz (RM) proposed a model for simulating multireader multicase (MRMC) data collected from a factorial study design in which readers read the same cases in all modalities. However, a major weakness of the RM model is that it generates data according to an equal-variance binormal model for each reader. This article extends the RM model by allowing the diseased and nondiseased decision-variable distributions to have unequal variances for each reader.

Materials and methods: I show how to modify the RM model so that it generates data according to an unequal-variance binormal model for each reader. In doing so, I preserve other important characteristics of the original simulation input values. The mean-to-sigma ratio, which describes the relationship between the means and variances of the diseased and nondiseased decision-variable distributions, is constrained to have a value that is representative of many data sets. This last point is illustrated with an example comparing the performances of spin echo and cine magnetic resonance imaging for detecting thoracic aortic dissection.

Results: A simulation study is performed to assess the performance of the MRMC methods proposed by Dorfman, Berbaum, and Metz and by Obuchowski and Rockette using the proposed unequal variance extension of the RM model. The methods show either excellent or acceptable performance when there are at least five readers and at least 25 normal and 25 abnormal cases.

Conclusions: The proposed extension of the RM simulation model generates data that are more similar to data collected from radiological studies.