Keeping the noise down: common random numbers for disease simulation modeling

Abstract

Disease simulation models are used to conduct decision analyses of the comparative benefits and risks associated with preventive and treatment strategies. To address increasing model complexity and computational intensity, modelers use variance reduction techniques to reduce stochastic noise and improve computational efficiency. One technique, common random numbers, further allows modelers to conduct counterfactual-like analyses with direct computation of statistics at the individual level. This technique uses synchronized random numbers across model runs to induce correlation in model output thereby making differences easier to distinguish as well as simulating identical individuals across model runs. We provide a tutorial introduction and demonstrate the application of common random numbers in an individual-level simulation model of the epidemiology of breast cancer.

Figure 1. Stylized example of the synchronization of a random event under a frequent screening scenario compared with infrequent screening

For frequent screening, the “ith” random number determines the treatment effectiveness event with a result of “not effective” for a breast cancer diagnosed in an ‘early’ stage (solid arrow, first column). With common random numbers, the ‘ith’ random number is also used determine the treatment effectiveness event in the infrequent screening scenario regardless of the stage at detection. If it was detected in Stage III, treatment tendency is preserved as her treatment is again determined to not be effective (solid arrow, second column). Without common random numbers, the event may be determined by a different random number, the “jth” number, regardless of the stage at detection. If it was detected in Stage III, treatment tendency is not preserved as the result is “effective” (dashed arrow, second column).

Figure 2. Effects of Common Random Numbers and Sample Size on Model Outputs

Each panel shows breast cancer incidence rates over time under two model scenarios, a baseline run (solid line) and an alternative run (dashed lines). Panels differ by the use of common random numbers and input sample size. For a standard model sample size, without common random numbers (Panel A) the two scenarios overlap while with common random number (Panel B) they are distinguishable. Increased sample size (Panels C and D) distinguishes the two scenarios regardless of common random numbers.

Figure 3. Change in Time to Death Attributable to the Mammogram of Diagnosis

In the original model run, a woman’s time to death is recorded starting from the time of breast cancer is detected by a particular mammogram. In the counterfactual model run, a woman’s time to death is recorded starting from the same time that her breast cancer would have been detected by a particular mammogram in the original model run. However in this run, it is assumed that she instead did not attend the particular mammogram that led to breast cancer detection. The difference in times to death for this simulated woman under the original and counterfactual scenario is her change in time to death attributable to the mammogram of diagnosis.

Figure 4. Distribution of Change in Time to Death Attributable to the Mammogram of Diagnosis

The histogram of the distribution of change in time to death across all simulated women shows that 92% of 60 year old women would not experience a change in time to death if they instead skipped the mammogram that would have detected breast cancer at age 60. The average across this distribution is the change in life expectancy.