Comparing simulation and threshold approaches when analysing data with probabilities of categories

Abstract

Objective: Constructing categories based on probabilities is not unusual in defining the outcome or the exposure. We compare the threshold approach and the simulation approach in making inferences.

Method: We used a simple structured example as well as published data to illustrate the difference between the simulation and the threshold approaches.

Results: We demonstrated that simulation results were different from the threshold approach in estimating the effect of a high-deductible health plan. For repeat visits, we estimated a statistically significant ratio of incident rate ratio (IRR) 0.78 (95% CI: 0.64, 0.93) for non-preventable emergency department visits using the simulation approach while the high-severity category showed no statistical significance with the ratio of IRR 0.98 (95% CI: 0.64, 1.49) using a threshold of 75%. We also demonstrated that none of the threshold values could achieve the results of the simulation approach.

Conclusions: The simulation approach is preferred over the threshold one when analysing data with probability-based outcome, exposure and/or other covariates if the size of the data permits.