Why are some studies of cardiovascular markers unreliable? The role of measurement variability and what an aspiring clinician scientist can do before it is too late

Abstract

Cardiology research suffers from the scourge of unreliable results, despite honest conduct. Investigators' prior belief, compromised blinding, and scope for measurement variability are a fatally synergistic combination. Can we stop these threats ruining the results? First, clinical researchers must realize that healthy clinical practice (including intelligently integrating all available information) may be catastrophic to research. Second, experienced clinicians know that variability may necessitate remeasurement to obtain a clinically correct result but must learn that doing so in research can cause surprisingly severe distortions of correlations or differences between groups. For example, a "best-of-four" approach in comparing two 50-patient groups that are in reality identical, with a variable whose intraclass correlation is 0.8, easily generates highly significant P values. Clinicians may be habituated to poorly reproducible clinical measurements and falsely reassured by their effectiveness for group mean effects in blinded randomized controlled trials. We need a more critical approach to clinical tests if we care about evaluating individual patients reliably or want our research to be reliable. Simple steps shown here, addressed during study design, will increase the reliability of research-if considered by researchers or the juniors whom they nurture.