Limits of observational data in determining outcomes from cancer therapy

Cancer. 2008 Jun;112(11):2456-66. doi: 10.1002/cncr.23452.


Background: Observational data are used increasingly to assess the effectiveness of therapies. However, selection biases are likely to have an impact on results and threaten the validity of these studies.

Methods: The primary objective of the current study was to explore the effect of selection biases in observational studies of treatment effectiveness in cancer care. Patients were identified from the Surveillance, Epidemiology, and End Results-Medicare linked database. The following groups of patients were included: 5245 men treated with and without androgen deprivation for locally advanced prostate cancer, 43,847 men with active treatment versus observation for low- and intermediate-risk prostate cancer, and 4860 patients with lymph node-positive colon cancer who were treated with and without fluorouracil chemotherapy. Patients were compared by therapy for the outcomes of cancer-specific mortality, other-cause mortality, and overall mortality.

Results: In all comparisons, the observational data produced improbable results. For example, when evaluating outcomes of men who were treated with and without androgen deprivation for locally advanced prostate cancer, men who underwent androgen deprivation had higher prostate cancer mortality (hazard ratio, 1.5; 95% confidence interval, 1.29-1.92) despite clinical trial evidence that this treatment improves cancer mortality. Controlling for comorbidity, extent of disease, and other characteristics by multivariate analyses or by propensity analyses had remarkably small impact on these improbable results.

Conclusions: The current results suggested that the results from observational studies of treatment outcomes should be viewed with caution.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Adenocarcinoma / drug therapy
  • Adenocarcinoma / radiotherapy
  • Adenocarcinoma / surgery
  • Adenocarcinoma / therapy*
  • Aged
  • Aged, 80 and over
  • Androgen Antagonists / therapeutic use
  • Chemotherapy, Adjuvant
  • Colonic Neoplasms / drug therapy*
  • Combined Modality Therapy
  • Humans
  • Lymph Nodes / pathology
  • Male
  • Middle Aged
  • Neoplasms, Hormone-Dependent / drug therapy
  • Neoplasms, Hormone-Dependent / radiotherapy
  • Neoplasms, Hormone-Dependent / surgery
  • Neoplasms, Hormone-Dependent / therapy*
  • Prostatectomy
  • Prostatic Neoplasms / drug therapy
  • Prostatic Neoplasms / radiotherapy
  • Prostatic Neoplasms / surgery
  • Prostatic Neoplasms / therapy*
  • SEER Program
  • Survival Rate
  • Treatment Outcome


  • Androgen Antagonists