Evaluating impact of clinical guidelines using a realist evaluation framework

J Eval Clin Pract. 2015 Dec;21(6):1114-20. doi: 10.1111/jep.12482. Epub 2015 Dec 7.


Rationale, aims and objectives: The Remote Primary Health Care Manuals (RPHCM) project team manages the development and publication of clinical protocols and procedures for primary care clinicians practicing in remote Australia. The Central Australian Rural Practitioners Association Standard Treatment Manual, the flagship manual of the RPHCM suite, has been evaluated for accessibility and acceptability in remote clinics three times in its 20-year history. These evaluations did not consider a theory-based framework or a programme theory, resulting in some limitations with the evaluation findings. With the RPHCM having an aim of enabling evidence-based practice in remote clinics and anecdotally reported to do so, testing this empirically for the full suite is vital for both stakeholders and future editions of the RPHCM.

Methods: The project team utilized a realist evaluation framework to assess how, why and for what the RPHCM were being used by remote practitioners. A theory regarding the circumstances in which the manuals have and have not enabled evidence-based practice in the remote clinical context was tested. The project assessed this theory for all the manuals in the RPHCM suite, across government and aboriginal community-controlled clinics, in three regions of Australia.

Conclusion: Implementing a realist evaluation framework to generate robust findings in this context has required innovation in the evaluation design and adaptation by researchers. This article captures the RPHCM team's experience in designing this evaluation.

Keywords: clinical guidelines; evidence-based medicine; realist evaluation; remote healthcare.

MeSH terms

  • Australia
  • Clinical Protocols
  • Guideline Adherence / standards
  • Guideline Adherence / statistics & numerical data*
  • Humans
  • Manuals as Topic*
  • Practice Guidelines as Topic / standards*
  • Primary Health Care / standards
  • Primary Health Care / statistics & numerical data*
  • Program Evaluation
  • Rural Health Services / standards
  • Rural Health Services / statistics & numerical data*