A quality improvement evaluation case study: impact on public health outcomes and agency culture

Abstract

Background: Quality improvement (QI) is increasingly recognized as an important strategy to improve healthcare services and health outcomes, including reducing health disparities. However, there is a paucity of evidence documenting the value of QI to public health agencies and services.

Purpose: The purpose of this project was to support and assess the impact on the outcomes and organizational culture of a QI project to increase immunization rates among children aged 2 years (4:3:1:3:3:1 series) within a large public health agency with a major pediatric health mission.

Methods: The intervention consisted of the use of a model-for-improvement approach to QI for the delivery of immunization services in public health clinics, utilizing plan-do-study-act cycles and multiple QI techniques. A mixed-method (qualitative and quantitative) model of evaluation was used to collect and analyze data from June 2009 to July 2011 to support both summative and developmental evaluation. The Florida Immunization Registry (Florida SHOTS [State Health Online Tracking System]) was used to monitor and analyze changes in immunization rates from January 2009 to July 2012. An interrupted time-series application of covariance was used to assess significance of the change in immunization rates, and paired comparison using parametric and nonparametric statistics were used to assess significance of pre- and post-QI culture items.

Results: Up-to-date immunization rates increased from 75% to more than 90% for individual primary care clinics and the overall county health department. In addition, QI stakeholder scores on ten key items related to organizational culture increased from pre- to post-QI intervention. Statistical analysis confirmed significance of the changes.

Conclusions: The application of QI combined with a summative and developmental evaluation supported refinement of the QI approach and documented the potential for QI to improve population health outcomes and improve public health agency culture.