Development of actionable quality indicators and an action implementation toolbox for appropriate antibiotic use at intensive care units: A modified-RAND Delphi study

PLoS One. 2018 Nov 29;13(11):e0207991. doi: 10.1371/journal.pone.0207991. eCollection 2018.


Introduction: Extensive antibiotic use makes the intensive care unit (ICU) an important focus for antibiotic stewardship programs. The aim of this study was to develop a set of actionable quality indicators for appropriate antibiotic use at ICUs and an implementation toolbox, which can be used to assess and improve the appropriateness of antibiotic use in the treatment of adult patients at an ICU.

Methods: A four round modified-RAND Delphi procedure was used. Potential indicators were identified by a multidisciplinary panel of 15 Dutch experts, from international literature and guidelines. Using an online survey, the identified indicators were rated on three criteria: relevance, actionability and feasibility. Experts discussed and rated the indicators for the second time during a face-to-face consensus meeting. During a final consensus meeting the toolbox was developed, containing potential barriers and improvement strategies which were identified using a validated checklist by Flottorp et al., and if available also containing supporting material.

Results: The first round resulted in 24 potential indicators. After the final meeting a set of three process indicators, one structure indicator and one quantity metric remained: 1) perform at least two sets of blood cultures before start of empirical systemic therapy; 2) perform therapeutic drug monitoring in patients treated with vancomycin or aminoglycosides; 3) perform surveillance cultures if selective digestive or oropharyngeal decontamination is applied at the ICU; 4) biannual face-to-face meetings between ICU and microbiology staff in which local resistance rates are discussed; and 5) quantitative antibiotic use at the ICU expressed in days of therapy (DOT). The toolbox contains 24 unique barriers and 37 improvement strategies.

Conclusions: Our study identified a set of four actionable quality indicators and one quantity metric, together with an implementation toolbox, to improve appropriate antibiotic use at ICUs.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Anti-Bacterial Agents / administration & dosage
  • Anti-Bacterial Agents / therapeutic use
  • Antimicrobial Stewardship / methods*
  • Decision Support Techniques*
  • Delphi Technique
  • Humans
  • Intensive Care Units
  • Netherlands
  • Quality Indicators, Health Care
  • Surveys and Questionnaires


  • Anti-Bacterial Agents

Grants and funding

This work was supported by the Stichting Kwaliteitsgelden Medisch Specialisten (SKMS, Dutch Foundation for Quality Funding for Medical Specialists), grant number [29762944]. Grant recipient is professor Nicolette de Keizer.