Reporting methodological search filter performance comparisons: a literature review

Health Info Libr J. 2014 Sep;31(3):176-94. doi: 10.1111/hir.12070. Epub 2014 Aug 1.


Background: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.

Methods: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.

Results: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.

Conclusions: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

Keywords: bibliographic databases; information storage and retrieval; methodological filters; precision; recall; review, literature.

Publication types

  • Research Support, Non-U.S. Gov't
  • Review

MeSH terms

  • Data Accuracy*
  • Database Management Systems / standards*
  • Evaluation Studies as Topic*
  • Humans
  • Search Engine / methods
  • Search Engine / standards*