Scientific citations favor positive results: a systematic review and meta-analysis

Abstract

Objectives: Citation bias concerns the selective citation of scientific articles based on their results. We brought together all available evidence on citation bias across scientific disciplines and quantified its impact.

Study design and setting: An extensive search strategy was applied to the Web of Science Core Collection and Medline, yielding 52 studies in total. We classified these studies on scientific discipline, selection method, and other variables. We also performed random-effects meta-analyses to pool the effect of positive vs. negative results on subsequent citations. Finally, we checked for other determinants of citation as reported in the citation bias literature.

Results: Evidence for the occurrence of citation bias was most prominent in the biomedical sciences and least in the natural sciences. Articles with statistically significant results were cited 1.6 (95% confidence interval [CI] 1.3-1.8) times more often than articles with nonsignificant results. Articles in which the authors explicitly conclude to have found support for their hypothesis were cited 2.7 (CI 2.0-3.7) times as often. Article results and journal impact factor were associated with citation more often than any other reported determinant.

Conclusion: Similar to what we already know on publication bias, also citation bias can lead to an overrepresentation of positive results and unfounded beliefs.

Keywords: Citation bias; Meta-analysis; Outcome bias; Questionable research practices; Research integrity; Systematic review.