Meta-research: Evaluation and Improvement of Research Methods and Practices

Abstract

As the scientific enterprise has grown in size and diversity, we need empirical evidence on the research process to test and apply interventions that make it more efficient and its results more reliable. Meta-research is an evolving scientific discipline that aims to evaluate and improve research practices. It includes thematic areas of methods, reporting, reproducibility, evaluation, and incentives (how to do, report, verify, correct, and reward science). Much work is already done in this growing field, but efforts to-date are fragmented. We provide a map of ongoing efforts and discuss plans for connecting the multiple meta-research efforts across science worldwide.

Fig 1. Number of meta-research–related publications registered by the Scopus database between January 1 and May 16 2015, by country of corresponding author and by affiliation of any coauthor.

Countries are attributed based on corresponding or first author address (legend, from light yellow to red, respectively, to 1–5, 5–10, 10–20, 20–50, 50–230 publications). Blue dots indicate the 100 institutions most frequently listed amongst coauthors’ addresses. Dot size is proportional to number of papers (range: 2–37). Papers were selected for inclusion from an initial list of 1,422 papers retrieved from the Scopus database using a combination of search terms aimed at capturing the core areas described in Table 1. Of the 851 records selected for inclusion, country or affiliation data could not be retrieved for 102 Scopus records, which therefore are not included in the map. Search terms, literature lists, and further details are available at metrics.stanford.edu. The map and plots therein were generated anew, using the packages ggmap and ggplot2 implemented in the open source statistical software R. Image Credit: Daniele Fanelli