Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 22 (11), 1359-66

False-positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

Affiliations

False-positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

Joseph P Simmons et al. Psychol Sci.

Abstract

In this article, we accomplish two things. First, we show that despite empirical psychologists' nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.

Comment in

  • False-Positive Citations.
    Simmons JP, Nelson LD, Simonsohn U. Simmons JP, et al. Perspect Psychol Sci. 2018 Mar;13(2):255-259. doi: 10.1177/1745691617698146. Perspect Psychol Sci. 2018. PMID: 29592640
  • Rein in the four horsemen of irreproducibility.
    Bishop D. Bishop D. Nature. 2019 Apr;568(7753):435. doi: 10.1038/d41586-019-01307-2. Nature. 2019. PMID: 31019328 No abstract available.
  • A journal club to fix science.
    Orben A. Orben A. Nature. 2019 Sep;573(7775):465. doi: 10.1038/d41586-019-02842-8. Nature. 2019. PMID: 31551562 No abstract available.

Similar articles

See all similar articles

Cited by 595 articles

See all "Cited by" articles

LinkOut - more resources

Feedback