Redefine statistical significance

doi:10.1038/s41562-017-0189-z

. 2018 Jan;2(1):6-10.

doi: 10.1038/s41562-017-0189-z.

Redefine statistical significance

Daniel J Benjamin¹, James O Berger², Magnus Johannesson³, Brian A Nosek^{4

5}, E-J Wagenmakers⁶, Richard Berk^{7

8}, Kenneth A Bollen⁹, Björn Brembs¹⁰, Lawrence Brown⁸, Colin Camerer¹¹, David Cesarini^{12

13}, Christopher D Chambers¹⁴, Merlise Clyde², Thomas D Cook^{15

16}, Paul De Boeck¹⁷, Zoltan Dienes¹⁸, Anna Dreber¹⁹, Kenny Easwaran²⁰, Charles Efferson²¹, Ernst Fehr²², Fiona Fidler²³, Andy P Field¹⁸, Malcolm Forster²⁴, Edward I George⁸, Richard Gonzalez²⁵, Steven Goodman²⁶, Edwin Green²⁷, Donald P Green²⁸, Anthony G Greenwald²⁹, Jarrod D Hadfield³⁰, Larry V Hedges³¹, Leonhard Held³², Teck Hua Ho³³, Herbert Hoijtink³⁴, Daniel J Hruschka³⁵, Kosuke Imai³⁶, Guido Imbens³⁷, John P A Ioannidis³⁸, Minjeong Jeon³⁹, James Holland Jones^{40

41}, Michael Kirchler⁴², David Laibson⁴³, John List⁴⁴, Roderick Little⁴⁵, Arthur Lupia⁴⁶, Edouard Machery⁴⁷, Scott E Maxwell⁴⁸, Michael McCarthy⁴⁹, Don A Moore⁵⁰, Stephen L Morgan⁵¹, Marcus Munafó^{52

53}, Shinichi Nakagawa⁵⁴, Brendan Nyhan⁵⁵, Timothy H Parker⁵⁶, Luis Pericchi⁵⁷, Marco Perugini⁵⁸, Jeff Rouder⁵⁹, Judith Rousseau⁶⁰, Victoria Savalei⁶¹, Felix D Schönbrodt⁶², Thomas Sellke⁶³, Betsy Sinclair⁶⁴, Dustin Tingley⁶⁵, Trisha Van Zandt⁶⁶, Simine Vazire⁶⁷, Duncan J Watts⁶⁸, Christopher Winship⁶⁹, Robert L Wolpert², Yu Xie⁷⁰, Cristobal Young⁷¹, Jonathan Zinman⁷², Valen E Johnson⁷³

Affiliations

¹ Center for Economic and Social Research and Department of Economics, University of Southern California, Los Angeles, CA, 90089-3332, USA. daniel.benjamin@gmail.com.
² Department of Statistical Science, Duke University, Durham, NC, 27708-0251, USA.
³ Department of Economics, Stockholm School of Economics, Stockholm, SE-113 83, Sweden. magnus.johannesson@hhs.se.
⁴ University of Virginia, Charlottesville, VA, 22908, USA.
⁵ Center for Open Science, Charlottesville, VA, 22903, USA.
⁶ Department of Psychology, University of Amsterdam, Amsterdam, 1018 VZ, The Netherlands.
⁷ School of Arts and Sciences and Department of Criminology, University of Pennsylvania, Philadelphia, PA, 19104-6286, USA.
⁸ Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA, 19104, USA.
⁹ Department of Psychology and Neuroscience, Department of Sociology, University of North Carolina Chapel Hill, Chapel Hill, NC, 27599-3270, USA.
¹⁰ Institute of Zoology - Neurogenetics, Universität Regensburg, Universitätsstrasse 31, 93040, Regensburg, Germany.
¹¹ Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, 91125, USA.
¹² Department of Economics, New York University, New York, NY, 10012, USA.
¹³ The Research Institute of Industrial Economics (IFN), Stockholm, SE-102 15, Sweden.
¹⁴ Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff, CF24 4HQ, UK.
¹⁵ Northwestern University, Evanston, IL, 60208, USA.
¹⁶ Mathematica Policy Research, Washington, DC, 20002-4221, USA.
¹⁷ Department of Psychology, Quantitative Program, Ohio State University, Columbus, OH, 43210, USA.
¹⁸ School of Psychology, University of Sussex, Brighton, BN1 9QH, UK.
¹⁹ Department of Economics, Stockholm School of Economics, Stockholm, SE-113 83, Sweden.
²⁰ Department of Philosophy, Texas A&M University, College Station, TX, 77843-4237, USA.
²¹ Department of Psychology, Royal Holloway University of London, Egham Surrey, TW20 0EX, UK.
²² Department of Economics, University of Zurich, 8006, Zurich, Switzerland.
²³ School of BioSciences and School of Historical & Philosophical Studies, University of Melbourne, Parkville, VIC, 3010, Australia.
²⁴ Department of Philosophy, University of Wisconsin - Madison, Madison, WI, 53706, USA.
²⁵ Department of Psychology, University of Michigan, Ann Arbor, MI, 48109-1043, USA.
²⁶ Stanford University, General Medical Disciplines, Stanford, CA, 94305, USA.
²⁷ Department of Ecology, Evolution and Natural Resources SEBS, Rutgers University, New Brunswick, NJ, 08901-8551, USA.
²⁸ Department of Political Science, Columbia University in the City of New York, New York, NY, 10027, USA.
²⁹ Department of Psychology, University of Washington, Seattle, WA, 98195-1525, USA.
³⁰ Institute of Evolutionary Biology School of Biological Sciences, The University of Edinburgh, Edinburgh, EH9 3JT, UK.
³¹ Weinberg College of Arts & Sciences Department of Statistics, Northwestern University, Evanston, IL, 60208, USA.
³² Epidemiology, Biostatistics and Prevention Institute (EBPI), University of Zurich, 8001, Zurich, Switzerland.
³³ National University of Singapore, Singapore, 119077, Singapore.
³⁴ Department of Methods and Statistics, Universiteit Utrecht, Utrecht, 3584 CH, The Netherlands.
³⁵ School of Human Evolution and Social Change, Arizona State University, Tempe, AZ, 85287-2402, USA.
³⁶ Department of Politics and Center for Statistics and Machine Learning, Princeton University, Princeton, NJ, 08544, USA.
³⁷ Stanford University, Stanford, CA, 94305-5015, USA.
³⁸ Departments of Medicine, of Health Research and Policy, of Biomedical Data Science, and of Statistics and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, 94305, USA.
³⁹ Advanced Quantitative Methods, Social Research Methodology, Department of Education, Graduate School of Education & Information Studies, University of California, Los Angeles, CA, 90095-1521, USA.
⁴⁰ Department of Life Sciences, Imperial College London, Ascot, SL5 7PY, UK.
⁴¹ Department of Earth System Science, Stanford, CA, 94305-4216, USA.
⁴² Department of Banking and Finance, University of Innsbruck and University of Gothenburg, Innsbruck, A-6020, Austria.
⁴³ Department of Economics, Harvard University, Cambridge, MA, 02138, USA.
⁴⁴ Department of Economics, University of Chicago, Chicago, IL, 60637, USA.
⁴⁵ Department of Biostatistics, University of Michigan, Ann Arbor, MI, 48109-2029, USA.
⁴⁶ Department of Political Science, University of Michigan, Ann Arbor, MI, 48109-1045, USA.
⁴⁷ Department of History and Philosophy of Science, University of Pittsburgh, Pittsburgh, PA, 15260, USA.
⁴⁸ Department of Psychology, University of Notre Dame, Notre Dame, IN, 46556, USA.
⁴⁹ School of BioSciences, University of Melbourne, Parkville, VIC, 3010, Australia.
⁵⁰ Haas School of Business, University of California at Berkeley, Berkeley, CA, 94720-1900A, USA.
⁵¹ Johns Hopkins University, Baltimore, MD, 21218, USA.
⁵² MRC Integrative Epidemiology Unit, University of Bristol, Bristol, BS8 1TU, UK.
⁵³ UK Centre for Tobacco and Alcohol Studies, School of Experimental Psychology, University of Bristol, Bristol, BS8 1TU, UK.
⁵⁴ Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia.
⁵⁵ Department of Government, Dartmouth College, Hanover, NH, 03755, USA.
⁵⁶ Department of Biology, Whitman College, Walla Walla, WA, 99362, USA.
⁵⁷ Department of Mathematics, University of Puerto Rico, Rio Piedras Campus, San Juan, PR, 00936-8377, Puerto Rico.
⁵⁸ Department of Psychology, University of Milan-Bicocca, Milan, 20126, Italy.
⁵⁹ Department of Cognitive Sciences, University of California, Irvine, CA, 92617, USA.
⁶⁰ Université Paris Dauphine, 75016, Paris, France.
⁶¹ Department of Psychology, The University of British Columbia, Vancouver, V6T 1Z4, BC, Canada.
⁶² Department Psychology, Ludwig-Maximilians-University Munich, Leopoldstraβe 13, 80802, Munich, Germany.
⁶³ Department of Statistics, Purdue University, West Lafayette, IN, 47907-2067, USA.
⁶⁴ Department of Political Science, Washington University in St. Louis, St. Louis, MO, 63130-4899, USA.
⁶⁵ Government Department, Harvard University, Cambridge, MA, 02138, USA.
⁶⁶ Department of Psychology, Ohio State University, Columbus, OH, 43210, USA.
⁶⁷ Department of Psychology, University of California, Davis, CA, 95616, USA.
⁶⁸ Microsoft Research, 641 Avenue of the Americas, 7th Floor, New York, NY, 10011, USA.
⁶⁹ Department of Sociology, Harvard University, Cambridge, MA, 02138, USA.
⁷⁰ Department of Sociology, Princeton University, Princeton, NJ, 08544, USA.
⁷¹ Department of Sociology, Stanford University, Stanford, CA, 94305-2047, USA.
⁷² Department of Economics, Dartmouth College, Hanover, NH, 03755-3514, USA.
⁷³ Department of Statistics, Texas A&M University, College Station, TX, 77843, USA. vejohnson@exchange.tamu.edu.

PMID: 30980045
DOI: 10.1038/s41562-017-0189-z

Free article

Redefine statistical significance

Daniel J Benjamin et al. Nat Hum Behav. 2018 Jan.

Free article

. 2018 Jan;2(1):6-10.

doi: 10.1038/s41562-017-0189-z.

Authors

Affiliations

¹ Center for Economic and Social Research and Department of Economics, University of Southern California, Los Angeles, CA, 90089-3332, USA. daniel.benjamin@gmail.com.
² Department of Statistical Science, Duke University, Durham, NC, 27708-0251, USA.
³ Department of Economics, Stockholm School of Economics, Stockholm, SE-113 83, Sweden. magnus.johannesson@hhs.se.
⁴ University of Virginia, Charlottesville, VA, 22908, USA.
⁵ Center for Open Science, Charlottesville, VA, 22903, USA.
⁶ Department of Psychology, University of Amsterdam, Amsterdam, 1018 VZ, The Netherlands.
⁷ School of Arts and Sciences and Department of Criminology, University of Pennsylvania, Philadelphia, PA, 19104-6286, USA.
⁸ Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA, 19104, USA.
⁹ Department of Psychology and Neuroscience, Department of Sociology, University of North Carolina Chapel Hill, Chapel Hill, NC, 27599-3270, USA.
¹⁰ Institute of Zoology - Neurogenetics, Universität Regensburg, Universitätsstrasse 31, 93040, Regensburg, Germany.
¹¹ Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, 91125, USA.
¹² Department of Economics, New York University, New York, NY, 10012, USA.
¹³ The Research Institute of Industrial Economics (IFN), Stockholm, SE-102 15, Sweden.
¹⁴ Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff, CF24 4HQ, UK.
¹⁵ Northwestern University, Evanston, IL, 60208, USA.
¹⁶ Mathematica Policy Research, Washington, DC, 20002-4221, USA.
¹⁷ Department of Psychology, Quantitative Program, Ohio State University, Columbus, OH, 43210, USA.
¹⁸ School of Psychology, University of Sussex, Brighton, BN1 9QH, UK.
¹⁹ Department of Economics, Stockholm School of Economics, Stockholm, SE-113 83, Sweden.
²⁰ Department of Philosophy, Texas A&M University, College Station, TX, 77843-4237, USA.
²¹ Department of Psychology, Royal Holloway University of London, Egham Surrey, TW20 0EX, UK.
²² Department of Economics, University of Zurich, 8006, Zurich, Switzerland.
²³ School of BioSciences and School of Historical & Philosophical Studies, University of Melbourne, Parkville, VIC, 3010, Australia.
²⁴ Department of Philosophy, University of Wisconsin - Madison, Madison, WI, 53706, USA.
²⁵ Department of Psychology, University of Michigan, Ann Arbor, MI, 48109-1043, USA.
²⁶ Stanford University, General Medical Disciplines, Stanford, CA, 94305, USA.
²⁷ Department of Ecology, Evolution and Natural Resources SEBS, Rutgers University, New Brunswick, NJ, 08901-8551, USA.
²⁸ Department of Political Science, Columbia University in the City of New York, New York, NY, 10027, USA.
²⁹ Department of Psychology, University of Washington, Seattle, WA, 98195-1525, USA.
³⁰ Institute of Evolutionary Biology School of Biological Sciences, The University of Edinburgh, Edinburgh, EH9 3JT, UK.
³¹ Weinberg College of Arts & Sciences Department of Statistics, Northwestern University, Evanston, IL, 60208, USA.
³² Epidemiology, Biostatistics and Prevention Institute (EBPI), University of Zurich, 8001, Zurich, Switzerland.
³³ National University of Singapore, Singapore, 119077, Singapore.
³⁴ Department of Methods and Statistics, Universiteit Utrecht, Utrecht, 3584 CH, The Netherlands.
³⁵ School of Human Evolution and Social Change, Arizona State University, Tempe, AZ, 85287-2402, USA.
³⁶ Department of Politics and Center for Statistics and Machine Learning, Princeton University, Princeton, NJ, 08544, USA.
³⁷ Stanford University, Stanford, CA, 94305-5015, USA.
³⁸ Departments of Medicine, of Health Research and Policy, of Biomedical Data Science, and of Statistics and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, 94305, USA.
³⁹ Advanced Quantitative Methods, Social Research Methodology, Department of Education, Graduate School of Education & Information Studies, University of California, Los Angeles, CA, 90095-1521, USA.
⁴⁰ Department of Life Sciences, Imperial College London, Ascot, SL5 7PY, UK.
⁴¹ Department of Earth System Science, Stanford, CA, 94305-4216, USA.
⁴² Department of Banking and Finance, University of Innsbruck and University of Gothenburg, Innsbruck, A-6020, Austria.
⁴³ Department of Economics, Harvard University, Cambridge, MA, 02138, USA.
⁴⁴ Department of Economics, University of Chicago, Chicago, IL, 60637, USA.
⁴⁵ Department of Biostatistics, University of Michigan, Ann Arbor, MI, 48109-2029, USA.
⁴⁶ Department of Political Science, University of Michigan, Ann Arbor, MI, 48109-1045, USA.
⁴⁷ Department of History and Philosophy of Science, University of Pittsburgh, Pittsburgh, PA, 15260, USA.
⁴⁸ Department of Psychology, University of Notre Dame, Notre Dame, IN, 46556, USA.
⁴⁹ School of BioSciences, University of Melbourne, Parkville, VIC, 3010, Australia.
⁵⁰ Haas School of Business, University of California at Berkeley, Berkeley, CA, 94720-1900A, USA.
⁵¹ Johns Hopkins University, Baltimore, MD, 21218, USA.
⁵² MRC Integrative Epidemiology Unit, University of Bristol, Bristol, BS8 1TU, UK.
⁵³ UK Centre for Tobacco and Alcohol Studies, School of Experimental Psychology, University of Bristol, Bristol, BS8 1TU, UK.
⁵⁴ Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia.
⁵⁵ Department of Government, Dartmouth College, Hanover, NH, 03755, USA.
⁵⁶ Department of Biology, Whitman College, Walla Walla, WA, 99362, USA.
⁵⁷ Department of Mathematics, University of Puerto Rico, Rio Piedras Campus, San Juan, PR, 00936-8377, Puerto Rico.
⁵⁸ Department of Psychology, University of Milan-Bicocca, Milan, 20126, Italy.
⁵⁹ Department of Cognitive Sciences, University of California, Irvine, CA, 92617, USA.
⁶⁰ Université Paris Dauphine, 75016, Paris, France.
⁶¹ Department of Psychology, The University of British Columbia, Vancouver, V6T 1Z4, BC, Canada.
⁶² Department Psychology, Ludwig-Maximilians-University Munich, Leopoldstraβe 13, 80802, Munich, Germany.
⁶³ Department of Statistics, Purdue University, West Lafayette, IN, 47907-2067, USA.
⁶⁴ Department of Political Science, Washington University in St. Louis, St. Louis, MO, 63130-4899, USA.
⁶⁵ Government Department, Harvard University, Cambridge, MA, 02138, USA.
⁶⁶ Department of Psychology, Ohio State University, Columbus, OH, 43210, USA.
⁶⁷ Department of Psychology, University of California, Davis, CA, 95616, USA.
⁶⁸ Microsoft Research, 641 Avenue of the Americas, 7th Floor, New York, NY, 10011, USA.
⁶⁹ Department of Sociology, Harvard University, Cambridge, MA, 02138, USA.
⁷⁰ Department of Sociology, Princeton University, Princeton, NJ, 08544, USA.
⁷¹ Department of Sociology, Stanford University, Stanford, CA, 94305-2047, USA.
⁷² Department of Economics, Dartmouth College, Hanover, NH, 03755-3514, USA.
⁷³ Department of Statistics, Texas A&M University, College Station, TX, 77843, USA. vejohnson@exchange.tamu.edu.

PMID: 30980045
DOI: 10.1038/s41562-017-0189-z

No abstract available

PubMed Disclaimer

Comment in

Redefine or justify? Comments on the alpha debate.
de Ruiter J. de Ruiter J. Psychon Bull Rev. 2019 Apr;26(2):430-433. doi: 10.3758/s13423-018-1523-9. Psychon Bull Rev. 2019. PMID: 30264238
Remove, rather than redefine, statistical significance.
Amrhein V, Greenland S. Amrhein V, et al. Nat Hum Behav. 2018 Jan;2(1):4. doi: 10.1038/s41562-017-0224-0. Nat Hum Behav. 2018. PMID: 30980046 No abstract available.

Cited by

Building machine learning prediction models for well-being using predictors from the exposome and genome in a population cohort.
Pelt DHM, Habets PC, Vinkers CH, Ligthart L, van Beijsterveldt CEM, Pool R, Bartels M. Pelt DHM, et al. Nat Ment Health. 2024;2(10):1217-1230. doi: 10.1038/s44220-024-00294-2. Epub 2024 Aug 14. Nat Ment Health. 2024. PMID: 39464304 Free PMC article.
An overlooked phenomenon: complex interactions of potential error sources on the quality of bacterial de novo genome assemblies.
Rádai Z, Váradi A, Takács P, Nagy NA, Schmitt N, Prépost E, Kardos G, Laczkó L. Rádai Z, et al. BMC Genomics. 2024 Jan 9;25(1):45. doi: 10.1186/s12864-023-09910-4. BMC Genomics. 2024. PMID: 38195441 Free PMC article.
The Structure of Negative Social Ties in Rural Village Networks.
Isakov A, Fowler JH, Airoldi EM, Christakis NA. Isakov A, et al. Sociol Sci. 2019;6:197-218. doi: 10.15195/v6.a8. Epub 2019 Mar 6. Sociol Sci. 2019. PMID: 32704522 Free PMC article.
The impact of lowering the study design significance threshold to 0.005 on sample size in randomized cancer clinical trials.
Leung TH, Ho JC, Wang X, Lam WW, Pang HH. Leung TH, et al. J Clin Transl Sci. 2023 Dec 18;8(1):e9. doi: 10.1017/cts.2023.699. eCollection 2024. J Clin Transl Sci. 2023. PMID: 38384917 Free PMC article.
An exploratory study of associations between the ICD-11 personality disorder model and eating pathology.
Stricker J, Barthels F, Müller R, Pietrowsky R. Stricker J, et al. J Eat Disord. 2022 Aug 31;10(1):130. doi: 10.1186/s40337-022-00658-y. J Eat Disord. 2022. PMID: 36045403 Free PMC article.

See all "Cited by" articles

MeSH terms

Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
Other Literature Sources
- H1 Connect

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Redefine statistical significance

Affiliations

Redefine statistical significance

Authors

Affiliations

Comment in

Similar articles

Cited by

MeSH terms

LinkOut - more resources

Full Text Sources

Other Literature Sources

Comment in

Similar articles

Cited by

MeSH terms

Related information

LinkOut - more resources

Full Text Sources

Other Literature Sources