Statistically Non-significant Papers in Environmental Health Studies included more Outcome Variables

Abstract

Objective: The number of analyzed outcome variables is important in the statistical analysis and interpretation of research findings. This study investigated published papers in the field of environmental health studies. We aimed to examine whether differences in the number of reported outcome variables exist between papers with non-significant findings compared to those with significant findings. Articles on the maternal exposure to mercury and child development were used as examples.

Methods: Articles published between 1995 and 2013 focusing on the relationships between maternal exposure to mercury and child development were collected from Medline and Scopus.

Results: Of 87 extracted papers, 73 used statistical significance testing and 38 (43.7%) of these reported 'non-significant' (P>0.05) findings. The median number of child development outcome variables in papers reporting 'significant' (n=35) and 'non-significant' (n=38) results was 4 versus 7, respectively (Mann-Whitney test P-value=0.014). An elevated number of outcome variables was especially found in papers reporting non-significant associations between maternal mercury and outcomes when mercury was the only analyzed exposure variable.

Conclusion: Authors often report analyzed health outcome variables based on their P-values rather than on stated primary research questions. Such a practice probably skews the research evidence.

Keywords: Child development; Literature review; Mercury exposure; Reporting bias.