An analysis of neuroscience and psychiatry papers published from 2009 and 2019 outlines opportunities for increasing discovery of sex differences

Abstract

Sex differences exist in many neurological and psychiatric diseases, but these have not always been addressed adequately in research. In order to address this, it is necessary to consider how sex is incorporated into the design (e.g. using a balanced design) and into the analyses (e.g. using sex as a covariate) in the published literature. We surveyed papers published in 2009 and 2019 across six journals in neuroscience and psychiatry. In this sample, we find a 30% increase in the percentage of papers reporting studies that included both sexes in 2019 compared with 2009. Despite this increase, in 2019 only 19% of papers in the sample reported using an optimal design for discovery of possible sex differences, and only 5% of the papers reported studies that analysed sex as a discovery variable. We conclude that progress to date has not been sufficient to address the importance of sex differences in research for discovery and therapeutic potential for neurological and psychiatric disease.

Fig. 1. Total number of papers sampled in 2009 and 2019 across six journals in neuroscience and psychiatry.

Reviews, viewpoints, brief communications and any other non-primary research articles were excluded. A total of 2456 studies did not match the inclusion criteria and were excluded. Only primary research articles containing human, rat, mice, fetal or cell lines were analysed further (n = 3193).

Fig. 2. Reported species model in each paper sampled across our survey.

a Rodents (mice (n = 1178) and rats (n = 681)) were the most common species by studies in the papers from the neuroscience journals analysed. b Human subjects (n = 550) were the most common species used by studies in the papers in the psychiatry journals analysed. Sample sizes (n) are the number of papers that reported studies that used the model systems and will total to greater than 3193 as some papers reported studies used two or more model systems. c Breakdown of type of cell line used in studies reported in the papers. The largest proportion of papers reported studies that used primary cell lines. Of the papers that reported studies that used cell lines, the majority reported use of primary cell culture (n = 291). The other types of cell line reported in the papers were stem cell derivatives (n = 54), immortalised with other cell types (n = 105), embryonic (n = 115), and immortalised only (n = 34). Sample sizes are the number of papers and will total greater than 397 as some papers reported studies used two or more cell lines. We relied on the paper to distinguish whether cell lines were conducted in males or females, regardless of the cell line used, and if information was not available on the sex of the cells, they were categorised as sex not reported. Source data are provided as a Source Data file.

Fig. 3. The percentage of papers in the sampled in 2009 and 2019 in neuroscience and psychiatry that reported use of both sexes, and the breakdown of how papers were using both sexes.

Plotted are the percentage of proportional papers within each journal and year, n = the number of research papers within each category. a Percentage of papers reporting studies using both sexes in any aspect of the paper, regardless of consistency or balanced ratios. The percentage of papers reporting studies including males and females increased significantly for neuroscience (Newman–Keul’s post hoc p = 0.003, two-tailed) but not psychiatry papers (interaction effect of year by discipline: F(1,8) = 8.844, p = 0.017, Newman–Keul’s post hoc p = 0.319, two-tailed). Number of papers: neuroscience 2009 n = 316, 2019 n = 646; psychiatry 2009 n = 288, 2019 n = 249. b Percentage of papers reporting studies using both sexes consistently throughout the paper with balanced ratios of the sexes. Number of papers: neuroscience 2009 n = 130, 2019 n = 158; psychiatry 2009 n = 103, 2019 n = 87. c Percentage of papers not reporting sex (sex omission) was decreased in the neuroscience discipline; discipline by year interaction (F(1,8) = 45.21, p = 0.0001, Newman–Keul’s post hoc p = 0.0002, two-tailed). Number of papers: neuroscience 2009 n = 617, 2019 n = 25; psychiatry 2009 n = 34, 2019 n = 14. Means +/− standard error of the mean. d Unbalanced design (i.e., more than 60% of the subjects were one sex) was 34.52% of all papers including both sexes Number of papers: neuroscience 2009 n = 105, 2019 n = 154; psychiatry 2009 n = 142, 2019 n = 98. e Papers reporting studies using both sexes but not disclosing sample sizes, increased in the neuroscience sample (a priori p = 0.015; interaction (F(1, 8) = 3.73, p = 0.089) but not in the psychiatry sample (p = 0.717). Number of papers: neuroscience 2009 n = 69, 2019 n = 304; psychiatry 2009 n = 27, 2019 n = 38. f Inconsistent use of sex within the studies reported in a paper (i.e., using a balanced ratio in one study within the paper, and an unbalanced ratio or one sex in the other studies within the paper) accounted for 15.11% of papers that we had identified as reporting studies using males and females Number of papers: neuroscience 2009 n = 55, 2019 n = 102; psychiatry 2009 n = 17, 2019 n = 34. Means ∓ standard error of the mean. Source data are provided as a Source Data file.

Fig. 4. Percentage of papers sampled in 2009 and 2019 in neuroscience and psychiatry, which used only a single sex in the paper or which used both sexes, and the breakdown of types of analyses used.

Plotted are the percentage of proportional papers within each journal and year, n = the number of research papers within each category. a Percentage of papers reporting studies using only one sex across years and disciplines. Male-only papers (26.96%) were 8.2 times higher than female-only (3.29%) papers (main effect of sex: F(1,8) = 324.39, p < 0.0001; number of papers: male-only: neuroscience 2009 n = 322, 2019 n = 229; psychiatry 2009 n = 184, 2019 n = 123; female-only neuroscience 2009 n = 55, 2019 n = 35; psychiatry 2009 n = 23, 2019 n = 10). b Of the studies using an unbalanced ratio of sex, there were more studies with greater proportion of males compared to females (number of papers: Male-skew: neuroscience 2009 n = 56, 2019 n = 84; psychiatry 2009 n = 97, 2019 n = 60; female-skew neuroscience 2009 n = 56, 2019 n = 70; psychiatry 2009 n = 51, 2019 n = 39). As the percentage is proportionally based on the number of publications that year per journal the number of papers will vary differently that the proportional representation. c Breakdown of the type of analyses used by papers that used both sexes. Categories of sex analysis include: main effect of sex, sexes analysed separately, sex analysed as a discovery variable, statistics not given (i.e., authors of the paper state some analysis was done but did not provide any statistics) mixed (i.e., any combination of analyses, which may or may not be consistent throughout the study), and sex as a covariate. Number of papers: main effect: neuroscience 2009 n = 4, 2019 n = 12; psychiatry 2009 n = 8, 2019 n = 5; sex analysed separately: neuroscience 2009 n = 9, 2019 n = 22; psychiatry 2009 n = 12, 2019 n = 19; analysed as discovery: neuroscience 2009 n = 9, 2019 n = 27; psychiatry 2009 n = 24, 2019 n = 34; stats not given: neuroscience 2009 n = 11, 2019 n = 32; psychiatry 2009 n = 8, 2019 n = 7; mixed: neuroscience 2009 n = 5, 2019 n = 19; psychiatry 2009 n = 39, 2019 n = 38; covariate: neuroscience 2009 n = 6, 2019 n = 33; psychiatry 2009 n = 54, 2019 n = 75. d Majority of papers using both sexes did not analyse by sex, but this decreased slightly over 10 years. Number of papers: neuroscience 2009 n = 270, 2019 n = 498; psychiatry 2009 n = 143, 2019 n = 76. e Any analysis of sex in studies using both sexes. Psychiatry papers were more likely to perform any type of sex analysis than neuroscience papers. Neuroscience 2009 n = 46, 2019 n = 148; psychiatry 2009 n = 145, 2019 n = 173. Means ∓ standard error of the mean. Source data are provided as a Source Data file.

Fig. 5. Institutional author affiliation of the authors of each paper sampled in neuroscience and psychiatry in 2009 and 2019.

If a different country was noted among the author affiliations within a paper, this was considered as a combination of countries. Plotted are the percentage of proportional papers within each journal and year, n = the number of research papers within each category. a–f Country or combination of countries of author affiliations and the respective percentage of papers reporting studies that analysed using sex as a discovery variable across years compared to the country total. Papers from research groups based in the USA, Canada, EU and a combination of countries had an increased percentage of studies that analysed by sex as a discovery variable but none of these were significant. Means ∓ standard error of the mean a E.U. is the European Union (number of papers: neuroscience 2009 n = 2, 2019 n = 1; psychiatry 2009 n = 1, 2019 n = 1. b U.K. is the United Kingdom (number of papers: neuroscience 2009 n = 0, 2019 n = 0; psychiatry 2009 n = 2, 2019 n = 0. c Canada (number of papers: neuroscience 2009 n = 0, 2019 n = 1; psychiatry 2009 n = 1, 2019 n = 1. d U.S.A. is the United States of America (number of papers: neuroscience 2009 n = 4, 2019 n = 15; psychiatry 2009 n = 10, 2019 n = 18. e Combination of countries: (number of papers: neuroscience 2009 n = 3, 2019 n = 10; psychiatry 2009 n = 8, 2019 n = 14. f Asia: (number of papers: neuroscience 2009 n = 0, 2019 n = 0; psychiatry 2009 n = 2, 2019 n = 0. g Country/region of author affiliation of all papers sampled and h breakdown of papers sampled that had studies using optimal analysis for discovery of sex differences by country or region of author affiliation. i–j Breakdown of papers reporting studies, which analysed by sex by country or region of author affiliation in 2009 (i) and 2019 (j) Means ∓ standard error of the mean. Source data are provided as a Source Data file.

Fig. 6. An infographic depicting the change in percentages of total papers sampled reporting studies in 2009 and 2019 that used both sexes, a single sex, omitted sex, papers reporting studies that used an optimal design or analyses for the discovery of possible sex differences irrespective of discipline.

Optimal design refers to relatively based sample size and use of males and females consistently across the experiments whereas optimal analyses refers to the use of sex as a discovery variable. Although the percentage of studies in the sample of neuroscience and psychiatry papers analysed has increased the use of optimal design and analyses has not changed as much and remain at low levels. There are nine times the percentage of male-only compared to female-only studies.