Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence

Abstract

Background: Various intentional and unintentional factors influence beliefs beyond what scientific evidence justifies. Two such factors are research lacking probative value (RLPV) and biased research reporting (BRR).

Objective: We investigated the prevalence of RLPV and BRR in research about the proposition that skipping breakfast causes weight gain, which is called the proposed effect of breakfast on obesity (PEBO) in this article.

Design: Studies related to the PEBO were synthesized by using a cumulative meta-analysis. Abstracts from these studies were also rated for the improper use of causal language and biased interpretations. In separate analyses, articles that cited an observational study about the PEBO were rated for the inappropriate use of causal language, and articles that cited a randomized controlled trial (RCT) about the PEBO were rated for misleadingly citing the RCT.

Results: The current body of scientific knowledge indicates that the PEBO is only presumed true. The observational literature on the PEBO has gratuitously established the association, but not the causal relation, between skipping breakfast and obesity (final cumulative meta-analysis P value <10(-42)), which is evidence of RLPV. Four examples of BRR are evident in the PEBO literature as follows: 1) biased interpretation of one's own results, 2) improper use of causal language in describing one's own results, 3) misleadingly citing others' results, and 4) improper use of causal language in citing others' work.

Conclusions: The belief in the PEBO exceeds the strength of scientific evidence. The scientific record is distorted by RLPV and BRR. RLPV is a suboptimal use of collective scientific resources.

FIGURE 1.

Map of countries where the proposed effect of breakfast on obesity has been studied. Dark gray denotes studies used in the meta-analysis or abstract analysis (n = 30). Black denotes studies used in the abstract analysis only (n = 6).

FIGURE 2.

Funnel plot of the meta-analysis. Dots represent each of 88 independent study group OR estimates. The vertical dashed line represents the meta-analyzed log OR. The solid line is an unweighted transformation of the Egger's linear regression test for funnel-plot asymmetry (9). The weighted intercept (0.5345) shows modest evidence of funnel-plot asymmetry (P = 0.0856), which may represent a publication bias, unaccounted heterogeneity in the meta-analysis, or asymmetry by chance.

FIGURE 3.

Cumulative meta-analysis of studies analyzed in the meta-analysis. Each data point represents the synthesis of all studies included year on year (n = 58 studies with 88 independent groups). Diamonds represent the ORs of skipping breakfast and being overweight or obese bounded by a 95% CI. The black horizontal line is set at an OR of 1. The final OR is 1.55 (95% CI: 1.46, 1.65). Squares represent P values of ORs for each year of the analysis. Two horizontal dotted lines represent the traditional P = 0.05 significance and P = 0.001, which was suggested to represent strong evidence of an association (38). The final P value is 10⁻⁴².

FIGURE 4.

Concordance between results and conclusions from 88 identified abstracts. Abstracts were rated on the basis of whether the results or conclusions stated that eating breakfast was associated with lower obesity (Positive), showed no relation between breakfast and obesity (No Relation), showed that eating breakfast was associated with higher obesity (Negative), showed mixed relations (Mixed), or did not mention an analysis between breakfast and obesity (None). Cell shading represents the percentage of conclusion ratings within a result-rating column, with black representing 100% and white representing 0%. Marginal counts are not shaded.

FIGURE 5.

Authors’ use of causative language in their own observational studies. The left pie chart shows that 48% (n = 42) of 88 abstracts made conclusions about breakfast and weight, which is broken down by the use of causal language in the right pie chart.

FIGURE 6.

Categorization of 91 articles that cited Schlundt et al (10). Articles that cited Schlundt et al (10) were categorized as shown in the table inset in the figure. All articles were included in the left pie chart, with only the relevant citations presented in the right pie chart (n = 42). ¹The one study that was explicitly misleadingly negative also cited Schlundt et al (10) accurately elsewhere in the article.

FIGURE 7.

The use of causal language in 91 articles that cited Wyatt et al (11). Articles that cited Wyatt et al (11) were categorized as shown in the table inset in the figure. The left pie chart represents all articles that cited Wyatt et al (11), whereas the right pie chart was limited to relevant citations (n = 72). ¹Unratable citations include accurate citations unrelated to breakfast and weight and citations for which it was unclear what was being attributed to the Wyatt et al (11) article.

FIGURE 8.

Hypothetical model of how BRR and RLPV may be involved in perpetuating presumptions. A: Exposure to such phrases as “breakfast is the most important meal of the day” may predispose individuals to believe positive things about breakfast because of the “halo effect” (58). B: The RLPV related to the PEBO may augment this predisposition to believe the PEBO through the “mere exposure effect,” (59) particularly when the research is presented in a biased manner (eg, in the presence of BRR). C: Individuals tend to seek out information confirming their point of view and reject information to the contrary (confirmation bias) to prevent or reduce cognitive dissonance (60), thereby retaining a biased sample of information. D. A hypothetical illustrative graph of the comparison of strength of conviction compared with the strength of existing evidence supporting the PEBO. E: Together, these cognitive biases may predispose researchers to bias their research reporting, which, thereby, would feed the cycle. BRR, biased research reporting; PEBO, proposed effect of breakfast on obesity; RLPV, research lacking probative value.