Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2019 Nov 6;367:l5896.
doi: 10.1136/bmj.l5896.

Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial

Affiliations
Free PMC article
Randomized Controlled Trial

Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial

Leslie K John et al. BMJ. .
Free PMC article

Abstract

Objective: To assess the effect of disclosing authors' conflict of interest declarations to peer reviewers at a medical journal.

Design: Randomized controlled trial.

Setting: Manuscript review process at the Annals of Emergency Medicine. PARTICIPANTS: Reviewers (n=838) who reviewed manuscripts submitted between 2 June 2014 and 23 January 2018 inclusive (n=1480 manuscripts).

Intervention: Reviewers were randomized to either receive (treatment) or not receive (control) authors' full International Committee of Medical Journal Editors format conflict of interest disclosures before reviewing manuscripts. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain "counterfactual scores"-that is, the scores they believed they would have given had they been assigned to the opposite arm-as well as attitudes toward conflicts of interest.

Main outcome measure: Overall quality score that reviewers assigned to the manuscript on submitting their review (1 to 5 scale). Secondary outcomes were scores the reviewers submitted for the seven more specific quality ratings and counterfactual scores elicited in the follow-up survey.

Results: Providing authors' conflict of interest disclosures did not affect reviewers' mean ratings of manuscript quality (Mcontrol=2.70 (SD 1.11) out of 5; Mtreatment=2.74 (1.13) out of 5; mean difference 0.04, 95% confidence interval -0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol= 2.85 (1.12) out of 5; Mtreatment=2.96 (1.16) out of 5; mean difference 0.11, -0.05 to 0.26). Similarly, no effect of the treatment was seen on any of the other seven quality ratings that the reviewers assigned. Reviewers acknowledged conflicts of interest as an important matter and believed that they could correct for them when they were disclosed. However, their counterfactual scores did not differ from actual scores (Mactual=2.69; Mcounterfactual=2.67; difference in means 0.02, 0.01 to 0.02). When conflicts were reported, a comparison of different source types (for example, government, for-profit corporation) found no difference in effect.

Conclusions: Current ethical standards require disclosure of conflicts of interest for all scientific reports. As currently implemented, this practice had no effect on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Figures

Fig 1
Fig 1
Study flow
Fig 2
Fig 2
CONSORT diagram. Phase 1: randomized trial of provision of conflict of interest (COI) information to reviewers during peer review process. Sample was restricted to manuscripts with at least one control review and one treatment review. Two factors contributed to exclusion of 759 reviews (384 in treatment arm; 375 in control arm) because no accompanying review from other arm for given manuscript was available. (1) Failure to complete review: reviewer assigned to complete review from other arm did not submit review. (2) Allocation error: after first few months of data collection, an error was detected in the algorithm that allocated reviewers to condition, with result that, for some manuscripts, all reviewers were inadvertently assigned to same condition. These manuscripts were therefore excluded from analysis. The error was remedied quickly once it was detected
Fig 3
Fig 3
CONSORT diagram. Phase 2: follow-up survey

Comment in

Similar articles

See all similar articles

References

    1. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev 2017;2:MR000033. - PubMed
    1. Bhandari M, Busse JW, Jackowski D, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 2004;170:477-80. - PMC - PubMed
    1. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003;326:1167-70. 10.1136/bmj.326.7400.1167 - DOI - PMC - PubMed
    1. Friedman LS, Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med 2004;19:51-6. 10.1111/j.1525-1497.2004.30617.x - DOI - PMC - PubMed
    1. International Committee of Medical Journal Editors (ICMJE). Conflicts of interest. 2018. http://www.icmje.org/conflicts-of-interest/.

Publication types

Feedback