Objective: Peer review is an important part of the publication of scientific results. We analysed the peer review process of abstracts submitted to the European Association for the Study of the Liver (EASL) congress in Copenhagen in 1995.
Design: Comparative study. All abstracts were reviewed by all six members of the EASL scientific committee. Furthermore, each abstract was evaluated by three to six experts within 17 specific study fields, appointed by the scientific committee. All abstracts received a score on a scale from 1 to 5, one being the best. For each abstract the average was calculated for the scores of the committee members and the expert reviewers separately.
Main outcome measure: Abstract score.
Results: There was a significant difference in the scores between the committee members, who scored 3.31 on average, and the expert reviewers, who scored 3.02 on average. Within most of the study fields the tendency was that the expert reviewers gave the highest scores. Moreover, there was a significant variation in the estimates among the individual reviewers as well as between the 2 groups of reviewers. Only 25% of the abstracts that received the best scores by the expert reviewers were classified in the same way by the committee members. The predictive value of a positive evaluation by the committee members was 0.6, the predictive value of a negative evaluation was 0.90.
Conclusions: There was a considerable variation in the peer review process for abstracts submitted to the 1995 EASL congress depending on selection of reviewers. Various types of peer review strategies should be tested and evaluated to secure the highest possible scientific quality of published research results.