The purposes of this study were to develop an instrument to assess the validity of randomized controlled trials and to report on the differences in the validity of randomized controlled trials between two podiatric medical journals and a mainstream medical journal. The study demonstrated that after adequate training, there can be agreement among reviewers evaluating the quality of published randomized controlled trials using an established instrument and guidelines. The results of the study indicate that randomized controlled trials published in podiatric medical journals are less credible than those published in a mainstream medical journal.