Dissemination of research findings through publication of results in peer reviewed journals is the gold standard in nursing science. Yet, little is known about quality of manuscript reviews or factors associated with review quality. The purpose of this project was to refine a methodology for assessing quality of reviews and then to evaluate review quality. We created a continuous quality improvement process to assess the narrative portion of 464 reviews of 203 manuscripts submitted to Nursing Research from August 2006-July 2007. The General Assessment of Reviews of Nursing Research (GARNR) was developed to measure quality. Inter-rater reliability of the average of 2 raters' scores was satisfactory for most items and the scales. Quality was better for technical (design and methods) rather than background (theory and review of literature) aspects of a manuscript. Based on assessment of global quality, 18.9% of the reviews were deemed poor or inadequate. On average, statistical reviews were rated more highly than regular reviews, and reviewers from research intensive institutions wrote higher quality reviews than others. Recommendations for monitoring quality in the review process and guidelines for preparation of the review narrative to improve quality are made.