Rankings of American medical schools published annually by the news magazine U.S. News & World Report are widely used to judge the quality of the schools and their programs. The authors describe and then critique the rankings on methodologic and conceptual grounds, arguing that the annual U.S. News medical school evaluation falls short in both areas. Three categories of program quality indicators different from those used by U.S. News are presented as alternative ways to judge medical schools. The authors conclude that the annual U.S. News & World Report rankings of American medical schools are ill-conceived; are unscientific; are conducted poorly; ignore medical school accreditation; judge medical school quality from a narrow, elitist perspective; and do not consider social and professional outcomes in program quality calculations. The medical school rankings have no practical value and fail to meet standards of journalistic ethics.