The World Wide Web creates new challenges and opportunities for medical educators. Prominent among these are the lack of consistent standards by which to evaluate web-based educational tools. We present the instrument that was used to review web-based innovations in medical education submissions to the 2003 Society of General Internal Medicine (SGIM) national meeting, and discuss the process used by the SGIM web-based clinical curriculum interest group to develop the instrument. The 5 highest-ranked submissions are summarized with commentary from the reviewers.