We evaluated the applicability, reliability, sensitivity, and specificity of six standardized case definitions for adverse events following immunization (AEFI) (for fever, generalized convulsive seizure, hypotonic-hyporesponsive episode, intussusception, nodule, and persistent crying) developed by the Brighton Collaboration using the U.S. Vaccine Adverse Event Reporting System (VAERS). The evaluation included: (a) the development of codified search strings using standardized coding terminology, and (b) for sensitivity and specificity analyses, the development of a "gold standard" for case determination by clinical expert reviews, and its comparison against the application of the definitions to VAERS reports by nonclinicians. Application of the case definitions in an automated approach proved to be valid, feasible, and unlikely to miss confirmed cases of the reported clinical event. The definitions had variable but generally high sensitivity and specificity compared to clinician review, which in itself yielded inconsistent case determination. The study demonstrated the need for the developed standardized definitions for AEFI and their usefulness in passive surveillance.