This study investigated the automated detection of antiretroviral toxicities in structured electronic health records data. The evaluation compared responses generated by 5 clinical pharmacists and 1 prototype knowledge-based application for 15 randomly selected test cases. The main outcomes were inter-subject dissimilarity of responses quantified by the Jaccard distance, and the mean proportion of correct responses by each subject. The statistical differences in inter-subject Jaccard distances suggested that the prototype was inferior to clinical pharmacists in the detection of possible antiretroviral toxicity associations from structured data. The reason for dissimilarities was attributable to inadequate domain coverage by the prototype. The differences in the mean proportion of correct responses between the clinical pharmacists and the prototype were statistically indistinguishable. Overall, this study suggests that knowledge-based applications have the potential to support automated detection of antiretroviral toxicities from structured patient records. Furthermore, the study demonstrates a systematic approach for validating such applications quantitatively.