Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Jul 8;351:h3480.
doi: 10.1136/bmj.h3480.

Evaluation of Symptom Checkers for Self Diagnosis and Triage: Audit Study

Affiliations
Free PMC article

Evaluation of Symptom Checkers for Self Diagnosis and Triage: Audit Study

Hannah L Semigran et al. BMJ. .
Free PMC article

Abstract

Objective: To determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).

Design: Audit study.

Setting: Publicly available, free symptom checkers.

Participants: 23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).

Main outcome measures: For symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).

Results: The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.

Conclusions: Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: all authors are affiliated with Harvard Medical School. Harvard Medical School’s Family Health Guide is used as the basis for one of the symptom checkers evaluated. This symptom checker is available both in print and online (www.health.harvard.edu/family_health_guide/symptoms). None of the authors have been or plan to be involved in the development, evaluation, promotion, or any other facet of a Harvard Medical School related symptom checker; the authors have no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Comment in

Similar articles

See all similar articles

Cited by 50 articles

See all "Cited by" articles

References

    1. Gann B. Giving patients choice and control: health informatics on the patient journey. Yearb Med Inform 2012;7:70-3. - PubMed
    1. Fox S, Duggan M. Health Online 2013. Internet and American life project. Pew Research Center and California Health Care Foundation, 2013:4.
    1. North F, Ward WJ, Varkey P, et al. Should you search the Internet for information about your acute symptom? Telemed J E Health 2012;18:213-8. - PubMed
    1. Black P. The dangers of using Google as a diagnostic aid. Br J Nurs 2009;18:1157. - PubMed
    1. Zhongbo C, Turner MR. The internet for self-diagnosis and prognostication in ALS. Amyotroph Lateral Scler 2010;11:566. - PMC - PubMed

Publication types

MeSH terms

Feedback