The relationship between commercial website ratings and traditional hospital performance measures in the USA

Abstract

Background: Our goal was to compare hospital scores from the most widely used commercial website in the USA to hospital scores from more systematic measures of patient experience and outcomes, and to assess what drives variation in the commercial website scores.

Methods: For a national sample of US hospitals, we compared scores on Yelp.com, which aggregates website visitor ratings (1-5 stars), with traditional measures of hospital quality. We calculated correlations between hospital Yelp scores and the following: hospital percent high ratings (9 or 10, scale 0-10) on the 'Overall' item on the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey; hospital individual HCAHPS domain scores (eg, nurse communication, pain control); hospital 30-day mortality; and hospital 30-day readmission rates.

Results: Of hospitals reporting HCAHPS (n=3796), 962 (25%) had scores on Yelp. Among hospitals with >5 Yelp ratings, the correlation of percent high ratings between Yelp and HCAHPS was 0.49 (p<0.001). The percent high ratings within each HCAHPS domain increased monotonically with increasing Yelp scores (p≤0.001 for all domains). Percent high ratings in Yelp and HCAHPS were statistically significantly correlated with lower mortality for myocardial infarction (MI; -0.19 for Yelp and -0.13 for HCAHPS) and pneumonia (-0.14 and -0.18), and fewer readmissions for MI (-0.17 and -0.39), heart failure (-0.31 and -0.39), and pneumonia (-0.18 and -0.27).

Conclusions: These data suggest that rater experiences for Yelp and HCAHPS may be similar, and that consumers posting ratings on Yelp may observe aspects of care related to important patient outcomes.

Figure 1

Relationship between percent of high ratings on Yelp and Hospital Consumer Assessment of Healthcare Providers and Systems. Hospitals with more than 5 Yelp ratings (n=270) were included. The line-of-fit (solid black line) and 95% confidence interval (grey shading) is portrayed. R, Pearson’s product moment correlation coefficient, used weights inversely proportional to the number of Yelp raters. p Value for correlation is <0.001.