Development of a model and method for hospice quality assessment from natural language processing (NLP) analysis of online caregiver reviews

Abstract

Objectives: With a fraction of hospices having their Consumer Assessment of Healthcare Providers and Systems (CAHPS®) scores on Hospice Compare, a significant reservoir of hospice quality data remains in online caregiver reviews. The purpose of this study was to develop a method and model of hospice quality assessment from caregiver reviews using Watson's carative model.

Methods: Retrospective mixed methods of pilot qualitative thematic analysis and sentiment analysis using NLP of Google and Yelp caregiver reviews between 2013 and 2023. We employed stratified sampling, weighted according to hospice size, to emulate the daily census of enrollees across the United States. Sentiment analysis was performed (n = 3393) using Google NLP.

Results: Two themes with the highest prevalence had moderately positive sentiments (S): Caring staff (+.47) and Care quality, comfort and cleanliness (+.41). Other positive sentiment scores with high prevalence were Gratitude and thanks (+.81), "Treating the patient with respect" (+.54), and "Emotional, spiritual, bereavement support" (+.60). Lowest sentiment scores were "Insurance, administrative or billing" (-.37), "Lack of staffing" (-.32), and "Communication with the family" (-.01).

Significance of results: In the developed quality model, caregivers recommended hospices with caring staff, providing quality care, responsive to requests, and offering family support, including bereavement care. All ten Watson's carative factors and all eight CAHPS measures were presented in the discovered review themes of the quality model. Close-ended CAHPS scores and open-ended online reviews have substantial conceptual overlap and complementary insights. Future hospice quality research should explore caregiver expectations and compare review themes by profit status.

Keywords: CAHPS® scores; Consumer Assessment of Healthcare Providers and Systems; Google Cloud NLP; Hospice caregivers; Hospice quality; Online caregiver reviews.