What Insights Can We Gain from Patients’ Comments?
Recent studies indicate sharing patient narratives, otherwise known as patient comments, can significantly enhance a hospital’s understanding of the care they provide and ways to enhance it. One study revealed sharing patient comments correlated with improved staff performance, particularly among staff confident in their knowledge1. Another study demonstrated structured feedback reporting methods incorporating patient narratives drove positive changes in staff learning, behavior change, and patient experience scores, especially in non-clinical interactions2.
At MHO, patient comments are gathered upon discharge through the Patient Satisfaction Survey. In addition to rating several standardized items, patients have the opportunity to share open feedback via comment. From 2022 – 2023, 45% of patients took advantage of this opportunity. Interestingly, patients who did and did not leave a comment reported similar levels of satisfaction with their stay, with both groups having an average satisfaction of 4.38 as well as similar likelihood to recommend the facility (Net Promoter Score; NPS, 38.92 Vs. 39.01).
What are patients telling us in their comments?
Sentiment analysis allows for mass analysis of comments, helping to derive themes and whether or not patients have positive or negative feelings on those themes. Overall, patients used slightly more positive words (55.86%, N=52,171) than negative. Furthermore, the positive word percentage did not vary amongst promoters, detractors, or passives (Net Promoter Score categories). In other words, patients who promoted, detracted, or were passive were similarly likely to use positive and negative sentiment words. Furthermore, there was a significant overlap in the most used words among the three groups, suggesting that patients express similar topics across the board.
Figure 1. Positive and Negative Word Frequency by Net Promoter Category (Not Interactive). Word size indicates frequency, where larger words occur more often.
Relationship with Satisfaction and NPS
The positive and negative sentiment of each word in a comment was used to determine an overall comment sentiment score, an indication of whether the comment conveyed positive (sentiment score above 0), negative (sentiment score below 0), or neutral (sentiment score=0) sentiment. Some words could not be assigned a sentiment, for example “food”, “offer”, or “individual”, as these words are not inherently positive or negative. As a result, more than half of comments (54.18%) could not be assigned a sentiment when they contained only those types of words. These comments are typically brief in nature, containing, on average, about half the characters compared to comments with a sentiment rating.
We investigated whether patients whose comments were rated as more negative or positive in sentiment also reported higher satisfaction scores or were more likely to recommend the facility. Surprisingly, we found no such relationship. It may be that patients use this qualitative survey item to express concerns or thoughts not captured by standard rating scale items, making it a separate factor from overall satisfaction or the likelihood to recommend/Net Promoter Score item. For example, a patient may express overall satisfaction and a high likelihood to recommend the facility, yet provide thoughtful feedback suggesting improvements, resulting in a negative sentiment score.
Figure 2. Comment sentiment score is not related to Overall Satisfaction Mean or NPS Score (Not Interactive)
Exploring Topics
To deepen our understanding of emerging topics in the comments, we analyzed the the frequency at which two words are paired, and then merged pairs that convey similar themes (for example, merging “violent patients” and “aggressive patients”, or “feeling unsafe” and “feeling safe”). We then illustrated the distribution of positive, negative, and neutral comments for these pairings. The most common pairings, regardless of sentiment, were “individual therapy”, “food options” and “mental health”. Topics related to a “comfortable environment” and expressions of gratitude appear in comments with an overall positive sentiment. Conversely, phrases like “violent patients”, “smoke breaks”, and “night shift” tend to appear in comments with an overall negative sentiment. It’s worth mentioning that in sentiment analysis, the term “smoke” is categorized as conveying negative sentiment.
Figure 3. Frequency of paired words and the breakdown of sentiment for comments containing them. (Interactive)
Conclusion
While further fine tuning is needed in sentiment analysis techniques, including comparing sentiment dictionaries, and employing more advanced thematic content analysis, current data suggest patients’ comments have a place in organizational strategy to uncover insights beyond standard rating scale satisfaction items. All feedback, regardless of sentiment, can offer valuable insight for staff improvement and can serve as encouragement when positive feedback is received. For this reason, MHO provides valuable, on demand comment reporting for all its clients participating in patient satisfaction.
References
1. Nembhard, I. M., Matta, S., Shaller, D., Lee, Y. S. H., Grob, R., & Schlesinger, M. (2024). Learning from patients: The impact of using patients’ narratives on patient experience scores. Health Care Management Review, 49(1), 2. https://doi.org/10.1097/HMR.0000000000000386
2. Lee, S. H. Y., Grob, R., Nembhard, I., Shaller, D., & Schlesinger, D. (2023) Leveraging Patients’ Creative Ideas for Innovation in Health Care, The Milbank Quarterly, 10.1111/1468-0009.12682