Evaluation of a digital system to predict unplanned admissions to the intensive care unit: A mixed-methods approach

James Malycha; Oliver Redfern; Marco Pimentel; Guy Ludbrook; Duncan Young; Peter Watkinson

doi:10.1016/j.resplu.2021.100193

Evaluation of a digital system to predict unplanned admissions to the intensive care unit: A mixed-methods approach

Resusc Plus. 2021 Dec 23:9:100193. doi: 10.1016/j.resplu.2021.100193. eCollection 2022 Mar.

Authors

James Malycha^{1

2

3}, Oliver Redfern¹, Marco Pimentel¹, Guy Ludbrook², Duncan Young¹, Peter Watkinson¹

Affiliations

¹ Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.
² Discipline of Acute Care Medicine, University of Adelaide, South Australia, Australia.
³ Intensive Care Unit, The Queen Elizabeth Hospital, Adelaide, South Australia, Australia.

Abstract

Background: We have developed the Hospital Alerting Via Electronic Noticeboard (HAVEN) which aims to identify hospitalised patients most at risk of reversible deterioration. HAVEN combines patients' vital-sign measurements with laboratory results, demographics and comorbidities using a machine learnt algorithm.

Objectives: The aim of this study was to identify variables or concepts that could improve HAVEN predictive performance.

Methods: This was an embedded, mixed methods study. Eligible patients with the five highest HAVEN scores in the hospital (i.e., 'HAVEN Top 5') had their medical identification details recorded. We conducted a structured medical note review on these patients 48 hours post their identifiers being recorded. Methods of constant comparison were used during data collection and to analyse patient data.

Results: The 129 patients not admitted to ICU then underwent constant comparison review, which produced three main groups. Group 1 were patients referred to specialist services (n = 37). Group 2 responded to ward-based treatment, (n = 38). Group 3 were frail and had documented treatment limitations (n = 47).

Conclusions: Digital-only validation methods code the cohort not admitted to ICU as 'falsely positive' in sensitivity analyses however this approach limits the evaluation of model performance. Our study suggested that coding for patients referred to other specialist teams, those with treatment limitations in place, along with those who are deteriorating but then respond to ward-based therapies, would give a more accurate measure of the value of the scores, especially in relation to cost-effectiveness of resource utilisation.

Keywords: Clinical deterioration; Critical care unit; Electronic patient record; Intensive care unit; Predictive score; Qualitative medical note review.