Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jan 8;19(1):16.
doi: 10.1186/s12913-018-3839-6.

The Harvard medical practice study trigger system performance in deceased patients

Dorthe O Klein  1 , Roger J M W Rennenberg  2 , Richard P Koopmans  2 , Martin H Prins  3
Affiliations

The Harvard medical practice study trigger system performance in deceased patients

Dorthe O Klein et al. BMC Health Serv Res. .

Abstract

Background: To detect possible threats to quality and safety, multiple systems have been developed. One of them is retrospective chart review. A team of experts scrutinizes medical records, selected by trigger systems, to detect possible adverse events (AEs). The most important AEs and more hints for possible improvement of care appear in deceased patients. Using triggers in a sample of these patients might increase the performance and lower the burden of scrutinizing records without possible preventable AEs. The aim of this study was therefore to determine the performance of the trigger system in a sample of deceased patients and to calculate the specificity and the sensitivity of this trigger system for predicting AEs.

Methods: We performed a study in which the records of deceased patients were screened for triggers by a team of trained nurses. A sample of 100 medical records was randomly selected out of records which had been screened between 2012 and 2015 for the first time, prior to the study in 2016. For the determination of significant differences between the first and second screening, McNemar's test of symmetry was used. Also, observed agreement, Cohen's Kappa and prevalence-adjusted and-bias-adjusted-kappa (PABAK) statistics were calculated. This was done for the two trigger rounds on both any trigger present and for every trigger separately.

Results: The observed agreement for any given trigger was 75% with a Kappa and PABAK of 0.5. For the individual triggers, the observed agreement was on average 90%. The corresponding Kappa was on average 0.42 (range: - 0.03-0.78) and the average PABAK was 0.8 (range: 0.44-0.92). Two adverse events were found in cases without triggers previously. The recalculated specificity and sensitivity for the original population were 58 and 92% respectively.

Conclusions: For the reproducibility of triggers it seems that some perform better than others, but on average this is to our opinion suboptimal. The low specificity implies that many records are selected without AEs. This leads to a high false-positive rate making this labour-intensive record review process costly. Therefore, research for better and more expedient systems is required.

Keywords: Hospital; Kappa; Medical record review; PABAK; Reproducibility; Triggers.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

The study protocol was approved by the medical ethics committee of the Maastricht University Medical Centre. Retrospective medical record review of deceased patients is not covered by the WMO. On such retrospective examination, only the Medical Treatment Agreement Act (WGBO) applies and, the Personal Data Protection Act (Wbp) has fallen. Furthermore, it was checked whether the patient included in this study, ever made objections to using their data for study purposes. If so, their data would not have been included.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Procedure of medical record review in our centre. (a) Identification details of all deceased patients are inserted in the Medirede® database (b) and the corresponding medical record is screened by one of the nurses. (c) When one or more triggers are found in the medical record (registered in Medirede), the case is flagged and will continue to the review committee. (d) When no trigger is found (also registered in Medirede), no further action will be taken in the normal screening procedure. (e) The committee evaluates the cases and will determine whether (f) an adverse event occurred or (g) not.
Fig. 2
Fig. 2
Study flow
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Kohn LT, Corrigan JM, Donaldson MS, editors. To Err is human: building a safer health system. Washington: National Academies Press (US); 2000. - PubMed
    1. Zegers M, et al. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care. 2009;18(4):297–302. doi: 10.1136/qshc.2007.025924. - DOI - PubMed
    1. Langelaan, M., Monitor zorggerelateerde schade 2011/2012 - Dossieronderzoek in Nederlandse ziekenhuizen. 2013, EMGO+ Instituut/VUmc, NIVEL, Nederlands Instituut voor onderzoek van de gezondheidszorg.
    1. Langelaan, M., Baines, R.J., Broekens, M.A. , Monitor Zorggerelateerde Schade 2008 - Dossieronderzoek in Nederlandse ziekenhuizen. 2008, EMGO+ Instituut/VUmc, NIVEL, Nederlands Instituut voor onderzoek van de gezondheidszorg.
    1. Landrigan CP, et al. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363(22):2124–2134. doi: 10.1056/NEJMsa1004404. - DOI - PubMed