Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

doi:10.1016/j.amjsurg.2013.08.031

Comparative Study

. 2014 Apr;207(4):584-95.

doi: 10.1016/j.amjsurg.2013.08.031. Epub 2013 Nov 7.

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

Affiliations

¹ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA; Department of Surgery, Boston University School of Medicine, Boston, MA, USA. Electronic address: hillary.mull@va.gov.
² Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA; Department of Health Policy and Management, Boston University School of Public Health, Boston, MA, USA.
³ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA.
⁴ Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA; Harvard Medical School, Boston, MA, USA.
⁵ Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA.
⁶ VA Boston Healthcare System, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
⁷ Department of Surgery, Boston University School of Medicine, Boston, MA, USA; VA Boston Healthcare System, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
⁸ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA; Department of Surgery, Boston University School of Medicine, Boston, MA, USA.

PMID: 24290888
PMCID: PMC4557788
DOI: 10.1016/j.amjsurg.2013.08.031

Comparative Study

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

Hillary J Mull et al. Am J Surg. 2014 Apr.

. 2014 Apr;207(4):584-95.

doi: 10.1016/j.amjsurg.2013.08.031. Epub 2013 Nov 7.

Authors

Affiliations

¹ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA; Department of Surgery, Boston University School of Medicine, Boston, MA, USA. Electronic address: hillary.mull@va.gov.
² Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA; Department of Health Policy and Management, Boston University School of Public Health, Boston, MA, USA.
³ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA.
⁴ Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA; Harvard Medical School, Boston, MA, USA.
⁵ Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, MA, USA.
⁶ VA Boston Healthcare System, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
⁷ Department of Surgery, Boston University School of Medicine, Boston, MA, USA; VA Boston Healthcare System, Boston, MA, USA; Harvard Medical School, Boston, MA, USA.
⁸ Center for Organization, Leadership, and Management Research, VA Boston Healthcare System, 150 S Huntington Avenue (152M), Boston, MA 02130, USA; Department of Surgery, Boston University School of Medicine, Boston, MA, USA.

PMID: 24290888
PMCID: PMC4557788
DOI: 10.1016/j.amjsurg.2013.08.031

Abstract

Background: The Patient Safety Indicators (PSIs) use administrative data to screen for select adverse events (AEs). In this study, VA Surgical Quality Improvement Program (VASQIP) chart review data were used as the gold standard to measure the criterion validity of 5 surgical PSIs. Independent chart review was also used to determine reasons for PSI errors.

Methods: The sensitivity, specificity, and positive predictive value of PSI software version 4.1a were calculated among Veterans Health Administration hospitalizations (2003-2007) reviewed by VASQIP (n = 268,771). Nurses re-reviewed a sample of hospitalizations for which PSI and VASQIP AE detection disagreed.

Results: Sensitivities ranged from 31% to 68%, specificities from 99.1% to 99.8%, and positive predictive values from 31% to 72%. Reviewers found that coding errors accounted for some PSI-VASQIP disagreement; some disagreement was also the result of differences in AE definitions.

Conclusions: These results suggest that the PSIs have moderate criterion validity; however, some surgical PSIs detect different AEs than VASQIP. Future research should explore using both methods to evaluate surgical quality.

Keywords: Administrative data; Adverse surgical events; Patient Safety Indicators; VA Surgical Quality Improvement Program; Veterans Health Administration.

Published by Elsevier Inc.

PubMed Disclaimer

Figures

**Figure 1. Results of Merge between VA Administrative and VASQIP Datasets**
NOTES: VASQIP=VA Surgical Quality Improvement Program PTF=Patient Treatment File DRG=Diagnosis-Related Group SCRSSN=Scrambled social security number

See this image and copyright information in PMC

Cited by

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data.
Connolly A, Kirwan M, Matthews A. Connolly A, et al. Int J Qual Health Care. 2024 May 10;36(2):mzae037. doi: 10.1093/intqhc/mzae037. Int J Qual Health Care. 2024. PMID: 38662407 Free PMC article. Review.
Incidence and Risk Factors for Complications and Mortality After Vertebroplasty or Kyphoplasty in the Osteoporotic Vertebral Compression Fracture-Analysis of 1,932 Cases From the American College of Surgeons National Surgical Quality Improvement.
Kim HJ, Zuckerman SL, Cerpa M, Yeom JS, Lehman RA Jr, Lenke LG. Kim HJ, et al. Global Spine J. 2022 Jul;12(6):1125-1134. doi: 10.1177/2192568220976355. Epub 2020 Dec 30. Global Spine J. 2022. PMID: 33380221 Free PMC article.
Novel Method to Flag Cardiac Implantable Device Infections by Integrating Text Mining With Structured Data in the Veterans Health Administration's Electronic Medical Record.
Mull HJ, Stolzmann KL, Shin MH, Kalver E, Schweizer ML, Branch-Elliman W. Mull HJ, et al. JAMA Netw Open. 2020 Sep 1;3(9):e2012264. doi: 10.1001/jamanetworkopen.2020.12264. JAMA Netw Open. 2020. PMID: 32955571 Free PMC article.
Sensitivity of Administrative Coding in Identifying Inpatient Acute Strokes Complicating Procedures or Other Diseases in UK Hospitals.
Li L, Binney LE, Carter S, Gutnikov SA, Beebe S, Bowsher-Brown K, Silver LE, Rothwell PM; Oxford Vascular Study. Li L, et al. J Am Heart Assoc. 2019 Jul 16;8(14):e012995. doi: 10.1161/JAHA.119.012995. Epub 2019 Jul 3. J Am Heart Assoc. 2019. PMID: 31266385 Free PMC article.
Frequency and Impact of Adverse Events in Inpatients: A Nationwide Analysis of Episodes between 2000 and 2015.
Sousa-Pinto B, Marques B, Lopes F, Freitas A. Sousa-Pinto B, et al. J Med Syst. 2018 Jan 26;42(3):48. doi: 10.1007/s10916-018-0898-5. J Med Syst. 2018. PMID: 29374332

See all "Cited by" articles

References

1. Bruce J, Russell EM, Mollison J, Krukowski ZH. The measurement and monitoring of surgical adverse events. Health Technol Assess. 2001;5(22):1–194. - PubMed
1. Morris JA, Jr, Carrillo Y, Jenkins JM, et al. Surgical adverse events, risk management, and malpractice outcome: morbidity and mortality review is not enough. Ann Surg. 2003 Jun;237(6):844–851. discussion 851–842. - PMC - PubMed
1. Institute of Medicine. To Err is Human: Building A Safer Health System. Washington, DC: National Academy Press; 1999.
1. Pronovost PJ, Goeschel CA, Marsteller JA, Sexton JB, Pham JC, Berenholtz SM. Framework for patient safety research and improvement. Circulation. 2009 Jan 20;119(2):330–337. - PubMed
1. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998 May-Jun;5(3):305–314. - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

VA999999/Intramural VA/United States

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations
Medical
- MedlinePlus Health Information

[1] Bruce J, Russell EM, Mollison J, Krukowski ZH. The measurement and monitoring of surgical adverse events. Health Technol Assess. 2001;5(22):1–194. - PubMed

[2] Bruce J, Russell EM, Mollison J, Krukowski ZH. The measurement and monitoring of surgical adverse events. Health Technol Assess. 2001;5(22):1–194. - PubMed

[3] Morris JA, Jr, Carrillo Y, Jenkins JM, et al. Surgical adverse events, risk management, and malpractice outcome: morbidity and mortality review is not enough. Ann Surg. 2003 Jun;237(6):844–851. discussion 851–842. - PMC - PubMed

[4] Morris JA, Jr, Carrillo Y, Jenkins JM, et al. Surgical adverse events, risk management, and malpractice outcome: morbidity and mortality review is not enough. Ann Surg. 2003 Jun;237(6):844–851. discussion 851–842. - PMC - PubMed

[5] Institute of Medicine. To Err is Human: Building A Safer Health System. Washington, DC: National Academy Press; 1999.

[6] Institute of Medicine. To Err is Human: Building A Safer Health System. Washington, DC: National Academy Press; 1999.

[7] Pronovost PJ, Goeschel CA, Marsteller JA, Sexton JB, Pham JC, Berenholtz SM. Framework for patient safety research and improvement. Circulation. 2009 Jan 20;119(2):330–337. - PubMed

[8] Pronovost PJ, Goeschel CA, Marsteller JA, Sexton JB, Pham JC, Berenholtz SM. Framework for patient safety research and improvement. Circulation. 2009 Jan 20;119(2):330–337. - PubMed

[9] Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998 May-Jun;5(3):305–314. - PMC - PubMed

[10] Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998 May-Jun;5(3):305–314. - PMC - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

Affiliations

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

Authors

Affiliations

Abstract

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Medical

Abstract

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Related information

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Medical