The value of psychometric analysis of the advanced trauma life support cognitive test: Outcome of an ACS-Accredited educational institute multisite study

Am J Surg. 2019 Apr;217(4):800-805. doi: 10.1016/j.amjsurg.2018.07.050. Epub 2018 Aug 3.


Background: The Advanced Trauma Life Support® (ATLS®) course provides a standard approach to trauma. Participants must pass the ATLS® post-test. We deployed the test online to allow ongoing psychometric item analysis and potential objective refinement.

Methods: A two-phase study was undertaken with the ACS COT permission. In the first phase, ATLS® post-test #2 was computerized and deployed using Qualtrics©. Data were collected from fourteen courses conducted between 2014 and 2015 (n = 306) at one ACS AEI site. In the second phase, the same post-test was administered to 238 trainees in 10 courses via secured computers at four ACS AEI sites in 2016.

Results: Phase 1 item analyses showed two items with very low percentages correct, and one of these also showed a low discrimination index. Phase 2 item analyses suggested four items as candidates for review and possible revision. We also found differences by learner background and by instructional site.

Conclusions: This study demonstrates computerized delivery of the ATLS post-test is feasible, promotes psychometric analysis, and could improve the quality of the test. Further collaboration between the ACS COT and ACS AEI would be beneficial.

Keywords: Advanced trauma life support; Educational measurement; Psychometrics.

Publication types

  • Multicenter Study

MeSH terms

  • Accreditation
  • Advanced Trauma Life Support Care*
  • Educational Measurement / methods*
  • Female
  • Humans
  • Male
  • Psychometrics*
  • Traumatology / education*
  • United States