Assessment tool validation and technical skill improvement in the simulation of the Norwood operation using three-dimensional-printed heart models

Eur J Cardiothorac Surg. 2020 Nov 29;ezaa321. doi: 10.1093/ejcts/ezaa321. Online ahead of print.

Abstract

Objectives: The hands-on surgical training course utilizes 3-dimensional (3D)-printed heart models to simulate complex congenital heart operations. This study aimed to validate a model and assessment tool in the simulation of 2 techniques of the Norwood operation and investigate whether technical performance improves following rehearsal with or without proctor presence.

Methods: Five 'experienced' and 5 'junior' surgeons performed 2 techniques of the Norwood operation on 3D-printed models of hypoplastic left heart syndrome. Performances were retrospectively assessed by 10 raters with varying experience in congenital heart surgery (CHS) (Medical Doctorate versus non-Medical Doctorate). Assessments were made with the procedure-specific Hands-On Surgical Training-CHS tool. Results were analysed for technical performance and rater consistency. Following validation, 30 surgeons (24 with proctor guidance and 6 with training videos and objective feedback only) simulated the Norwood operation twice.

Results: Performance scores were consistently higher for experienced surgeons and raters discriminated clearly between the experienced and junior surgeons (P ≤ 0.001). The hands-on surgical training-CHS tool showed high inter-rater (0.86) and intra-rater (0.80) reliability among all raters. Scores for both experienced and junior surgeons were highly consistent across all raters, with no statistically significant difference (P = 0.50). All surgeons successfully performed the Norwood operation. Sixty attempts were scored in total. Eighty-seven percentage (26/30) of surgeons' scores (mean: attempt 1 = 92, attempt 2 = 104) and times [mean: attempt 1 = 1:22:00, attempt 2 = 1:08:00 (h:mm:ss)] improved between the 2 attempts by 9% and 15% respectively (P ≤ 0.001). Total scores of all surgeons in the non-proctored subgroup (6) improved by 15% on average (mean: attempt 1 = 86, attempt 2 = 105, P = 0.002).

Conclusions: Procedure-specific assessment tools can be developed to evaluate technical performance for complex CHS simulation and be performed reliably by non-expert raters. Rehearsal both under supervision and independently leads to technical skill improvement further supporting its value in CHS training.

Keywords: Congenital heart surgery; Hands-on surgical training; Norwood operation; Objective assessment; Simulation; Three-dimensional printing.