Objective: To assess the performance of shortlisting against appointability to public health specialty training under the Medical Training Application Service (MTAS) selection methodology using multiple modality in person assessment.
Methods: Candidates who had applied to public health specialty training programme in Wales and East of England and shortlisted were assessed in the first assessment round. Further to MTAS review, candidates not previously short listed were offered assessment in the second round. Receiver operating characteristic (ROC) analysis was done.
Results: In both the programmes, the shortlisting scores of candidates considered appointable were substantially higher than those considered not appointable, a score difference of 13.0 (95% confidence interval (CI) 3.0-23.0) and 13.5 (95% CI 3.4-23.5) respectively. The area under the ROC curve (ROCAUC) was 0.88 (95% CI 0.63-1.00) in Wales and 0.77 (95% CI 0.57-0.97) in East of England. The shortlisting scores of the two programmes that gave an optimum performance (maximum sum of the sensitivity and specificity) were comparable (scores of 62 and 63 respectively).
Conclusion: MTAS shortlisting undertaken in two independent public health specialty training programmes discriminated well between appointable and not appointable candidates.