Objective: To evaluate the brief Decision Support Analysis Tool (DSAT-10) for auditing the quality of nurse-standardized patient encounters, structuring feedback for nurses, and testing instrument reliability.
Methods: A systematic process was used to develop standardized patient scenarios, pilot-test scenarios, calibrate DSAT-10 coders, analyze taped telephone encounters using DSAT-10, and provide feedback. Inter-rater reliability was calculated using coder agreement, kappa, and intra-class correlation coefficients.
Results: Six scenarios portrayed patients' decisional uncertainty from either: pressure from others (n=2), unclear values (n=2), or inadequate information (n=2). Scenarios were easy to use over the telephone, produced realistic role performance, and were practical for audio-recording interactions. DSAT-10 analysis of 76 nurse-standardized patient encounters revealed nurses' strengths (e.g., information provision) and their limitations (e.g., lack of discussion of values and/or support needs). Scores discriminated between trained and untrained nurses. The kappa coefficient over all items was 0.55 (95% CI: 0.49, 0.61) with higher agreement for encounters involving trained nurses (0.62; 95% CI: 0.43, 0.80).
Conclusion: Auditing nurse-standardized patient encounters using DSAT-10 and providing feedback to nurses was feasible. Although DSAT-10 items had adequate inter-rater reliability and discriminated between trained/untrained nurses, some items were problematic.
Practice implications: Providing feedback on nurse encounters with standardized patients experiencing uncertainty has the potential to enhance nurses' decision support skills.