Assessing the reliability of an automated dose-rounding algorithm

Abstract

Objective: Pediatric dose rounding is a unique and complex process whose complexity is rarely supported by e-prescribing systems, though amenable to automation and deployment from a central service provider. The goal of this project was to validate an automated dose-rounding algorithm for pediatric dose rounding.

Methods: We developed a dose-rounding algorithm, STEPSTools, based on expert consensus about the rounding process and knowledge about the therapeutic/toxic window for each medication. We then used a 60% subsample of electronically-generated prescriptions from one academic medical center to further refine the web services. Once all issues were resolved, we used the remaining 40% of the prescriptions as a test sample and assessed the degree of concordance between automatically calculated optimal doses and the doses in the test sample. Cases with discrepant doses were compiled in a survey and assessed by pediatricians from two academic centers. The response rate for the survey was 25%.

Results: Seventy-nine test cases were tested for concordance. For 20 cases, STEPSTools was unable to provide a recommended dose. The dose recommendation provided by STEPSTools was identical to that of the test prescription for 31 cases. For 14 out of the 24 discrepant cases included in the survey, respondents significantly preferred STEPSTools recommendations (p<0.05, binomial test). Overall, when combined with the data from all test cases, STEPSTools either matched or exceeded the performance of the test cases in 45/59 (76%) of the cases. The majority of other cases were challenged by the need to provide an extremely small dose. We estimated that with the addition of two dose-selection rules, STEPSTools would achieve an overall performance of 82% or higher.

Conclusions: Results of this pilot study suggest that automated dose rounding is a feasible mechanism for providing guidance to e-prescribing systems. These results also demonstrate the need for validating decision-support systems to support targeted and iterative improvement in performance.

Keywords: Biomedical informatics; Clinical practice; Computer software; Electronic prescribing; Medical informatics; Prescriptions.