Evaluating people's ability to assess treatment claims: Validating a test in Mandarin from Claim Evaluation Tools database

Abstract

Objective: To describe the psychometric testing using Rasch analysis of a test in Mandarin developed from the Claim Evaluation Tools database.

Methods: We translated selected MCQs from the IHC Claim Evaluation Tools database to Mandarin and created a test including 24 MCQs covering 11 key concepts. We used purposeful sampling and surveyed children and adults in the Lanzhou area. In total 389 responses were entered into the analysis. We evaluated the psychometric properties of the test using Rasch analysis and the RUMM2030 software, testing for internal construct validity (multidimensionality), invariance of the items (item-person interaction), and item bias (differential item functioning).

Results: Overall, the psychometric properties of the test were found to be satisfactory. Based on findings from the Rasch analysis, we deleted three MCQs with suboptimal fit.

Conclusions: The resulting test includes 21 MCQs and can be used in school and other teaching settings, in randomized trials evaluating outcomes of educational interventions, or in cross-sectional studies in Mandarin-speaking populations in China.

Keywords: evidence-based medicine; health literacy; outcome measurement; patient education; shared decision making.