Objectives: The aim of this study was to assess the construct validity of survey questions about the use of herbal and other nonvitamin/nonmineral dietary supplements.
Design: We conducted one-on-one, in-depth cognitive interviews with 32 respondents to test questions from the complementary and alternative medicine (CAM) supplement for the 2007 National Health Interview Survey. Respondents were sampled purposively according to their use of CAM. Interviewers probed respondents for their understanding of the questions, and analysis was guided by grounded theory, an approach that generates explanations of response error that are closely tied to the empirical data.
Results: We found two sources of misinterpretation of CAM questions. First, some respondents did not have any pre-established definition of what constitutes an herbal supplement while others had interpretations that did not match the intended definitions. These problems are common to many survey questions. However, a second finding is that misinterpretation also arose when respondents incorporated notions of self-concept into the act of taking "natural herbs," and answered based on their understanding of this image rather than on actual behavior.
Conclusions: There are several sources of misinterpretation of CAM questions. One of the most important sources is whether or not the respondent has created a concept of self-image that includes the use of herbal supplements. Common questionnaire design techniques such as providing definitions to respondents will not help to eliminate misinterpretation due to self-image. We found that careful question wording that does not evoke definitions of self, combined with visual aids that narrow the focus of the questions, can lead to more accurate answers.