Development and Evaluation of a Serious Game Application to Engage University Students in Critical Thinking About Health Claims: Mixed Methods Study

Abstract

Background: Misleading health claims are widespread in the media, and making choices based on such claims can negatively affect health. Thus, developing effective learning resources to enable people to think critically about health claims is of great value. Serious games can become an effective learning resource in this respect, as they can affect motivation and learning.

Objective: This study aims to document how user insights and input can inform the concept and development of a serious game application in critical thinking about health claims in addition to gathering user experiences with the game application.

Methods: This was a mixed methods study in 4 successive phases with both qualitative and quantitative data collected in the period from 2020-2022. Qualitative data on design and development were obtained from 4 unrecorded discussions, and qualitative evaluation data were obtained from 1 recorded focus group interview and 3 open-ended questions in the game application. The quantitative data originate from user statistics. The qualitative data were analyzed thematically, and user data were analyzed using nonparametric tests.

Results: The first unrecorded discussion revealed that the students' (3 participants') assessment of whether a claim was reliable or not was limited to performing Google searches when faced with an ad for a health intervention. On the basis of the acquired knowledge of the target group, the game's prerequisites, and the technical possibilities, a pilot of the game was created and reviewed question by question in 3 unrecorded discussions (6 participants). After adjustments, the game was advertised at the Oslo Metropolitan University, and 193 students tested the game. A correlation (r=0.77; P<.001) was found between the number of replays and total points achieved in the game. There was no demonstrable difference (P=.07) between the total scores of students from different faculties. Overall, 36.3% (70/193) of the students answered the evaluation questions in the game. They used words such as "fun" and "educational" about the experiences with the game, and words such as "motivating" and "engaging" related to the learning experience. The design was described as "varied" and "user-friendly." Suggested improvements include adding references, more games and modules, more difficult questions, and an introductory text explaining the game. The results from the focus group interview (4 participants) corresponded to a large extent with the results of the open-ended questions in the game.

Conclusions: We found that user insights and inputs can be successfully used in the concept and development of a serious game that aims to engage students to think critically about health claims. The mixed methods evaluation revealed that the users experienced the game as educational and fun. Future research may focus on assessing the effect of the serious game on learning outcomes and health choices in randomized trials.

Keywords: Informed Health Choices; behind the headlines; critical health literacy; critical thinking; evaluation; game application; mixed methods study; mobile phone; serious games.