Examining the Public's Most Frequently Asked Questions Regarding COVID-19 Vaccines Using Search Engine Analytics in the United States: Observational Study

Abstract

Background: The emergency authorization of COVID-19 vaccines has offered the first means of long-term protection against COVID-19-related illness since the pandemic began. It is important for health care professionals to understand commonly held COVID-19 vaccine concerns and to be equipped with quality information that can be used to assist in medical decision-making.

Objective: Using Google's RankBrain machine learning algorithm, we sought to characterize the content of the most frequently asked questions (FAQs) about COVID-19 vaccines evidenced by internet searches. Secondarily, we sought to examine the information transparency and quality of sources used by Google to answer FAQs on COVID-19 vaccines.

Methods: We searched COVID-19 vaccine terms on Google and used the "People also ask" box to obtain FAQs generated by Google's machine learning algorithms. FAQs are assigned an "answer" source by Google. We extracted FAQs and answer sources related to COVID-19 vaccines. We used the Rothwell Classification of Questions to categorize questions on the basis of content. We classified answer sources as either academic, commercial, government, media outlet, or medical practice. We used the Journal of the American Medical Association's (JAMA's) benchmark criteria to assess information transparency and Brief DISCERN to assess information quality for answer sources. FAQ and answer source type frequencies were calculated. Chi-square tests were used to determine associations between information transparency by source type. One-way analysis of variance was used to assess differences in mean Brief DISCERN scores by source type.

Results: Our search yielded 28 unique FAQs about COVID-19 vaccines. Most COVID-19 vaccine-related FAQs were seeking factual information (22/28, 78.6%), specifically about safety and efficacy (9/22, 40.9%). The most common source type was media outlets (12/28, 42.9%), followed by government sources (11/28, 39.3%). Nineteen sources met 3 or more JAMA benchmark criteria with government sources as the majority (10/19, 52.6%). JAMA benchmark criteria performance did not significantly differ among source types (χ ² ₄=7.40; P=.12). One-way analysis of variance revealed a significant difference in mean Brief DISCERN scores by source type (F _4,23=10.27; P<.001).

Conclusions: The most frequently asked COVID-19 vaccine-related questions pertained to vaccine safety and efficacy. We found that government sources provided the most transparent and highest-quality web-based COVID-19 vaccine-related information. Recognizing common questions and concerns about COVID-19 vaccines may assist in improving vaccination efforts.

Keywords: COVID-19; SARS-CoV-2; content; frequently asked questions; internet; machine learning; natural language processing; quality; question; search analytics; search engine; transparency; vaccine; vaccine hesitancy; web-based health information.