Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 19;31(5):1062-1073.
doi: 10.1093/jamia/ocae032.

Sociotechnical feasibility of natural language processing-driven tools in clinical trial eligibility prescreening for Alzheimer's disease and related dementias

Betina Idnay  1   2 , Jianfang Liu  1 , Yilu Fang  2 , Alex Hernandez  1 , Shivani Kaw  1 , Alicia Etwaru  1 , Janeth Juarez Padilla  1   3 , Sergio Ozoria Ramírez  1   4 , Karen Marder  5 , Chunhua Weng  2 , Rebecca Schnall  1   6
Affiliations

Sociotechnical feasibility of natural language processing-driven tools in clinical trial eligibility prescreening for Alzheimer's disease and related dementias

Betina Idnay et al. J Am Med Inform Assoc. .

Abstract

Background: Alzheimer's disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants.

Objective: This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools' cognitive complexity's effect on usability to identify cognitive support strategies.

Methods: A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models.

Results: Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability.

Discussion: Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools' usability and efficacy in clinical research prescreening.

Conclusion: User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.

Keywords: clinical trial; informatics; research recruitment.

PubMed Disclaimer

Conflict of interest statement

All authors have no competing interests to declare.

Similar articles

See all similar articles

References

    1. World Health Organization. Global Dementia Observatory (GDO). World Health Organization. Updated 2022. Accessed April 11, 2022. https://www.who.int/data/gho/data/themes/global-dementia-observatory-gdo
    1. Alzheimer’s Association. 2022 Alzheimer's disease facts and figures. Alzheimers Dement. 2022;18(4):700-789. 10.1002/alz.12638 - DOI - PubMed
    1. Kramarow EA, Tejada-Vera B.. Dementia mortality in the United States, 2000-2017. Natl Vital Stat Rep. 2019;68(2):1-29. - PubMed
    1. Mokdad AH, Ballestros K, Echko M, et al., US Burden of Disease Collaborators. The state of US health, 1990-2016: burden of diseases, injuries, and risk factors among US states. JAMA. 2018;319(14):1444-1472. 10.1001/jama.2018.0158 - DOI - PMC - PubMed
    1. Alzheimer’s Disease International. World Alzheimer Report 2019: Attitudes to Dementia. 2019. Accessed April 11, 2022. https://www.alz.co.uk/research/WorldAlzheimerReport2019.pdf