Cognitive domain harmonization and cocalibration in studies of older adults

Abstract

Objective: Studies use different instruments to measure cognitirating cognitive tests permit direct comparisons of individuals across studies and pooling data for joint analyses.

Method: We began our legacy item bank with data from the Adult Changes in Thought study (n = 5,546), the Alzheimer's Disease Neuroimaging Initiative (n = 3,016), the Rush Memory and Aging Project (n = 2,163), and the Religious on such as the Mini-Mental State Examination, the Alzheimer's Disease Assessment Scale-Cognitive Subscale, the Wechsler Memory Scale, and the Boston Naming Test. CocalibOrders Study (n = 1,456). Our workflow begins with categorizing items administered in each study as indicators of memory, executive functioning, language, visuospatial functioning, or none of these domains. We use confirmatory factor analysis models with data from the most recent visit on the pooled sample across these four studies for cocalibration and derive item parameters for all items. Using these item parameters, we then estimate factor scores along with corresponding standard errors for each domain for each study. We added additional studies to our pipeline as available and focused on thorough consideration of candidate anchor items with identical content and administration methods across studies.

Results: Prestatistical harmonization steps such qualitative and quantitative assessment of granular cognitive items and evaluating factor structure are important steps when trying to cocalibrate cognitive scores across studies. We have cocalibrated cognitive data and derived scores for four domains for 76,723 individuals across 10 studies.

Conclusions: We have implemented a large-scale effort to harmonize and cocalibrate cognitive domain scores across multiple studies of cognitive aging. Scores on the same metric facilitate meta-analyses of cognitive outcomes across studies or the joint analysis of individual data across studies. Our systematic approach allows for cocalibration of additional studies as they become available and our growing item bank enables robust investigation of cognition in the context of aging and dementia. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Figure 1. Cocalibration Workflow

Note. Each of these steps is explained in more detail below. M = memory; E = executive functioning; L = language; V = visuospatial functioning; ACT = Adult Changes in Thought; ADNI = Alzheimer’s Disease Neuroimaging Initiative; ROS/MAP = Religious Orders Study/Memory and Aging Project.

Figure 2. Single Factor (Left) and Bifactor (Right) Models of 14 Items From a Single Study

Note. The figure to the left depicts a single factor model of 14 items (1–7 and 11–17) that are depicted as loading on a single common factor. There are no secondary domains or residual covariances; this model forces all covariance between items to be captured by the single general factor (labeled “Domain” here). The figure to the right depicts the same 14 items, and a relationship with a general factor that captures covariance across all of the items. But different from the figure to the left, this bifactor model includes two subdomains (labeled “Subdomain 1” and “Subdomain 2”). These subdomains capture covariance among the subdomain items (e.g., Items 1–7 for Subdomain 1, and Items 11–17 for Subdomain 2) that is not shared with items outside that subdomain. A subdomain could be based on a methods effect (e.g., the same words from a word list learning task), or based on a common subset of a higher order domain (e.g., several items tapping set shifting in a model of executive functioning), or a data-driven subset based on agglomerative hierarchical clustering.

Figure 3. Data From Two Studies Illustrating Anchor Items

Note. This figure depicts data from a single domain for two studies. The blue study items are the same as those shown in Figure 2 in the bifactor model. The red study items appear to have some overlap, as depicted in the dashed blue boxes—red items 4–7 appear to be the same as blue items 4–7, and red items 15–17 appear to be the same as blue items 15–17. We pay close attention to these candidate anchor items, ensuring that the stimuli are identical and that the response coding is identical. The subset of items for which that turns out to be the case then are treated as anchor items, where the item parameters are forced to be the same between the blue study and the red study. Other items are treated as study-specific items, including those already understood to be study-unique (e.g., blue items 1–3 and 11–14, and red items 8–10 and 18–21).

Figure 4. Cocalibration of the Red and Blue Studies

Note. Cocalibration model for data from Study 1 and Study 2. Study 1 data include blue and purple items, while Study 2 data include purple and red items. Beige items are anchors, which received extra attention and quality control (see above). This is referred to in this document as the “megacalibration model.”

Figure 5. Violin Plot of the Distributions for Each of the Cognitive Scores Across All Time Points by Study Used in Legacy Model

Note. The violin plot displays the median as a circle, the first-to-third interquartile range as a narrow, shaded box, and the lower-to-upper adjacent value range as a vertical line. The violins are mirrored density curves. ACT = adult changes in thought; ADNI = Alzheimer’s Disease Neuroimaging Initiative; ROS = Religious Orders Study; MAP = Memory and Aging Project.

Figure 6. Violin Plot Showing Distribution of Scores Across All Time Points for Four Cognitive Domains in MARS

Note. MARS = Minority Aging Research Study.

Figure 7. Violin Plot Showing Distribution of Scores Across All Time Points for Three Cognitive Domains in UDS 1 & 2 and UDS 3

Note. UDS = uniform data set.