Statistical approaches to harmonize data on cognitive measures in systematic reviews are rarely reported

Abstract

Objectives: To identify statistical methods for harmonization, the procedures aimed at achieving the comparability of previously collected data, which could be used in the context of summary data and individual participant data meta-analysis of cognitive measures.

Study design and setting: Environmental scan methods were used to conduct two reviews to identify (1) studies that quantitatively combined data on cognition and (2) general literature on statistical methods for data harmonization. Search results were rapidly screened to identify articles of relevance.

Results: All 33 meta-analyses combining cognition measures either restricted their analyses to a subset of studies using a common measure or combined standardized effect sizes across studies; none reported their harmonization steps before producing summary effects. In the second scan, three general classes of statistical harmonization models were identified (1) standardization methods, (2) latent variable models, and (3) multiple imputation models; few publications compared methods.

Conclusion: Although it is an implicit part of conducting a meta-analysis or pooled analysis, the methods used to assess inferential equivalence of complex constructs are rarely reported or discussed. Progress in this area will be supported by guidelines for the conduct and reporting of the data harmonization and integration and by evaluating and developing statistical approaches to harmonization.

Keywords: Cognition; Combination; Data pooling; Harmonization; Individual participant data; Meta-analysis; Transformation.

Figure 1

Proposed harmonization recommendations for researchers and systematic review organization conducting IPD meta-analyses and data pooling