Brain-Cognition Fingerprinting via Graph-GCCA with Contrastive Learning

Abstract

Many longitudinal neuroimaging studies aim to improve the understanding of brain aging and diseases by studying the dynamic interactions between brain function and cognition. Doing so requires accurate encoding of their multidimensional relationship while accounting for individual variability over time. For this purpose, we propose an unsupervised learning model (called Contrastive Learning-based Graph Generalized Canonical Correlation Analysis (CoGraCa)) that encodes their relationship via Graph Attention Networks and generalized Canonical Correlational Analysis. To create brain-cognition fingerprints reflecting unique neural and cognitive phenotype of each person, the model also relies on individualized and multimodal contrastive learning. We apply CoGraCa to longitudinal dataset of healthy individuals consisting of resting-state functional MRI and cognitive measures acquired at multiple visits for each participant. The generated fingerprints effectively capture significant individual differences and outperform current single-modal and CCA-based multimodal models in identifying sex and age. More importantly, our encoding provides interpretable interactions between those two modalities.

Fig. 1.

Overview of our model. (A) Brain functional connectivity of each subject is encoded using graph attention networks (GAT) into graph embedding. The learned embedding (graph variables) and cognitive measures (cognitive variables) are mapped to a shared “brain-cognition” space via generalized canonical correlation analysis. The GAT weights are updated by optimizing the maximum correlation between two modalities. (B) An individualized contrastive learning differentiates inter-subjects brain connectivity and a multimodal contrastive learning to align brain connectivity and cognitive measures across multiple visits within subjects, capturing intra-subject and cross-modal dynamics.

Fig. 2.

Histograms show the similarity in the representations between visits within subject (intra-subject, red) vs. across subjects (inter-subject, blue). Of all methods, CoGraCa model produced the most individualized representation, i.e., the intra-subject similarity is relatively high compared to the inter-subject similarity.

Fig. 3.

Identified functional connectivity and rank-ordered positive loadings of cognitive variables of task-related CCA component, in line with [9,17,19,23,29,31]. Functional regions and cognitive variables were detailed in Supplemental Fig. S4.