Multi-person feature fusion transfer learning-based convolutional neural network for SSVEP-based collaborative BCI

Penghai Li; Jianxian Su; Abdelkader Nasreddine Belkacem; Longlong Cheng; Chao Chen

doi:10.3389/fnins.2022.971039

Multi-person feature fusion transfer learning-based convolutional neural network for SSVEP-based collaborative BCI

Front Neurosci. 2022 Jul 26:16:971039. doi: 10.3389/fnins.2022.971039. eCollection 2022.

Authors

Penghai Li¹, Jianxian Su¹, Abdelkader Nasreddine Belkacem², Longlong Cheng³, Chao Chen⁴

Affiliations

¹ School of Integrated Circuit Science and Engineering, Tianjin University of Technology, Tianjin, China.
² Department of Computer and Network Engineering, College of Information Technology, UAE University, Al Ain, United Arab Emirates.
³ China Electronics Cloud Brain Technology Co., Ltd., Tianjin, China.
⁴ Key Laboratory of Complex System Control Theory and Application, Tianjin University of Technology, Tianjin, China.

Abstract

Objective: The conventional single-person brain-computer interface (BCI) systems have some intrinsic deficiencies such as low signal-to-noise ratio, distinct individual differences, and volatile experimental effect. To solve these problems, a centralized steady-state visually evoked potential collaborative BCI system (SSVEP-cBCI), which characterizes multi-person electroencephalography (EEG) feature fusion was constructed in this paper. Furthermore, three different feature fusion methods compatible with this new system were developed and applied to EEG classification, and a comparative analysis of their classification accuracy was performed with transfer learning-based convolutional neural network (TL-CNN) approach.

Approach: An EEG-based SSVEP-cBCI system was set up to merge different individuals' EEG features stimulated by the instructions for the same task, and three feature fusion methods were adopted, namely parallel connection, serial connection, and multi-person averaging. The fused features were then input into CNN for classification. Additionally, transfer learning (TL) was applied first to a Tsinghua University (THU) benchmark dataset, and then to a collected dataset, so as to meet the CNN training requirement with a much smaller size of collected dataset and increase the classification accuracy. Ten subjects were recruited for data collection, and both datasets were used to gauge the three fusion algorithms' performance.

Main results: The results predicted by TL-CNN approach in single-person mode and in multi-person mode with the three feature fusion methods were compared. The experimental results show that each multi-person mode is superior to single-person mode. Within the 3 s time window, the classification accuracy of the single-person CNN is only 90.6%, while the same measure of the two-person parallel connection fusion method can reach 96.6%, achieving better classification effect.

Significance: The results show that the three multi-person feature fusion methods and the deep learning classification algorithm based on TL-CNN can effectively improve the SSVEP-cBCI classification performance. The feature fusion method of multi -person parallel feature connection achieves better classification results. Different feature fusion methods can be selected in different application scenarios to further optimize cBCI.

Keywords: collaborative BCI; convolutional neural network; feature fusion; steady-state visually evoked potential; transfer learning.