EEG-based image classification via a region-level stacked bi-directional deep learning framework

Abstract

Background: As a physiological signal, EEG data cannot be subjectively changed or hidden. Compared with other physiological signals, EEG signals are directly related to human cortical activities with excellent temporal resolution. After the rapid development of machine learning and artificial intelligence, the analysis and calculation of EEGs has made great progress, leading to a significant boost in performances for content understanding and pattern recognition of brain activities across the areas of both neural science and computer vision. While such an enormous advance has attracted wide range of interests among relevant research communities, EEG-based classification of brain activities evoked by images still demands efforts for further improvement with respect to its accuracy, generalization, and interpretation, yet some characters of human brains have been relatively unexplored.

Methods: We propose a region-level stacked bi-directional deep learning framework for EEG-based image classification. Inspired by the hemispheric lateralization of human brains, we propose to extract additional information at regional level to strengthen and emphasize the differences between two hemispheres. The stacked bi-directional long short-term memories are used to capture the dynamic correlations hidden from both the past and the future to the current state in EEG sequences.

Results: Extensive experiments are carried out and our results demonstrate the effectiveness of our proposed framework. Compared with the existing state-of-the-arts, our framework achieves outstanding performances in EEG-based classification of brain activities evoked by images. In addition, we find that the signals of Gamma band are not only useful for achieving good performances for EEG-based image classification, but also play a significant role in capturing relationships between the neural activations and the specific emotional states.

Conclusions: Our proposed framework provides an improved solution for the problem that, given an image used to stimulate brain activities, we should be able to identify which class the stimuli image comes from by analyzing the EEG signals. The region-level information is extracted to preserve and emphasize the hemispheric lateralization for neural functions or cognitive processes of human brains. Further, stacked bi-directional LSTMs are used to capture the dynamic correlations hidden in EEG data. Extensive experiments on standard EEG-based image classification dataset validate that our framework outperforms the existing state-of-the-arts under various contexts and experimental setups.

Keywords: Classification of brain activities; EEG; Region-level information; Stacked bi-directional LSTM.

Fig. 1

Structural illustration of the proposed deep framework

Fig. 2

Two stacked layer structure in bi-directional LSTM with three consecutive time steps

Fig. 3

Long short-term Memory cell

Fig. 4

The average power in Fz, Cz, Pz, and Oz locations when stimuli are from the categories of “gun” and “phone”, respectively

Fig. 5

The average power of the AFz and Fz locations for the categories of “gun”, “panda”, and “phone”

Fig. 6

Scalp distribution of the average energy at Gamma frequency sub-band for all participants and sessions of the three categories: “gun”, “phone”, and “panda”