Chinese Clinical Named Entity Recognition with ALBERT and MHA Mechanism

Dongmei Li; Jiao Long; Jintao Qu; Xiaoping Zhang

doi:10.1155/2022/2056039

Chinese Clinical Named Entity Recognition with ALBERT and MHA Mechanism

Evid Based Complement Alternat Med. 2022 May 23:2022:2056039. doi: 10.1155/2022/2056039. eCollection 2022.

Authors

Dongmei Li^{1

2}, Jiao Long^{1

2}, Jintao Qu^{1

2}, Xiaoping Zhang³

Affiliations

¹ School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China.
² Engineering Research Center for Forestry-oriented Intelligent Information Processing, National Forestry and Grassland Administration, Beijing 100083, China.
³ Institute of Information on Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing 100053, China.

Abstract

Traditional clinical named entity recognition methods fail to balance the effectiveness of feature extraction of unstructured text and the complexity of neural network models. We propose a model based on ALBERT and a multihead attention (MHA) mechanism to solve this problem. Structurally, the model first obtains character-level word embeddings through the ALBERT pretraining language model, then inputs the word embeddings into the iterated dilated convolutional neural network model to quickly extract global semantic information, and decodes the predicted labels through conditional random fields to obtain the optimal label sequence. Also, we apply the MHA mechanism to capture intercharacter dependencies from multiple aspects. Furthermore, we use the RAdam optimizer to boost the convergence speed and improve the generalization ability of our model. Experimental results show that our model achieves an F1 score of 85.63% on the CCKS-2019 dataset-an increase of 4.36% compared to the baseline model.