Transformer-based approach to variable typing

Charles Arthel Rey; Jose Lorenzo Danguilan; Karl Patrick Mendoza; Miguel Francisco Remolona

doi:10.1016/j.heliyon.2023.e20505

Transformer-based approach to variable typing

Heliyon. 2023 Sep 29;9(10):e20505. doi: 10.1016/j.heliyon.2023.e20505. eCollection 2023 Oct.

Authors

Charles Arthel Rey¹, Jose Lorenzo Danguilan¹, Karl Patrick Mendoza¹, Miguel Francisco Remolona¹

Affiliation

¹ Chemical Engineering Intelligence Learning Laboratory, Department of Chemical Engineering, University of the Philippines Diliman, Quezon City, 1101 Philippines.

Abstract

The upsurge of multifarious endeavors across scientific fields propelled Big Data in the scientific domain. Despite the advancements in management systems, researchers find that mathematical knowledge remains one of the most challenging to manage due to the latter's inherent heterogeneity. One novel recourse being explored is variable typing where current works remain preliminary and, thus, provide a wide room for contribution. In this study, a primordial attempt to implement the end-to-end Entity Recognition (ER) and Relation Extraction (RE) approach to variable typing was made using the BERT (Bidirectional Encoder Representations from Transformers) model. A micro-dataset was developed for this process. According to our findings, the ER model and RE model, respectively, have Precision of 0.8142 and 0.4919, Recall of 0.7816 and 0.6030, and F1-Scores of 0.7975 and 0.5418. Despite the limited dataset, the models performed at par with values in the literature. This work also discusses the factors affecting this BERT-based approach, giving rise to suggestions for future implementations.

Keywords: Entity recognition; Machine learning; Mathematical knowledge; Natural language processing; Relation extraction; Transformers; Variable typing.