A figure of merit that includes 5 distinct performance indicators to improve research evaluation of academic scholars'

Heliyon. 2024 Feb 10;10(4):e26235. doi: 10.1016/j.heliyon.2024.e26235. eCollection 2024 Feb 29.

Abstract

The h index has become a widely known indicator to assess the research impact of academic scholars. However, its application has been associated with some criticism regarding its ability to fully capture the quality and significance of an author's research contributions. In this paper, we present a novel approach to improve the evaluation of authors' publications by means of a Figure-of-Merit (FOM) that includes 5 distinct indicators, of which, an enhanced version of the h index. Named the Enhanced Research Quality Index (ERQI), it addresses the current limitations of existing solutions and offers a more comprehensive evaluation of research quality. The ERQI builds upon the concept that one metric is never sufficient to capture the performance of an academic scholar, while multiple ones are complex to handle and interpret. The proposed ERQI considers the total number of citations, papers and co-authors and can further differentiate researchers with equal h index. By incorporating measurable, and quantitative metrics, ERQI moves away from subjective and indirect factors such as journal reputation, citation context, citation patterns and self-citation righteousness, to offer a more nuanced and accurate representation of research quality. To demonstrate the effectiveness of the proposed metric, we conducted a comparative study using a real dataset of 31 researchers in one of the top 3 engineering faculties in Lebanon, and a randomly generated dataset of 1000 author profiles with >1 million citations. Our findings indicate that ERQI provides a more balanced assessment of research quality by reducing the shortcomings of one indicator. Furthermore, it exhibits a multidimensional effect that captures more efficiently the intrinsic value of scholarly contributions. By adopting ERQI, institutions can make informed decisions that recognize both the quantity and quality of an author's research output and can obtain insightful evaluation enabling fairer recognition of academic scholars' impact and innovation.

Keywords: Figure-of-Merit; Publications metrics; Research evaluation; Research impact.