Learning on knowledge graph dynamics provides an early warning of impactful research

Nat Biotechnol. 2021 Oct;39(10):1300-1307. doi: 10.1038/s41587-021-00907-6. Epub 2021 May 17.


The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for 'impactful' research by autonomously learning high-dimensional relationships among features calculated across time from the scientific literature. We prototype this framework and deduce its performance and scaling properties on time-structured publication graphs from 1980 to 2019 drawn from 42 biotechnology-related journals, including over 7.8 million individual nodes, 201 million relationships and 3.8 billion calculated metrics. We demonstrate the framework's performance by correctly identifying 19/20 seminal biotechnologies from 1980 to 2014 via a blinded retrospective study and provide 50 research papers from 2018 that DELPHI predicts will be in the top 5% of time-rescaled node centrality in the future. We propose DELPHI as a tool to aid in the construction of diversified, impact-optimized funding portfolios.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Bibliometrics*
  • Biomedical Research / economics
  • Biomedical Research / statistics & numerical data*
  • Biomedical Research / trends
  • Biotechnology / economics
  • Biotechnology / statistics & numerical data
  • Biotechnology / trends
  • Humans
  • Journal Impact Factor
  • Machine Learning*
  • Pattern Recognition, Automated
  • Time Factors