Reinforcement learning improves LLM accuracy and reasoning in disease classification from radiology reports

Yishu Wei; Yi Lin; Adam Flanders; George Shih; Yifan Peng

doi:10.1038/s41746-026-02685-4

Reinforcement learning improves LLM accuracy and reasoning in disease classification from radiology reports

NPJ Digit Med. 2026 Apr 30. doi: 10.1038/s41746-026-02685-4. Online ahead of print.

Authors

Yishu Wei^{1

2}, Yi Lin¹, Adam Flanders³, George Shih², Yifan Peng^{4

5}

Affiliations

¹ Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA.
² Department of Radiology, Weill Cornell Medicine, New York, NY, USA.
³ Department of Radiology, Thomas Jefferson University, Philadelphia, PA, USA.
⁴ Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA. yip4002@med.cornell.edu.
⁵ Department of Radiology, Weill Cornell Medicine, New York, NY, USA. yip4002@med.cornell.edu.

PMID: 42062541
DOI: 10.1038/s41746-026-02685-4

Abstract

Accurate disease classification from radiology reports is essential for many applications. While supervised fine-tuning (SFT) of lightweight LLMs improves accuracy, it can degrade reasoning. We propose a two-stage approach: SFT on disease labels followed by Group Relative Policy Optimization (GRPO) to refine predictions by optimizing accuracy and format without reasoning supervision. Across three radiologist-annotated datasets, SFT outperformed baselines and GRPO further improved classification and enhanced reasoning recall and comprehensiveness.

Abstract

Grants and funding