Wheat-Net: An Automatic Dense Wheat Spike Segmentation Method Based on an Optimized Hybrid Task Cascade Model

Jiajing Zhang; An Min; Brian J Steffenson; Wen-Hao Su; Cory D Hirsch; James Anderson; Jian Wei; Qin Ma; Ce Yang

doi:10.3389/fpls.2022.834938

Wheat-Net: An Automatic Dense Wheat Spike Segmentation Method Based on an Optimized Hybrid Task Cascade Model

Front Plant Sci. 2022 Feb 10:13:834938. doi: 10.3389/fpls.2022.834938. eCollection 2022.

Authors

Jiajing Zhang^{1

2

3}, An Min⁴, Brian J Steffenson⁵, Wen-Hao Su⁶, Cory D Hirsch⁵, James Anderson⁷, Jian Wei¹, Qin Ma¹, Ce Yang⁴

Affiliations

¹ College of Information and Electrical Engineering, China Agricultural University, Beijing, China.
² The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China.
³ School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
⁴ Department of Bioproducts and Biosystems Engineering, University of Minnesota, Saint Paul, MN, United States.
⁵ Department of Plant Pathology, University of Minnesota, Saint Paul, MN, United States.
⁶ College of Engineering, China Agricultural University, Beijing, China.
⁷ Department of Agronomy and Plant Genetics, University of Minnesota, Saint Paul, MN, United States.

Abstract

Precise segmentation of wheat spikes from a complex background is necessary for obtaining image-based phenotypic information of wheat traits such as yield estimation and spike morphology. A new instance segmentation method based on a Hybrid Task Cascade model was proposed to solve the wheat spike detection problem with improved detection results. In this study, wheat images were collected from fields where the environment varied both spatially and temporally. Res2Net50 was adopted as a backbone network, combined with multi-scale training, deformable convolutional networks, and Generic ROI Extractor for rich feature learning. The proposed methods were trained and validated, and the average precision (AP) obtained for the bounding box and mask was 0.904 and 0.907, respectively, and the accuracy for wheat spike counting was 99.29%. Comprehensive empirical analyses revealed that our method (Wheat-Net) performed well on challenging field-based datasets with mixed qualities, particularly those with various backgrounds and wheat spike adjacence/occlusion. These results provide evidence for dense wheat spike detection capabilities with masking, which is useful for not only wheat yield estimation but also spike morphology assessments.

Keywords: Hybrid Task Cascade model; challenging dataset; instance segmentation; non-structural field; wheat spike.