Noninvasive heart rate estimation using semantic segmentation and parameter optimization on 4K UAV videos

Abstract

We propose a method for noninvasive heart rate (HR) estimation from ultra-high-definition (4K) UAV-recorded videos, leveraging general-purpose instance segmentation without retraining or manual annotation. The method integrates Fast Segment Anything (FastSAM) with silhouette tracking and signal processing techniques. To address segmentation reliability, we introduce a lightweight heuristic metric (C_seg) that detects abrupt changes in the region of interest, enabling the rejection of unreliable frames. Furthermore, a genetic algorithm is further employed to fine-tune preprocessing and estimation parameters. The method was evaluated on two datasets: the indoor, controlled UBFC-rPPG dataset and a challenging UAV dataset recorded outdoors. On the UBFC-rPPG dataset, our approach achieved up to 76% improvement in heart rate estimation accuracy compared to a baseline algorithm. On the UAV dataset, results indicate more modest gains, limited by motion artifacts, compression, and lighting variability. This work highlights the potential of generic foundation segmentation models in remote physiological monitoring and identifies key challenges for real-world UAV-based applications. Of particular importance here is the fact that the studied objects are moving, while previous studies mainly have focused on stationary objects. Our findings demonstrate that carefully designed segmentation and parameter tuning strategies can enhance rPPG performance in favorable conditions, while also emphasizing the need for future work in motion compensation, segmentation validation, and deployment on embedded systems.

Keywords: Heart rate estimation; Image processing; Noninvasive measurements; Remote photoplethysmography; Remote sensing technology; Semantic segmentation; Signal processing; Skin segmentation; Telemedicine; Ultra-high-definition; Unmanned aerial vehicle; Videoplethysmography.