Analysis of CNN-based remote-PPG to understand limitations and sensitivities

Qi Zhan; Wenjin Wang; Gerard de Haan

doi:10.1364/BOE.382637

Analysis of CNN-based remote-PPG to understand limitations and sensitivities

Biomed Opt Express. 2020 Feb 7;11(3):1268-1283. doi: 10.1364/BOE.382637. eCollection 2020 Mar 1.

Authors

Qi Zhan¹, Wenjin Wang^{2

3}, Gerard de Haan³

Affiliations

¹ Department of Electrical and Information Engineering, Hunan University, China.
² Remote Sensing Group, Philips Research, The Netherlands.
³ Electronic Systems Group, Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands.

Abstract

Deep learning based on convolutional neural network (CNN) has shown promising results in various vision-based applications, recently also in camera-based vital signs monitoring. The CNN-based photoplethysmography (PPG) extraction has, so far, been focused on performance rather than understanding. In this paper, we try to answer four questions with experiments aiming at improving our understanding of this methodology as it gains popularity. We conclude that the network exploits the blood absorption variation to extract the physiological signals, and that the choice and parameters (phase, spectral content, etc.) of the reference-signal may be more critical than anticipated. The availability of multiple convolutional kernels is necessary for CNN to arrive at a flexible channel combination through the spatial operation, but may not provide the same motion-robustness as a multi-site measurement using knowledge-based PPG extraction. We also find that the PPG-related prior knowledge may still be helpful for the CNN-based PPG extraction, and recommend further investigation of hybrid CNN-based methods that include prior knowledge in their design.