Clinical Validation of Heart Rate Apps: Mixed-Methods Evaluation Study

Thijs Vandenberk; Jelle Stans; Christophe Mortelmans; Ruth Van Haelst; Gertjan Van Schelvergem; Caroline Pelckmans; Christophe Jp Smeets; Dorien Lanssens; Hélène De Cannière; Valerie Storms; Inge M Thijs; Bert Vaes; Pieter M Vandervoort

doi:10.2196/mhealth.7254

Clinical Validation of Heart Rate Apps: Mixed-Methods Evaluation Study

JMIR Mhealth Uhealth. 2017 Aug 25;5(8):e129. doi: 10.2196/mhealth.7254.

Authors

Thijs Vandenberk^{1

2}, Jelle Stans¹, Christophe Mortelmans³, Ruth Van Haelst³, Gertjan Van Schelvergem¹, Caroline Pelckmans¹, Christophe Jp Smeets^{1

2}, Dorien Lanssens¹, Hélène De Cannière^{1

2}, Valerie Storms¹, Inge M Thijs¹, Bert Vaes³, Pieter M Vandervoort^{1

2}

Affiliations

¹ Mobile Health Unit, Faculty of Medicine and Life Sciences, Hasselt University, Hasselt, Belgium.
² Department of Cardiology, Ziekenhuis Oost-Limburg, Genk, Belgium.
³ Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium.

Abstract

Background: Photoplethysmography (PPG) is a proven way to measure heart rate (HR). This technology is already available in smartphones, which allows measuring HR only by using the smartphone. Given the widespread availability of smartphones, this creates a scalable way to enable mobile HR monitoring. An essential precondition is that these technologies are as reliable and accurate as the current clinical (gold) standards. At this moment, there is no consensus on a gold standard method for the validation of HR apps. This results in different validation processes that do not always reflect the veracious outcome of comparison.

Objective: The aim of this paper was to investigate and describe the necessary elements in validating and comparing HR apps versus standard technology.

Methods: The FibriCheck (Qompium) app was used in two separate prospective nonrandomized studies. In the first study, the HR of the FibriCheck app was consecutively compared with 2 different Food and Drug Administration (FDA)-cleared HR devices: the Nonin oximeter and the AliveCor Mobile ECG. In the second study, a next step in validation was performed by comparing the beat-to-beat intervals of the FibriCheck app to a synchronized ECG recording.

Results: In the first study, the HR (BPM, beats per minute) of 88 random subjects consecutively measured with the 3 devices showed a correlation coefficient of .834 between FibriCheck and Nonin, .88 between FibriCheck and AliveCor, and .897 between Nonin and AliveCor. A single way analysis of variance (ANOVA; P=.61 was executed to test the hypothesis that there were no significant differences between the HRs as measured by the 3 devices. In the second study, 20,298 (ms) R-R intervals (RRI)-peak-to-peak intervals (PPI) from 229 subjects were analyzed. This resulted in a positive correlation (rs=.993, root mean square deviation [RMSE]=23.04 ms, and normalized root mean square error [NRMSE]=0.012) between the PPI from FibriCheck and the RRI from the wearable ECG. There was no significant difference (P=.92) between these intervals.

Conclusions: Our findings suggest that the most suitable method for the validation of an HR app is a simultaneous measurement of the HR by the smartphone app and an ECG system, compared on the basis of beat-to-beat analysis. This approach could lead to more correct assessments of the accuracy of HR apps.

Keywords: heart rate; remote sensing technology; software validation.

©Thijs Vandenberk, Jelle Stans, Gertjan Van Schelvergem, Caroline Pelckmans, Christophe JP Smeets, Dorien Lanssens, Hélène De Cannière, Valerie Storms, Inge M Thijs, Pieter M Vandervoort. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 25.08.2017.