Performance of popular pulse oximeters compared with simultaneous arterial oxygen saturation or clinical-grade pulse oximetry: a cross-sectional validation study in intensive care patients

BMJ Open Respir Res. 2021 Sep;8(1):e000939. doi: 10.1136/bmjresp-2021-000939.


Objectives: To evaluate the performance of direct-to-consumer pulse oximeters under clinical conditions, with arterial blood gas measurement (SaO2) as reference standard.

Design: Cross-sectional, validation study.

Setting: Intensive care.

Participants: Adult patients requiring SaO2-monitoring.

Interventions: The studied oximeters are top-selling in Europe/USA (AFAC FS10D, AGPTEK FS10C, ANAPULSE ANP 100, Cocobear, Contec CMS50D1, HYLOGY MD-H37, Mommed YM101, PRCMISEMED F4PRO, PULOX PO-200 and Zacurate Pro Series 500 DL). Directly after collection of a SaO2 blood sample, we obtained pulse oximeter readings (SpO2). SpO2-readings were performed in rotating order, blinded for SaO2 and completed <10 min after blood sample collection.

Outcome measures: Bias (SpO2-SaO2) mean, root mean square difference (ARMS), mean absolute error (MAE) and accuracy in identifying hypoxaemia (SaO2 ≤90%). As a clinical index test, we included a hospital-grade SpO2-monitor (Philips).

Results: In 35 consecutive patients, we obtained 2258 SpO2-readings and 234 SaO2-samples. Mean bias ranged from -0.6 to -4.8. None of the pulse oximeters met ARMS ≤3%, the requirement set by International Organisation for Standardisation (ISO)-standards and required for Food and Drug Administration (FDA) 501(k)-clearance. The MAE ranged from 2.3 to 5.1, and five out of ten pulse oximeters met the requirement of ≤3%. For hypoxaemia, negative predictive values were 98%-99%. Positive predictive values ranged from 11% to 30%. Highest accuracy (95% CI) was found for Contec CMS50D1; 91% (86-94) and Zacurate Pro Series 500 DL; 90% (85-94). The hospital-grade SpO2-monitor had an ARMS of 3.0% and MAE of 1.9, and an accuracy of 95% (91%-97%).

Conclusion: Top-selling, direct-to-consumer pulse oximeters can accurately rule out hypoxaemia, but do not meet ISO-standards required for FDA-clearance.

Keywords: COVID-19; respiratory measurement.

Publication types

  • Validation Study

MeSH terms

  • Aged
  • Blood Gas Analysis / instrumentation*
  • Critical Care
  • Cross-Sectional Studies
  • Female
  • Humans
  • Male
  • Middle Aged
  • Oximetry* / instrumentation
  • Oxygen*


  • Oxygen