Prospective clinical validation of the Empatica EmbracePlus wristband as a reflective pulse oximeter

Introduction: Respiratory diseases such as chronic obstructive pulmonary disease, obstructive sleep apnea syndrome, and COVID-19 may cause a decrease in arterial oxygen saturation (SaO₂). The continuous monitoring of oxygen levels may be beneficial for the early detection of hypoxemia and timely intervention. Wearable non-invasive pulse oximetry devices measuring peripheral oxygen saturation (SpO₂) have been garnering increasing popularity. However, there is still a strong need for extended and robust clinical validation of such devices, especially to address topical concerns about disparities in performances across racial groups. This prospective clinical validation aimed to assess the accuracy of the reflective pulse oximeter function of the EmbracePlus wristband during a controlled hypoxia study in accordance with the ISO 80601-2-61:2017 standard and the Food & Drug Administration (FDA) guidance.

Methods: Healthy adult participants were recruited in a controlled desaturation protocol to reproduce mild, moderate, and severe hypoxic conditions with SaO₂ ranging from 100% to 70% (ClinicalTrials.gov registration #NCT04964609). The SpO₂ level was estimated with an EmbracePlus device placed on the participant's wrist and the reference SaO₂ was obtained from blood samples analyzed with a multiwavelength co-oximeter.

Results: The controlled hypoxia study yielded 373 conclusive measurements on 15 subjects, including 30% of participants with dark skin pigmentation (V–VI on the Fitzpatrick scale). The accuracy root mean square (A_rms) error was found to be 2.4%, within the 3.5% limit recommended by the FDA. A strong positive correlation between the wristband SpO₂ and the reference SaO₂ was observed (r = 0.96, P < 0.001), and a good concordance was found with Bland–Altman analysis (bias, 0.05%; standard deviation, 1.66; lower limit, −4.7%; and upper limit, 4.8%). Moreover, acceptable accuracy was observed when stratifying data points by skin pigmentation (A_rms 2.2% in Fitzpatrick V–VI, 2.5% in Fitzpatrick I-IV), and sex (A_rms 1.9% in females, and 2.9% in males).

Discussion: This study demonstrates that the EmbracePlus wristband could be used to assess SpO₂ with clinically acceptable accuracy under no-motion and high perfusion conditions for individuals of different ethnicities across the claimed range. This study paves the way for further accuracy evaluations on unhealthy subjects and during prolonged use in ambulatory settings.

1. Introduction

Respiratory diseases like chronic obstructive pulmonary disease (COPD), asthma, obstructive sleep apnea syndrome (OSA), and COVID-19 account for a significant burden of disease (1, 2). COPD, associated with persistent and progressive respiratory symptoms (3), has a global prevalence of 3.9%, and, in 2019, was the third leading cause of death worldwide (1, 3). Asthma is caused by inflammation and narrowing of the small airways in the lungs (4) and affects approximately one in five individuals (5). In the U.S., 26% of individuals between 30 and 70 years suffer from OSA, which manifests in complete or partial upper airway obstruction (6). Over the past three years, clinical COVID-19 associated with the CoV-2 coronavirus has also resulted in substantial morbidity and mortality, causing over 6 million deaths worldwide to date (7). The common pathophysiology underlying these respiratory diseases is impaired gas exchange, which depending on the severity and duration of impairment, can result in hypoxemia (1–8) and associated clinical signs and symptoms ranging from headaches and dyspnea to cellular hypoxia, organ failure, and death in extreme cases (9, 10).

Since arterial oxygen saturation (SaO₂) below the physiological 95%–100% range can be suggestive of respiratory pathology but is not always associated with apparent symptoms such as dyspnea, monitoring of blood oxygenation in individuals at risk of new or worsening hypoxemia is an important tool to enable early identification of clinical decompensation, and to inform timely decision-making about interventions including hospitalization, ICU admission, and supplemental oxygen therapies (e.g., mechanical ventilation) (10). The clinical standard and most accurate method for assessing SaO₂ is co-oximetry, which requires invasive measurements to detect SaO₂ values from blood samples. Co-oximetry allows only intermittent and point-in-time measurements of SaO₂, which are not compatible with continuous monitoring in ambulatory settings and are thus not appropriate for analysis of longitudinal trends in the SaO₂ (9, 11, 12).

Pulse oximetry is a non-invasive modality for monitoring SaO₂ estimations by measuring the peripheral oxygen saturation (SpO₂) using photoplethysmography (PPG) technology. PPG-based fingertip pulse oximeters have become the de facto standard for assessing SpO₂ in hospitalized patients due to their non-invasiveness, flexibility (13, 14), and ability to meet clinical accuracy thresholds established by the ISO 80601-2-61:2017 standard and the US Food & Drug Administration (FDA) guidance on pulse oximetry validation (15, 16), which were established in 2017 and in 2013, respectively. Recently, PPG-based sensors have been increasingly adopted also in mobile digital health technologies for remote monitoring applications, as part of the modern paradigm shift of clinical care and clinical research to home-centered healthcare and decentralized clinical trials (11, 13, 14, 17–19). Among various physiological parameters, SpO₂ measurements in ambulatory settings are now provided by several wireless commercial devices, with different degrees of validation (20) and different form factors, including devices worn on the upper arm (19, 21), on the chest (19, 22), on the ear (23), or on the wrist (11, 18, 19, 24–31).

Among wearables, wrist-worn devices have several advantages thanks to their portability, comfort, easy acceptance, and non-stigmatization, resulting in high compliance (32). However, getting SpO₂ estimations from the wrist is challenging given lower arterial blood perfusion and lower signal-to-noise ratio resulting from multiple tissue scattering in the dorsal wrist compared to other locations (e.g., finger, ear) (14, 33, 34). The number of wrist-worn devices that received clearance from the FDA for PPG-based technologies remains quite low (27–31). These wrist-worn devices have been cleared based on clinical evidence abiding by the FDA/ISO standardized protocols which require a minimum of 200 data points each for 10 or more healthy volunteers that vary in age, sex, and skin pigmentation, including 2 darkly pigmented subjects or 15% of study pool, whichever is greater (15, 16). To date, however, details of validation studies supporting clearance of these devices have not been widely published (26, 35).

Recently, the publication of post-market data related to the real-world use of cleared pulse oximeters has raised questions about the performance of these devices, particularly in individuals with darker skin tones (36), leading to increased scrutiny by the FDA and the scientific and clinical community (37, 38). This issue was documented based on data from finger pulse oximeters using transmissive technology, which has a superior signal-to-noise ratio compared to reflective sensors. Thus, a higher impact of skin color could be expected on devices with reflective PPG technology, since scattering due to melanin would further increase the scattering seen at baseline in wrist-worn devices (39).

Given the paucity of published validation studies for clinical data from wrist-worn wearables, and recently raised concerns that existing pulse oximeters may not perform accurately in racial minorities (36–39), it is critical to rigorously test new wrist-worn pulse oximeters, especially to verify their accuracy and reliability across different skin pigmentations. This work contributes to filling this gap by reporting the prospective validation of the SpO₂ measurements computed by a wrist-worn medical device, i.e., the EmbracePlus wristband, on a pool of subjects enriched for individuals with darker skin pigmentation (30% of study participants). This device and its associated monitoring platform recently received clearance from the FDA to allow healthcare professionals to monitor SpO₂ in no-motion and high perfusion conditions in ambulatory individuals aged 18 years and older in home healthcare settings (40). This work details the methods and the results of the comparison between EmbracePlus SpO₂ measurements and gold standard measures of SaO₂, performed during a controlled hypoxia study following the ISO 80601-2-61:2017 standard (15) and the FDA guidance (16). The validation presented herein complies with recently published suggestions to increase the statistical robustness of results, transparency, and understanding of limitations of pulse oximetry. Moreover, this study reports individual subjects accuracy levels, subgroup analysis by different skin pigmentation on a dataset that doubles the representation of darkly pigmented individuals with respect to FDA guidance (16) and clinically relevant information about outliers (41).

2. Methods

2.1. PPG principles

The SpO₂ algorithm is based on PPG technology and harnesses two principles: (i) the different absorption spectra of the oxygenated (HbO₂) and the deoxygenated (HbH) hemoglobin, and (ii) the presence of a pulsatile arterial blood flow (42, 43). The former is exploited to compute the concentrations of HbO₂ and HbH by illuminating the skin with two different wavelengths, typically red (∼660 nm) and infrared (∼940 nm). Indeed, HbO₂ absorbs more infrared light while the red light more easily passes through, whereas HbH absorbs more red light and allows more infrared light to pass through (42–44). The relative amount of red and infrared light that is reflected towards a photodetector (PD) after being partially absorbed by the arterial blood can be used to ultimately estimate the proportion of the hemoglobin bound to oxygen and therefore the SpO₂ level, based on the Lambert-Beer law of absorbance (45). The second principle leverages the respective inherent contractility of arteries and veins. During each cardiac cycle, the arterial blood volume increases during systole and decreases during diastole, leading to fluctuations in the absorbed red and infrared light, which form the pulsatile (AC) component. By contrast, light that is reflected and reaches the PD from blood volume in veins and capillaries or non-vascular tissues presents a relatively stable, non-pulsatile component, which forms the steady (DC) component (33, 44). The perfusion ratio (R) (i.e., the ratio of AC/DC of red and infrared PPG) is used to derive the SpO₂-estimation using an empirical calibration function derived from the relationship between R and SpO₂, which is obtained in experimental conditions through a stable and controlled hypoxic condition spanning the claimed measurement range (e.g., 70%–100% SpO₂) (Figure 1) (44, 46).

FIGURE 1

Figure 1. Schematic diagram of the basic principle of a pulse oximeter showing the components of the blood flow in the tissues in relation to the optical metrics of interest for SpO₂ computation (AC and DC), with the diagram of a typical regression function that maps AC and DC composite metrics into a SpO₂ Reading.

2.2. EmbracePlus SpO₂ algorithm

The inputs of the SpO₂ algorithm are data obtained illuminating the skin with green, red, and infrared-light PPG sensors and data recorded by three-axis accelerometry (ACM) sensors embedded in the EmbracePlus device (Figure 2). The algorithm analyzes the red and infrared PPG signals to extract the amplitude of the AC pulsatile component and of the DC baseline component (44), using the green PPG to support the detection of AC. Using the Lambert-Beer model (47), these metrics can be used to estimate the SpO₂ value using a calibration model previously obtained during the training phase, which harnessed data from a controlled hypoxia calibration study and from real-life data. The data used to develop the algorithm were completely independent from the datasets used for the prospective clinical validation presented in this work, i.e., the development and validation datasets did not contain data from the same subjects.

FIGURE 2

Figure 2. Back view of the EmbracePlus wristband with the reflectance PPG sensor embedded in the device.

Each value of SpO₂ is estimated on a 10 s rolling window offset by 1 s. A missing SpO₂ value indicates that the algorithm does not have enough confidence to compute an output. A low level of confidence is principally driven by the user not wearing or improperly wearing the device, or by the presence of low-quality raw sensor data (e.g., during motion conditions or low perfusion). Indeed, reflective PPG positioned on the dorsal portion of the wrist has been associated with a lower signal-to-noise ratio compared to the transmissive PPG sensors (26, 48, 49). This effect might be due to both the sensor placement, since only a small portion of the light is reflected and reaches the PD, and to the sensor design accounting for multiple scattering through the skin layers and movement artifact contamination associated with the probe contact pressure (49). In addition, blood perfusion is lower on the back of the wrist as compared to the finger (26). Thus, since SpO₂ measured at the wrist can be prone to lower signal quality and is sensitive to movement contamination, the EmbracePlus utilizes an automated data rejection mechanism (i.e., quality index) based on PPG and ACM to detect a signal quality threshold for which it is possible to estimate trustable SpO₂, discarding inconclusive SpO₂ measurements in the condition of low perfusion, movement, and low signal quality (34, 50).

The algorithm involves signal processing steps in time-domain (e.g., linear filtering, cross-correlation) and frequency-domain (e.g., spectral analysis), and a linear regression model which are compatible with an online implementation where output SpO₂ values are estimated over consecutive 10 s windows offset by 1 s. No “future-time” data point is used to estimate SpO₂ in a given 10 s window.

2.3. Recruitment

The protocol received IRB approval (Laurel Heights Committee—approval number 10-00437; Clinicaltrials.gov registration #NCT04964609) to test the accuracy of the EmbracePlus SpO₂ measurements during mild, moderate, and severe hypoxia. A single-center and interventional clinical study was conducted on 16 healthy participants in a laboratory at the University of California San Francisco between June 2021 and January 2023. Healthy male and female subjects between the ages of 18 and 55 years were recruited, excluding current smokers, women who were pregnant, lactating, or trying to get pregnant, and participants with obesity (body mass index, BMI > 30 kg/m²) or who had an injury, deformity, tattoos, or other physical abnormality at the sensor sites. Exclusion criteria also included participants with serious systemic illnesses, and those who use continuous positive airway pressure, have unacceptable collateral circulation, or any other condition which in the investigators’ opinion would make them unsuitable for the study.

Participants were primarily selected to represent a heterogeneous population in terms of skin tone which was assessed by the Clinical Coordinator at Hypoxia Lab based on the Fitzpatrick scale for skin pigmentation assessment, recruiting 30% of participants with skin tone classified as Fitzpatrick V or VI, i.e., doubling FDA requirements for inclusion of individuals with dark skin (16). Moreover, the recruitment process aimed at including a participant pool with varying ages, BMI, and sex balance. The sample size was selected according to ISO 80601-2-61:2017, which recommends including at least 200 data points from at least 10 subjects (15).

2.4. Study design

The test was conducted in accordance with ISO standards (15) and FDA guidance (16) for SpO₂ testing, which require evaluation against a SaO₂ reference measurement ranging from 70% to 100% during a controlled desaturation protocol.

Each participant was placed in a comfortable semi-recumbent position for approximately 45 min and asked to remain still while breathing a mixture of gas through a mouthpiece while supervised by a medical monitor. Participants’ hands and arms were maintained at ambient room temperature during data collection. Data were collected from an EmbracePlus wristband and two FDA-cleared finger-tip pulse oximeters (Masimo Rad-5 by Masimo Corp and Nellcor N-595 by Nellcor Puritan Bennett Inc.) which were used to monitor the hand perfusion and synchronize the reference SaO₂ data with the EmbracePlus data. All the devices were positioned according to their respective instructions for use. The non-dominant radial artery of each participant was connected to an arterial line (a 22-gauge catheter) to draw blood samples on which the SaO₂ was analyzed by a laboratory multiwavelength co-oximeter (ABL-90 blood gas analyzer, Radiometer Medical ApS) and used as ground truth for the SpO₂ algorithm evaluation. Then, each participant underwent two desaturation runs consisting of a stepwise decrease of the oxygen concentration in the inspired gas mixture, as illustrated in Figure 3.

FIGURE 3

Figure 3. Graphical representation of each desaturation run, consisting of a stepwise decrease of blood oxygen through stable plateaus during which a minimum of two blood samples (red circles) were taken to measure reference SaO₂.

At the beginning of each ramp, a baseline blood sample was collected at room air. Approximately 10 s later, the inspired oxygen was progressively reduced to reach the next SpO₂ plateau level, identified by a stable level of oxygen saturation between the reference finger pulse oximeters. At each target SpO₂ plateau, two to four blood samples were collected approximately 30 s apart, and within 10 s from the conclusion of each plateau, inspired oxygen was progressively changed again to reach the next SpO₂ target level. The target SpO₂ at each run was chosen to allow an even sampling within the [70%–100%] SaO₂ range. Every participant underwent a maximum of 6 plateaus in each ramp before being exposed to a high oxygen saturation level (100% O₂) by breathing oxygen-enriched air for 2 min. The collected blood samples (approximately 20–26 samples per participant) were immediately analyzed with the co-oximeter to measure the reference SaO₂.

2.5. Data handling

Prior to the analysis, periods corresponding to EmbracePlus recording failure (i.e., missing raw data) and out-of-range values from the reference SaO₂ measurement device (i.e., SaO₂< 67%) were identified and removed. EmbracePlus and finger pulse oximeter data were aligned using the pulse rate series estimated on EmbracePlus PPG and the pulse rate series logged by the finger pulse oximeters. The procedure was blind, and it was performed without the knowledge of SaO₂ values. However, since the timestamps of the reference SaO₂ were already synchronized with the finger pulse oximeter data, this procedure allowed automatic alignment with the SpO₂ algorithm outputs and the reference SaO₂.

EmbracePlus continuous SpO₂ data were analyzed to select the values associated with each SaO₂ reference reading. The median SpO₂ value inside a 10 s window within the plateau associated with each blood sample was computed and used to determine the paired value for performance evaluation. The window of SpO₂ values was adjusted to fit a segment with good-quality EmbracePlus data within the selected plateau, to discard the effect of involuntary movements occurring during the procedure (e.g., when the sample was taken).

2.6. Statistical analysis

The primary endpoint of both studies was the accuracy root mean square (A_rms), which is a combination of the systematic and random components of error, computed as the root-mean-square differences between the algorithm output (SpO_2i) and the reference (SaO_2i) (Equation 1), where N is the total number of data points. Data from all the subjects were pooled together for A_rms computation to verify the primary effectiveness endpoint. Moreover, individual A_rms values were computed on each subject's data. Only the data pairs with evaluable values for both the EmbracePlus and the reference device were used.

$A_{\mathrm{rms}}=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^N{(\mathrm{Sp}{\mathrm{O}}_{2i}-\mathrm{Sa}{\mathrm{O}}_{2i})}^2}{N}}}(1)$

According to FDA guidelines (16), a passing result required an A_rms ≤ 3.5% across the fully tested range under no-motion conditions, computed pooling the data points collected from all subjects. This threshold specifies that approximately two-thirds of the device measurements fall within ±3.5% of the reference measurement.

As additional performance measures, the mean bias (i.e., the average of the difference between SpO₂ and SaO₂) and the mean absolute error (MAE) (i.e., the average of the absolute difference between SpO₂ and SaO₂) were computed. A Bland–Altman analysis was performed on all the data points by plotting SaO₂ versus error (SpO₂—SaO₂) with linear regression fit and upper 95% and lower 95% limits of agreement (LoAs) corrected for repeated measurements, indicating the error boundary where approximately 95% of data points fall (51). Additionally, a correlation analysis with linear regression fitting was performed on the pooled data to evaluate the correlation between the EmbracePlus SpO₂ and the reference SaO₂ values. Performance metrics were also evaluated on sex and skin-tone subgroups separately, namely on female and male subjects and on individuals with dark (Fitzpatrick class V and VI) and light skin pigmentation (Fitzpatrick classes I to IV). Furthermore, to investigate possible differences in the SpO₂ estimation error between successive desaturation ramps, a mixed effect model was performed with the subject ID as a random effect and the desaturation ramp ID as fixed effect. An ANOVA was then performed to test the hypothesis that the coefficient representing the fixed-effect term is 0 (F-test with significance level at 0.05).

3. Results

The 16 participants in this study included 8 men and 8 women aged 18–43 years with various skin tones. Table 1 and Table 2 report the demographic summary and listing of the participants, respectively. No undesirable effects or adverse events were reported during the study.

TABLE 1

Table 1. Summary of demographic characteristics of the study participants (n = 16).

TABLE 2

Table 2. Demographic listing of the study participants.

Data from all subjects were included in the analysis, for a total of 398 samples of paired reference SaO₂ and EmbracePlus SpO₂ measurements. The recorded data did not include any missing EmbracePlus data or data affected by sensor issues. Out of the 398 blood samples, 1 sample from subject #6 could not be used for performance computation due to a missing timestamp for the reference SaO₂. Out of the remaining 397 samples, the quality index embedded in the SpO₂ algorithm automatically classified 24 samples collected on one subject (i.e., subject #2) as inconclusive, because of motion or low-quality PPG data, possibly related to the high level of stress experienced by the subject during the desaturation procedure, which was excluded in the performance assessment. Consequently, 373 SpO₂ and reference SaO₂ pairs from 15 subjects were used for the accuracy computation. An example of the EmbracePlus SpO₂ trace during the ramps, superimposed with the SaO₂ samples is reported in Figure 4. The measurements were drawn from a female subject classified as Fitzpatrick VI by skin tone.

FIGURE 4

Figure 4. An example of SpO₂ measurements during the two desaturation ramps: in black, the 1 s EmbracePlus SpO₂ outputs; in red, the SaO₂ samples; in grey, the clinical acceptance boundaries of ±3.5% centered on the EmbracePlus SpO₂ outputs; #bs, number of the blood sample corresponding to each SaO₂ measure.

The reference SaO₂ showed a median value of 86.9% on the analyzed data, ranging from 68.1% to 100%. The paired measurements by the SpO₂ algorithm showed a median value of 87% and ranged from 67% to 100%. The pooled A_rms was 2.4%, the bias 0.05%, the MAE 1.82%, and the upper and lower 95% LoAs were 4.8% and −4.7%, respectively (Table 3). Three levels of oxygen saturation were analyzed (i.e., SaO₂< 80%, 80% ≤ SaO₂< 90%, and SaO₂≥ 90%), on which pooled A_rms of 3.2%, 1.9%, and 2.2% and pooled bias values of 1.6%, 0.3%, and −1.2% were observed, respectively. In each SaO₂ decile, pooled MAE was equal to 2.59%, 1.43%, and 1.68% and upper and lower 95% LoAs were ranging from −4.9% to 7.3% (Table 3). Additionally, Table 4 reports the individual A_rms, bias, and MAE on the full SaO₂ range together with the number of samples for each subject, which ranged from 20 to 26. Thirteen subjects (87%) demonstrated an A_rms lower than 3.5%.

TABLE 3

Table 3. Distribution of the conclusive measurements collected by the EmbracePlus wristband and performance of the SpO₂ algorithm in terms of A_rms, bias and MAE for different SaO₂ ranges.

TABLE 4

Table 4. Performance of the SpO₂ algorithm in terms of A_rms, bias and MAE for each individual study participant.

In Figure 5, the Bland–Altman plot for all the analyzed subjects illustrates the difference between the EmbracePlus SpO₂ and the reference SaO₂ with respect to the reference SaO₂ values. A total of 26 outliers outside the pooled LoAs (i.e., −4.7%; 4.8%) were identified, representing ∼7% of total data points, and are listed in Table 5. Figure 6 reports the regression plot on the pooled data, illustrating a positive, strong correlation (Pearson's correlation coefficient of 0.96) between the SpO₂ algorithm and the reference SaO₂. In addition, Figure 7 shows the distribution of the difference between the EmbracePlus SpO₂ and the reference SaO₂ measurements with a normal density function fitting.

FIGURE 5

Figure 5. Bland–Altman plot for multiple observations with all subjects pooled (N = 373 samples from n = 15 subjects) showing the reference SaO₂ vs. the difference between the EmbracePlus SpO₂ and the reference SaO₂. Data from individual subjects are color coded. Bland–Altman linear regression fit (red bold line), upper and lower limits of agreement (thin red lines), and mean bias (black line) are shown. In the bottom right corner, the number of subjects (n), data points (N), and linear fit equation are reported.

TABLE 5

Table 5. List of SpO₂ samples recognized as outliers.

FIGURE 6

Figure 6. Regression plot for comparing the EmbracePlus SpO₂ values against the reference SaO₂ values (N = 373 samples from n = 15 subjects). Single subject reference SaO₂ data are plotted against the SpO₂, with a superimposed bisect line (dashed black line) and a linear regression line (solid red line). In the legend, the slope and the intercept, Pearson's correlation coefficient (rho), its P value, and the R2 value of the linear fitting are reported.

FIGURE 7

Figure 7. The distribution of the difference between the EmbracePlus SpO₂ and the reference SaO₂ (grey) with a normal density function fit (dashed black line).

Table 6 reports the SpO₂ algorithm performance considering skin pigmentation and gender subgroups, as listed in the Statistical Analysis section. Each subgroup analyzed meets or exceeds the FDA-recommended clinical threshold (A_rms equal or lower than 3.5%). Moreover, comparable LoAs and error metrics were observed between skin and gender subgroups.

TABLE 6

Table 6. Accuracy metrics of the SpO₂ algorithm in terms of A_rms, bias, MAE, upper 95% LoA, and lower 95% LoA as described in the statistical analysis section.

Finally, a non-significant effect of the desaturation ramp on the estimation errors was determined (p > 0.05).

4. Discussion

4.1. Principal results

Wrist-worn wearable devices are increasingly used to monitor SpO₂ and contribute to promoting home-centered healthcare and decentralized clinical trials (11, 18, 26, 32, 35, 52), with the potential to significantly improve clinical outcomes by supporting prompt diagnosis and early detection of clinical deterioration in patients with respiratory diseases such as COPD, OSA, and COVID-19 (9–12). However, commercially available wearable devices using PPG to measure SpO₂ are often sold as consumer products rather than medical devices, thereby reducing requirements for evidence of clinical performance and independent review by relevant regulatory authorities (11, 19, 52, 53). Even when cleared as medical devices, concerns have recently emerged about racial bias impacting the performance of pulse oximeters in individuals with darker skin pigmentation (37, 38), highlighting the need for additional data evaluating the reliability of wearable PPG technology.

In this work, we examined the validity of the EmbracePlus wristband reflective pulse oximeter in monitoring SpO₂ during a controlled hypoxia study. The clinical study was conducted in accordance with the ISO 80601-2-61:2017 standard (15) and FDA guidance (16). Data from sixteen healthy adults were analyzed during a standardized desaturation protocol, and performances of the SpO₂ EmbracePlus measurements were compared with the gold standard SaO₂. In this study, we report performance on 373 paired samples from 15 participants, exceeding FDA guidelines on validation of pulse oximeters, which recommend a minimum of 70 data points (i.e., 200/3) in each tested SaO₂ decile (16). Our dataset included 142, 138, and 93 data points in SaO₂≥ 90%, 80% ≤ SaO₂< 90%, and SaO₂< 80%, respectively. However, given the limited stability of induced desaturations below 80%, the distribution of the collected data is skewed towards values greater than 80%. The A_rms accuracy between the estimated SpO₂ and the reference SaO₂ did meet the passing criterion of A_rms under 3.5%, reaching a value of 2.4%, indicating that approximately 2/3 of the SpO₂ values were within ±2.4% of the reference SaO₂. The effectiveness of the SpO₂ algorithm was further supported by low bias (<0.5%) and MAE (<2%) and a significantly strong correlation with the reference SaO₂ (Pearson's correlation coefficient >0.95, P < 0.001), suggesting the possibility to track chronic reductions of the average SpO₂ over time, which could be suggestive of an underlying issue.

Recently, a positive bias and a larger error for SpO₂ measurements at low blood oxygen saturations in darkly pigmented subjects have been reported by manufacturers of several different pulse oximeters (36, 41), raising concern for potential racial bias (37, 38). Given these concerns, this study enrolled five (i.e., 5/16, ∼31%) subjects with skin pigmentation classified as Fitzpatrick scale V or VI, doubling the FDA requirements of having at least 2 subjects or 15% of the participants (whichever is greater) with dark skin tone (16). The results highlight a global error of 0.1% and 0.04% when considering dark and light skin tones, respectively, with a slightly larger positive bias for darkly pigmented participants but comparable LoAs. Each subgroup analyzed, moreover, was found to have an A_rms below 3.5% (i.e., Fitzpatrick V–VI, Fitzpatrick I–IV, males, females), indicating acceptable thresholds for accuracy.

An analysis of outliers found a limited number of data points falling outside the pooled Limits of Agreements (i.e., −4.7%; 4.8%), representing <7% of the total. In a risk analysis, the clinical impact of these outliers was determined to be marginal, as no data points fell within a window of occult hypoxemia, defined as a SaO₂ of <88% with a measured SpO₂ of 92%–96% (36, 54). For the majority of outliers (17/26, ∼65%), the wearable device produced SpO₂ overestimates of SaO₂ for SaO₂ values ≤84.1%. In each instance, the degree of SpO₂ overestimation would not change the clinical interpretation of the subject as being critically hypoxic. Nearly all remaining outliers (7, i.e., ∼27%) involved underestimating SaO₂ values over 95%. These underestimated values remained within the highest SpO₂ decile, and the bias towards underestimation reduces the risk of occult hypoxemia. Finally, in two cases (i.e., 8%), an outlier was recorded at the upper extreme of the lowest decile (SaO₂ ≈ 80%) involving underestimation of SaO₂ within the lowest decile. The degree of SpO₂ underestimation in these instances would not change the clinical interpretation of critical hypoxia but may bias clinicians toward earlier triage and evaluation. As the device is intended for retrospective data review without alarms or reliance on output for real-time clinical decision-making, the listed outliers do not raise new questions of safety or effectiveness.

The presented results contributed to receiving FDA clearance for the EmbracePlus and its monitoring platform to be used by trained healthcare professionals or researchers to remotely monitor physiological parameters in ambulatory individuals 18 years of age and older in home-healthcare environments (40). The Empatica EmbracePlus wristband is one of the few wearable devices that received 510(k) marketing authorization from the US FDA for SpO₂ monitoring (21, 22, 27, 28). An analysis of performance data for FDA-cleared wrist-worn SpO₂ monitoring devices showed comparable levels of overall accuracy, but significant limitations in publicly available data. The Oxitone 1000 M was validated on data recorded during a desaturation protocol similar to that used to test the EmbracePlus algorithm, and the authors reported an A_rms of 1.9% (27, 35). Given fewer total data points included in the analysis (240 samples from 10 subjects) and a paucity of published data on individuals with darker skin pigmentation (18, 27, 35), the impact of racial bias on performance for this device remains unclear. Similarly, a recent study on 14 subjects wearing the ScanWatch during a similar desaturation protocol of this study showed an accuracy A_rms of 2.97% (right wrist) and 3% (left wrist) on a comparable population but did not include subgroup analyses evaluating the impact of sex or skin tone on performance (26, 28). Finally, Biobeat Technologies Ltd reports an A_rms of 2% for their wearable devices; however, to the best of our knowledge, no complete information about the population, number of samples, or performance by subgroups is available (22).

4.2. Limitations and future work

The validation presented in this work advances recent efforts from research and public health communities to increase transparency and understanding of the limitations of pulse oximetry. Recommendations that have been put forth to date include publishing subgroup analyses, providing justification for outliers, and increasing the number of data samples analyzed beyond the FDA-suggested sample size (41). Nevertheless, the results of this study must be interpreted in the context of several limitations.

During the hypoxia study, data were collected in a controlled environment with a standardized desaturation protocol to maintain SpO₂ levels as stable as possible. While the controlled study design fulfills the ISO and FDA guidelines and allows for better assessment of the impact of certain confounders (e.g., demographic variables) on performance through careful experimental control of other covariates, the results do not support analysis of the generalizability of the EmbracePlus wristband to monitor SpO₂ in real-world conditions, where SpO₂ exhibits dynamic changes over time.

Additionally, the study pool was limited to healthy subjects aged 18–43 years, due to higher risks induced by hypoxia in older and/or unhealthy patients. There are an increasing number of investigations evaluating the performance of wrist-worn devices in uncontrolled studies and/or on patients with cardiovascular and lung diseases, yet these are mostly performed with consumer smartwatches thanks to their wider availability and lower costs (11, 19, 32, 52). As there are not rigorous evaluation requirements for SpO₂ computation in these consumer products, and they do not go through independent regulatory review, however, it is unclear to what degree clinical study results using these products can be relied upon to understand the impact of race, comorbidities, or real-world conditions on medical device performance (25).

Following promising preliminary data collected during a separate, ambulatory, uncontrolled desaturations in darkly pigmented adults (Figure 8), future work will focus on extensively testing the EmbracePlus wristband in real-life conditions (e.g., sleep), in diverse age ranges, and on specific pathological conditions (e.g., lung diseases like COPD and obstructive sleep apnea). Future investigations may assess the feasibility of using the SpO₂ data not only to detect hypoxemia but also to assess cardiopulmonary function [e.g., track SpO₂ recovery after exercise (55)] or for stress monitoring (56).

FIGURE 8

Figure 8. Examples of SpO₂ measurements during hyperventilation-induced desaturations: in red, the 1 s EmbracePlus SpO₂ outputs; in black, the finger pulse oximeters samples; in grey, the clinical acceptance boundaries for finger pulse oximeter of ±2%. (A) 52 years male subject with Fitzpatrick V skin pigmentation; (B) 48 years female subject with Fitzpatrick VI skin pigmentation.

5. Conclusions

To conclude, the study results demonstrate that the SpO₂ measurements performed by the non-invasive EmbracePlus wristband show high clinical accuracy between 70%–100% SpO₂ in the intended use conditions of no-motion and high-perfusion, across individuals with a range of skin pigmentation. This study contributes to the current state of scientific knowledge on the impact of racial bias on SpO₂ measurement and paves the way for further validation during prolonged use in uncontrolled settings and on patients at risk of hypoxemia.

Data availability statement

The datasets presented in this article are not readily available because the sensor data recorded on the patients are proprietary of Empatica Inc. The authors can share the data on arterial oxygen saturation and the output of the SpO₂ algorithm for each patient upon request to the corresponding author. Requests to access the datasets should be directed to gg@empatica.com.

Ethics statement

The studies involving humans were approved by Laurel Heights Committee—approval number 10-00437. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

GG: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Project administration, Software, Supervision, Writing – review & editing. GC: Data curation, Formal Analysis, Visualization, Writing – original draft. WC: Methodology, Writing – review & editing. JL: Methodology, Writing – review & editing. MM: Methodology, Supervision, Writing – review & editing. RP: Formal Analysis, Methodology, Supervision, Writing – review & editing. MC: Formal Analysis, Methodology, Supervision, Writing – review & editing. GR: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Project administration, Software, Supervision, Writing – review & editing.

Funding

The author(s) declare that no external financial support was received for the research, authorship, and/or publication of this article from agencies in the public, commercial, or non-for-profit sectors.

Acknowledgments

We are grateful to the Designers’ team at Empatica Inc. for providing support in finalizing the graphical material presented in the paper.

Conflict of interest

All authors disclose being share-holders of Empatica Inc. and having received a salary or consulting fees from Empatica Inc., which manufactured the EmbracePlus wristband validated in this work. Moreover, RP is a co-founder and chairman of the board of Empatica Inc.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Labaki WW, Han MK. Chronic respiratory diseases: a global view. Lancet Respir Med. (2020) 8(6):531–3. doi: 10.1016/S2213-2600(20)30157-0

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Chronic respiratory diseases. Available at: https://www.who.int/health-topics/chronic-respiratory-diseases (Accessed July 2023).

3. Chronic Obstructive Pulmonary Disease (COPD). Available at: https://www.who.int/news-room/fact-sheets/detail/chronic-obstructive-pulmonary-disease-(copd) (Accessed July 2023).

4. Asthma. Available at: https://www.who.int/news-room/fact-sheets/detail/asthma (Accessed July 2023).

5. Hughes KM, Price D, Torriero AAJ, Symonds MRE, Suphioglu C. Impact of fungal spores on asthma prevalence and hospitalization. Int J Mol Sci. (2022) 23(8):4313. doi: 10.3390/ijms23084313

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Santilli M, Manciocchi E, D’Addazio G, Di Maria E, D’Attilio M, Femminella B, et al. Prevalence of obstructive sleep apnea syndrome: a single-center retrospective study. Int J Environ Res Public Health. (2021) 18(19):10277. doi: 10.3390/ijerph181910277

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Chen C, Haupert SR, Zimmermann L, Shi X, Fritsche LG, Mukherjee B. Global prevalence of post-coronavirus disease 2019 (COVID-19) condition or long COVID: a meta-analysis and systematic review. J Infect Dis. (2022) 226(9):1593–607. doi: 10.1093/infdis/jiac136

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Allado E, Poussel M, Valentin S, Kimmoun A, Levy B, Nguyen DT, et al. The fundamentals of respiratory physiology to manage the COVID-19 pandemic: an overview. Front Physiol. (2021) 11:615690. doi: 10.3389/fphys.2020.615690

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Henry NR, Hanson AC, Schulte PJ, Warner NS, Manento MN, Weister TJ, et al. Disparities in hypoxemia detection by pulse oximetry across self-identified racial groups and associations with clinical outcomes. Crit Care Med. (2022) 50(2):204–11. doi: 10.1097/CCM.0000000000005394

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Samuel J, Franklin C. Hypoxemia and hypoxia. In: Myers JA, Millikan KW, Saclarides TJ, editors. Common surgical diseases. New York, NY: Springer New York (2008). p. 391–4. doi: 10.1007/978-0-387-75246-4_97

11. Pipek LZ, Nascimento RFV, Acencio MMP, Teixeira LR. Comparison of SpO₂ and heart rate values on apple watch and conventional commercial oximeters devices in patients with lung disease. Sci Rep. (2021) 11(1):18901. doi: 10.1038/s41598-021-98453-3

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Hafen BB, Sharma S. Oxygen saturation. Statpearls. Treasure Island (FL): StatPearls Publishing (2022). Available at: https://www.ncbi.nlm.nih.gov/books/NBK525974/ (Accessed Februrary 15, 2023)

13. Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiol Meas. (2007) 28(3):R1–39. doi: 10.1088/0967-3334/28/3/R01

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Park J, Seok HS, Kim SS, Shin H. Photoplethysmogram analysis and applications: an integrative review. Front Physiol. (2022) 12:808451. doi: 10.3389/fphys.2021.808451

PubMed Abstract | CrossRef Full Text | Google Scholar

15. International standard. Particular requirements for basic safety and essential performance of pulse oximeter equipment. ISO 80601-2-61:2017. (2017). Available at: https://www.iso.org/standard/67963.html (Accessed July 2023).

16. FDA. Pulse-oximeters—premarket-notification submissions [510(k)s]—guidance for industry and food and drug-administration staff. (2013). Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/pulse-oximeters-premarket-notification-submissions-510ks-guidance-industry-and-food-and-drug (Accessed July 2023).

17. Downey CL, Chapman S, Randell R, Brown JM, Jayne DG. The impact of continuous versus intermittent vital signs monitoring in hospitals: a systematic review and narrative synthesis. Int J Nurs Stud. (2018) 84:19–27. doi: 10.1016/j.ijnurstu.2018.04.013

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Guber A, Epstein Shochet G, Kohn S, Shitrit D. Wrist-sensor pulse oximeter enables prolonged patient monitoring in chronic lung diseases. J Med Syst. (2019) 43(7):230. doi: 10.1007/s10916-019-1317-2

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Haveman ME, van Rossum MC, Vaseur RME, van der Riet C, Schuurmann RCL, Hermens HJ, et al. Continuous monitoring of vital signs with wearable sensors during daily life activities: validation study. JMIR Form Res. (2022) 6(1):e30863. doi: 10.2196/30863

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Castaneda D, Esparza A, Ghamari M, Soltanpur C, Nazeran H. A review on wearable photoplethysmography sensors and their potential future applications in health care. Int J Biosens Bioelectron. (2018) 4(4):195–202. doi: 10.15406/ijbsbe.2018.04.00125

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Current health monitoring system gen 2 (G2), FDA clearance (K210133)). (2021). Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K210133 (Accessed July 2023).

22. Biobeat platform2, BB-613 WP patch, FDA clearance (K222010)). Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K222010 (Accessed July 2023).

23. Masimo U.S. E1 ear sensor. Available at: https://www.masimo.com/siteassets/us/documents/pdf/plm-11219c_product_information_e1_ear_sensor_us.pdf (Accessed July 2023).

24. Hermand E, Coll C, Richalet JP, Lhuissier FJ. Accuracy and reliability of pulse O₂ saturation measured by a wrist-worn oximeter. Int J Sports Med. (2021) 42(14):1268–73. doi: 10.1055/a-1337-2790

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Windisch P, Schröder C, Förster R, Cihoric N, Zwahlen DR. Accuracy of the apple watch oxygen saturation measurement in adults: a systematic review. Cureus. (2023) 15(2):e35355. doi: 10.7759/cureus.35355

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kirszenblat R, Edouard P. Validation of the withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study. J Med Internet Res. (2021) 23(4):e27503. doi: 10.2196/27503

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Oxitone 1000M, FDA clearance (K163382). (2017). Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K163382 (Accessed July 2023).

28. Scan monitor, FDA clearance (K201456). (2021). Available at: https://www.accessdata.fda.gov/cdrh_docs/pdf20/K201456.pdf (Accessed July 2023).

29. Loop system, FDA clearance (K181352). (2019). Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K181352 (Accessed July 2023).

30. Polso watch, chronisense medical, FDA clearance (K220351). Available at: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm?ID=K220351 (Accessed July 2023).

31. Biobeat platform, BB-613 WP, FDA clearance (K190792). Available at: https://www.accessdata.fda.gov/cdrh_docs/pdf19/K190792.pdf (Accessed July 2023).

32. Alharbi S, Hu S, Mulvaney D, Barrett L, Yan L, Blanos P, et al. Oxygen saturation measurements from green and orange illuminations of multi-wavelength optoelectronic patch sensors. Sensors. (2019) 19(1):118. doi: 10.3390/s19010118

CrossRef Full Text | Google Scholar

33. Tamura T, Maeda Y, Sekine M, Yoshida M. Wearable photoplethysmographic sensors—past and present. Electronics (Basel). (2014) 3(2):282–302. doi: 10.3390/electronics3020282

CrossRef Full Text | Google Scholar

34. Jarchi D, Salvi D, Velardo C, Mahdi A, Tarassenko L, Clifton DA. Estimation of HRV and SpO₂ from wrist-worn commercial sensors for clinical settings. 2018 IEEE 15th international conference on wearable and implantable body sensor networks (BSN); Las Vegas, NV, USA: IEEE (2018). p. 144–7. Available at: http://ieeexplore.ieee.org/document/8329679/ (Accessed January 14, 2019).

35. Final Test Report For Oxitone Medical, Ltd. Results of the SpO2 Accuracy Validation of Oxitone 1000 Pulse Oximeter via Reference CO-Oximetry. Study ID# PR 2016-192. (2016). Report No.: TR# 2016-192. Available at: https://www.oxitone.com/wp-content/uploads/2020/01/TR2016-192-Oxitone-1000-Pulse-Oximeter-Accuracy-Study-Final-Test-Report-081816-signed.pdf (Accessed July 2023).

36. Sjoding MW, Dickson RP, Iwashyna TJ, Gay SE, Valley TS. Racial bias in pulse oximetry measurement. N Engl J Med. (2020) 383:2477–8. doi: 10.1056/NEJMc2029240

PubMed Abstract | CrossRef Full Text | Google Scholar

37. FDA Executive Summary. Review of pulse oximeters and factors that can impact their accuracy. Maryland: Anesthesiology Devices Advisory Committee Center for Devices and Radiological Health (CDRH) United States Food and Drug Administration (2022). Available at: https://www.fda.gov/media/162709/download (Accessed July 2023).

38. Commissioner O of the. FDA In Brief: FDA warns about limitations and accuracy of pulse oximeters. Maryland: FDA (2021). Available at: https://www.fda.gov/news-events/fda-brief/fda-brief-fda-warns-about-limitations-and-accuracy-pulse-oximeters (Accessed July 2023).

39. Keller MD, Harrison-Smith B, Patil C, Arefin MS. Skin colour affects the accuracy of medical oxygen sensors. Nature. (2022) 610(7932):449–51. doi: 10.1038/d41586-022-03161-1

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Empatica health monitoring platform. Report No.: 510(k) Number K221282. (2022). Available at: https://www.accessdata.fda.gov/cdrh_docs/pdf22/K221282.pdf (Accessed July 2023).

41. Okunlola OE, Lipnick MS, Batchelder PB, Bernstein M, Feiner JR, Bickler PE. Pulse oximeter performance, racial inequity, and the work ahead. Respir Care. (2022) 67(2):252–7. doi: 10.4187/respcare.09795

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jubran A. Pulse oximetry. Crit Care. (2015) 19(1):1–7. doi: 10.1186/s13054-015-0984-8

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Kennedy SM. University of Wisconsin Madison - ECE/BME 462 - Biomedical Instrumentation' Available at: https://studylib.net/doc/7274733/an-introduction-to-pulse-oximeters--equations-and-theory

44. Chan ED, Chan MM, Chan MM. Pulse oximetry: understanding its basic principles facilitates appreciation of its limitations. Respir Med. (2013) 107(6):789–99. doi: 10.1016/j.rmed.2013.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Aoyagi T. Pulse oximetry: its invention, theory, and future. J Anesth. (2003) 17(4):259–66. doi: 10.1007/s00540-003-0192-6

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Moyle JT. Pulse oximetry. 2nd ed. London: BMJ Books (2002).

47. Nitzan M, Romem A, Koppel R. Pulse oximetry: fundamentals and technology update. Med Devices Evid Res. (2014) 7:231. doi: 10.2147/MDER.S47319

CrossRef Full Text | Google Scholar

48. Rincon F, Pidoux J, Murali S, Goy JJ. Performance of the new SmartCardia wireless, wearable oximeter: a comparison with arterial SaO₂ in healthy volunteers. BMC Anesthesiol. (2022) 22:77. doi: 10.1186/s12871-022-01604-w

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Tamura T. Current progress of photoplethysmography and SPO₂ for health monitoring. Biomed Eng Lett. (2019) 9(1):21–36. doi: 10.1007/s13534-019-00097-w

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Clarke G. Signal quality analysis in pulse oximetry: Modelling and detection of motion artifact. [master’s thesis]. Ottawa, ON: Carleton University (2015). Available at: https://curve.carleton.ca/a26de6c9-79f0-4654-b56e-bd334327d1a7 (Accessed April 4, 2020).

51. Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. (2007) 17(4):571–82. doi: 10.1080/10543400701329422

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Schiefer LM, Treff G, Treff F, Schmidt P, Schäfer L, Niebauer J, et al. Validity of peripheral oxygen saturation measurements with the garmin fēnix® 5X plus wearable device at 4559m. Sensors. (2021) 21(19):6363. doi: 10.3390/s21196363

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Fitbit SpO₂ user manual. (2022). Available at: https://help.fitbit.com/manuals/manual_spo2_en_US.pdf (Accessed July 2023).

54. Barker SJ, Wilson WC. Racial effects on masimo pulse oximetry: a laboratory study. J Clin Monit Comput. (2022) 37:567–74. doi: 10.1007/s10877-022-00927-w

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Inagaki T, Terada J, Yahaba M, Kawata N, Jujo T, Nagashima K, et al. Heart rate and oxygen saturation change patterns during 6-min walk test in subjects with chronic thromboembolic pulmonary hypertension. Respir Care. (2018) 63(5):573–83. doi: 10.4187/respcare.05788

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Sevil M, Rashid M, Hajizadeh I, Askari MR, Hobbs N, Brandt R, et al. Discrimination of simultaneous psychological and physical stressors using wristband biosignals. Comput Methods Programs Biomed. (2021) 199:105898. doi: 10.1016/j.cmpb.2020.105898

PubMed Abstract | CrossRef Full Text | Google Scholar