Remember me
Vital signs measured in clinical practice include heart rate, breathing rate, blood pressure, and body temperature. These serve as objective measurements of normal physiological functions and play a fundamental role in the assessment of health [,]. With aging, there is an increased incidence of functional limitations and chronic conditions, including hypertension, coronary heart disease, stroke, type 2 diabetes, and sleep apnea. More than 65% of adults aged ≥65 years report multimorbidity. The presence of uncontrolled comorbidities reduces the quality of life; leads to loss of independence; and increases the incidence of falls, hospitalization, and mortality [-].
Standardized, continuous vital signs monitoring systems, when implemented for at-home monitoring and care of older adults, including people living with dementia, can serve as an important tool for early identification of changes in health, improve care in older people, and reduce the burden on the health care system [-]. Commercially available wearable devices (wearables) and contactless technologies (nearables) are increasingly used for home monitoring and have the potential to enable remote health monitoring and promote independent living [-]. These technologies offer secure digital infrastructure that allows reliable and seamless transfer of collected data to cloud servers and can facilitate long-term remote monitoring opportunities for health care.
Wearables are widely used for continuous, at-home monitoring of heart rate, and some have been evaluated in-laboratory settings, predominantly in younger age groups [-]. Although several wearables have been shown to be acceptable for older adults, lower technology adoption rate; user comfort trade-off; and burden of maintenance (eg, removal during some daily activities such as showers, periodic recharging, and mobile app use) may make them unsuitable for long-term use in people with cognitive impairment due to their associated behavioral and psychological symptoms [,].
Contactless technologies can be embedded in the living environment such as under the bed mattress (undermattress devices or bed sensors) or on the bedside table (eg, bedside radars) and allow contactless monitoring when the user is in bed []. They are powered by the mains and securely stream the collected data wirelessly. They use several contactless sensing modalities to measure a composite signal (ballistographic signal) containing movements resulting from breathing and cardiac activity to extract vital signs (heart rate and breathing rate) information. The bedside radars use the Doppler radar technique, while the undermattress devices use several technologies, such as electromechanical films and pneumatic sensors [,]. Due to their inconspicuous nature and low maintenance, they do not pose any of the burdens imposed by wearables and are an ideal tool for continuous monitoring of vital signs, behavioral information, and sleep in community-dwelling older adult populations with long-term conditions and in people living with dementia [,,,].
ObjectiveTo realize the potential of contactless technologies for monitoring vital signs such as heart rate and breathing rate in the community, the validity of their measurements needs to be evaluated in relevant populations. While the validity of the heart rate and breathing rate estimates collected from a few contactless technologies has been evaluated in younger populations, to the best of our knowledge, there are no vital signs evaluation studies in older adults (aged >65 years) although these devices have been implemented in longitudinal studies [,,-]. Here, we evaluated the validity of heart rate and breathing rate measurements collected from 3 contactless technologies (a bedside radar and 2 undermattress devices) against polysomnography (PSG) electrocardiogram (ECG)-derived heart rate and respiratory inductance plethysmography thorax (RIP thorax)-derived breathing rate in a laboratory setting. Throughout this paper, we have used the term vital signs to denote heart rate or breathing rate and vice versa. The evaluation addresses aspects of overnight average estimates; the ability to capture overnight trends; variability in heart rate and breathing rate; and accuracy at different sleep stages and time resolutions (60-, 10-, and 1-minute intervals) of estimates. We also discuss the data collection reliability in a home environment and summary estimates of breathing disturbance from the devices. To enhance the relevance of this study, we applied liberal inclusion and exclusion criteria for the participant selection, such that several participants had comorbidities that were representative of the general older population.
The study data were collected at home for a period of 7 to 14 days, followed by an overnight laboratory session (with full PSG) at the Surrey Sleep Research Centre in 2 cohorts (cohort 1: n=18, from January to March 2020; cohort 2: n=17, from June to November 2021). The participant group consisted of 35 individuals (men: n=21, 60%) aged between 65 and 83 (mean 70.8 SD 4.9) years. The participants were identified and recruited through the Surrey Clinical Research Facility. All participants attended an in-person screening visit, during which a range of assessments were completed. These included medical history (self-reported), the Pittsburgh Sleep Quality Index, the International Consultation on Incontinence Questionnaire-Urinary Incontinence, intermediate activities of daily living questionnaire, and the Epworth Sleepiness Scale. The vital signs (heart rate, breathing rate, blood pressure, and temperature) were also collected and reviewed to determine inclusion in the study. To ensure the ecological validity of the collected data in this population, participants with stable comorbidities such as hypertension, type 2 diabetes, arthritis, and so on, were included in the study if their comorbidity and concomitant medications were stable and were not considered to pose a safety risk. Eligible participants had to be able to independently perform activities of daily life, as assessed by the intermediate activities of daily living questionnaire, and comply with study procedures. Participants were provided with detailed information about the study and provided written informed consent before any study procedures were performed. A detailed description of the inclusion and exclusion criteria can be found in our previous publications, in which we evaluated the ability of these technologies to estimate sleep timing in the home environment and sleep stages as defined by PSG [,].
Ethical ConsiderationsThe study received a favorable opinion from the University of Surrey Ethics Committee (UEC-2019-065-FHMS) and was conducted in line with the Declaration of Helsinki and the principles of Good Clinical Practice. Potential participants were given detailed information about the study protocol, and they provided written informed consent before any study procedures were performed. Complete details of the study, along with the data management, privacy, and confidentiality measures, are discussed in the protocol [].
Study ProtocolThe undermattress devices (Withings Sleep Analyzer [WSA]; Withings] and Emfit QS; Emfit Ltd) were deployed both in the laboratory and at home, while the bedside radar (Somnofy; VitalThings) was only used in the laboratory. The participants also used an actigraphy device (Actiwatch Spectrum [AWS]; Philips Respironics) and maintained a consensus sleep diary at home []. During the home deployment period, the contactless technologies did not require any manual intervention or maintenance by the participants, and the data was transmitted automatically via Wi-Fi. To ensure the anonymity of the data, no personally identifiable participant information was added to the device applications, and a portable Wi-Fi router was used for data transfer. The contactless devices remained active and collected data continuously for the entire period of the study. The participants wore AWS on their nondominant hands throughout the study period, removing them briefly in scenarios that could lead to the device becoming wet. AWS data have been reported elsewhere [].
After the home data collection period, 1 overnight full clinical PSG recording was conducted, which included an extended time in bed of 10 hours. On average, the participants slept 385.97 (SD 65.67) minutes in the sleep laboratory as measured via PSG and 405.96 (SD 84.00) minutes as measured via consensus sleep diary at home [,]. The WSA was used in both cohort 1 and cohort 2, while the Somnofy and Emfit were deployed only in cohort 2. The data from the contactless technologies were collected simultaneously along with PSG in the laboratory and with AWS and sleep diary at home. Empatica E4 (Empatica Srl), a wrist-worn device that collects activity and photoplethysmography, was also deployed during the laboratory session, and our evaluation of this device is reported elsewhere []. All devices and PSG data were collected for the entire 10-hour in-bed period in the laboratory. A detailed description of the study protocol is given in the protocol [].
The Reference Vital Signs DataDuring the in-laboratory session, PSG data were collected using the SomnoHD system (SOMNOmedics GmbH). The collected data included electroencephalography (256 Hz; F3-M2, C3-M2, O1-M2, F4-M1, C4-M1, and O2-M1); ECG (modified lead II subclavicular electrode placement; 256 Hz); RIP thorax and abdomen (128 Hz); photoplethysmography (128 Hz); electromyography (256 Hz, both submental and limb); and electrooculography (256 Hz; E2-M1 and E1-M2). In addition, data on the snore sensor (256 Hz) and airflow via nasal cannula and flow thermistor (128 Hz) were also collected. Sleep was scored at 30-second intervals in the Domino software environment as per American Academy of Sleep Medicine (AASM) guidelines by 2 independent scorers (a registered polysomnographic technologist and a trained scorer), and a consensus hypnogram was generated []. The sleep hypnogram contains 5 stages: rapid eye movement (REM), stage 1 of non-REM sleep (N1), stage 2 of non-REM sleep (N2), stage 3 of non-REM sleep (N3), and wake. The apnea-hypopnea index (AHI; number of apnea/hypopnea events per hour) and period limb movement index (PLMI; number of period limb movement events per hour) were determined by the Registered Polysomnographic Technologist using scoring rules recommended by AASM. An apnea was scored when there was a ≥90% drop in airflow lasting for at least 10 seconds, while a hypopnea was scored using the 3% drop in oxygen saturation and an arousal in the electroencephalogram criteria. The severity of apnea was determined using the following thresholds as per AASM guidelines: AHI score of <5 (normal), 5 to <15 (mild apnea), 15 to <30 (moderate apnea), and ≥30 (severe). A periodic limb movement event was scored when at least 4 consecutive limb movements occurred, each separated from the preceding limb movement by at least 5 seconds but not more than 90 seconds apart. PLMI of >15 seconds is used as the cutoff for the presence of periodic limb movement disorder [,]. In addition, participants with cardiac arrhythmia were identified using the arrhythmia index generated by the Domino software and verified by visual inspection of the record.
The PSG data were exported as standard EDF+ files along with recording markers and the consensus hypnogram. The ECG from the PSG was used for the extraction of the heart rate reference data, while the RIP thorax was used as the breathing rate reference data. For one of the participants in whom RIP thorax was unavailable, RIP abdomen was used to create the breathing rate reference data. MATLAB 2021b was used for all data analyses reported. The RR intervals used for the computation of the heart rate were derived from the ECG using the PhysioNet cardiovascular signal toolbox and a well-evaluated beat detection toolbox [,]. This beat-to-beat information was used to estimate reference heart rate (beats per minute [bpm]) at 30-second intervals, which is the same as the PSG hypnogram. For extracting the breathing rate (cycles per minute [cpm]) from the RIP thorax signal at 30-second intervals, the RRest package was used [,].
Contactless Technologies: Data OverviewThe WSA and Somnofy data (json format) were downloaded using the respective manufacturer’s application programming interface, while the Emfit data (CSV format) were downloaded from the manufacturer’s web interface. All the compared contactless technologies (WSA, Emfit, and Somnofy) provided breathing rate data, while only the undermattress devices (WSA and Emfit) provided heart rate data.
The devices provided vital signs (heart rate and breathing rate) data and sleep hypnograms at different resolutions (WSA: 60 seconds; Emfit and Somnofy: 30 seconds). The WSA and Somnofy heart rate and breathing rate estimations were available at 60 and 30-second resolutions (same as the respective device hypnogram resolution), while the Emfit estimated heart rate at a 4-second resolution. These 4-second estimates were averaged to generate estimates at 30-second intervals to match the Emfit sleep label intervals. To allow data analysis relative to local time, daylight savings correction was applied to the Coordinated Universal Time timeseries generated by the devices. The sleep hypnograms generated by the devices contain 4 stages: deep sleep (DS=N3), light sleep (LS=N2/N1), REM, and wake.
Apart from heart rate and breathing rate, Emfit generates continuous heart rate variability and activity measures, while Somnofy provides the estimates of movement and environmental variables such as ambient light, sound, temperature, pressure, humidity, and indoor air quality, which were out of scope for this evaluation and are not discussed here.
Both at home and in the laboratory, all devices were connected to the same network, and the devices used the manufacturer’s time synchronization protocol such as network time protocol to timestamp the data. Although this ensured that the devices were synchronized to local time, we performed another synchronization step to allow an accurate comparison of the data between the devices. The device vital signs measures were aligned to the PSG reference vital signs estimates via cross-correlation between the device and PSG vital signs and hypnograms, and the lag (within a 5-minute window) that provided the best alignment of both the vital signs data and hypnograms was then applied. The WSA data were converted from 60- to 30-second intervals by upsampling. Epochs in the PSG and device hypnograms that were scored as artifacts or no presence were excluded from the assessment.
Vital Signs AssessmentOverviewThe evaluation of the epoch-by-epoch heart rate and breathing rate data collected by the contactless technologies was performed against reference estimates derived from PSG ECG and RIP thorax. The accuracy and reliability of the heart rate and breathing rate estimates were performed at different levels of time resolution to determine use cases in which the contactless technologies can be used. These include accuracy assessment of overnight average estimates; ability to capture overnight trends; variability in vital signs; and accuracy in different sleep stages and at different time resolutions (60-, 10-, and 1-minute intervals) of estimates. All laboratory data analyses were performed over the total recording period of the PSG. At all temporal resolutions of comparisons, only complete or valid pairs of estimates were used.
Performance MeasuresTo assess the accuracy of the vital signs estimates (heart and breathing rates), mean absolute error (MAE) and mean absolute percentage error (MAPE) were used as the primary metrics. MAE and MAPE are used to measure the error in the estimate between the device and the PSG reference vital signs. Bland-Altman metrics such as minimum detectable change (MDC), bias, and limits of agreement (LoA) were also computed to provide an overview of the agreement of the estimates and to allow comparison with the evaluations reported in the literature [,]. All the measures are reported with 95% CIs. MDC is the smallest change in the estimate that can be detected by the device that exceeds the measurement error. It is equal to half the agreement width [],
MDC = (LoAUpper bound – LoALower bound) / 2 (1)We used intraclass correlation (ICC) with 2-way random effects to measure the reliability of measurement and standardized absolute difference, a directionless Cohen d, described by Guruswamy Ravindran et al [] and Haghayegh et al [], for measuring the dispersion in the bias. All ICC values (range 0-1) were estimated with α of .05; ranges used for interpretation were as follows: ICC<0.5 (poor), ICC=0.5 to 0.75 (moderate), and ICC>0.75 (good reliability). Apart from the above metrics, the coefficient of determination (r2, a measure of how close the measured estimates are to the reference, computed using simple linear regression) was also used for the concordance analysis. For estimating the significance of differences between vital signs during different sleep stages, devices, and time courses, we used ANOVA followed by linear mixed effects models with the different groups (devices, sleep stages, and time) as fixed effects (with interactions) and participant as a random effect. MATLAB 2021b was used for all statistical analyses.
Acceptable Agreement for Heart RateThe satisfactory level of agreement between the PSG reference heart rate and the device-determined heart rate was set to an error of 10% or +5 or –5 bpm as recommended by the Association for the Advancement of Medical Instrumentation [,].
Acceptable Agreement for Breathing RateFor breathing rate, to the best of our knowledge, no device-specific satisfactory level of agreement is discussed in the literature. Hence, the agreement in breathing rate estimates between human observers is used to set the acceptable level for our evaluation. We set the permissible level of error in breathing rate estimation to be +4 or –4 cpm, as reported by Lim et al [].
Breathing Disturbance EstimatesApart from epoch-by-epoch heart rate and breathing rate, WSA also provides a “snoring” signal, which is a binary variable depicting snore presence detected by the device. In addition, the WSA also provides the summary estimates of snoring duration (WSA snore), breathing disturbance intensity (WSA breathing disorder index [BDI]), and AHI (WSA AHI). The Emfit and Somnofy devices do not generate breathing disturbance measures. Only complete or valid pairs of estimates were used to analyze the summary measures. For WSA BDI and AHI estimates, we explored the relationship between them and the concordance of these WSA estimates to the PSG AHI in the laboratory.
For the WSA snore estimates, we explored the concordance of the all-night snore duration estimated by the PSG snore sensor (ie, PSG snore microphone placed on the side of the neck) and the WSA. The PSG snore sensor data were scored by the Somnomedics Domino software using 30 dB as the snore amplitude threshold and a minimum snore duration of 300 ms. We further explored the differences in the snore intensity as measured by the PSG snore sensor for the epochs determined to contain snore events by the WSA, followed by an exploration of the distribution of the snore events during the different sleep stages.
Of the 35 participants, more than half of the participants (n=20, 57%) reported comorbidities, including type 2 diabetes, obesity, arthritis, and hypertension, with concomitant medication. In this study population, the mean heart rate was 62.2 (SD 8.9) bpm (men: n=21, 60%; mean 60.6, SD 9.3 bpm; women: n=14; 64.4, SD 7.8 bpm), and mean breathing rate was 14.7, SD 2.9 cpm (men: 14.6, SD 2.9 cpm; women: 14.7, SD 3.0 cpm), as assessed from the overnight laboratory PSG. The average BMI of the participants was 27.0 (SD 4.8) kg/m2, with 17% (n=6, BMI >30) being obese. The mean systolic and diastolic blood pressures measured during screening were 148.7 (SD 16.2) mmHg and 87.0 (SD 9.6) mmHg, respectively. During the clinical PSG, it emerged that 95% (n=33) of the participants in the study had some degree of apnea. Of the participants with apnea, 8 (23%) had severe (AHI>30), 9 (26%) had moderate (AHI=15 to <30), and 16 (46%) had mild apnea (AHI=5 to <15). A total of 10 (29%) participants had PLMI>15, which is similar to the prevalence of periodic limb movement syndrome in community-dwelling older adults [,]. Some form of cardiac arrhythmia was found in 51% (n=18) of the participants, with 29% (n=10) of them also having severe or moderate apnea. A detailed description of the population characteristics can be found in the study by Ravindran et al [].
Overview of Vital Signs DataExample CaseAn example of vital sign data collected by WSA for 14 days at home followed by 1 day in the laboratory is shown in . The participant had moderate apnea with an AHI of 24.1 events/hour, as determined by the clinical PSG during the laboratory visit (day 0). The raster plot (A) shows heart rate, sleep-wake stage, and breathing rate as estimated by the WSA. The vital sign data were available when the participant was in bed at night and during daytime in-bed periods, which were also reported as naps by the participant. The days –14 to –1 depict the data collected at home, while day 0 depicts data collected in the sleep laboratory. The darker, purple-colored regions denote sleep, while the lighter regions denote wake as identified by the WSA. The gray areas represent out-of-bed periods (ie, periods during which the device did not record data). The first day of data from the WSA at home were lost. This participant had irregular nocturnal bed timing, with an average time in bed during the nocturnal period of 10 hours 8 minutes at home. Mean nocturnal heart rate and breathing rate varied across nights. The WSA AHI showed night-to-night variability (B), and the WSA AHI during the laboratory visit was 25 events/hour, which was close to the PSG AHI value. The heart rate showed a trend across the nocturnal sleep period, with a higher heart rate at the beginning of the nocturnal sleep period and a lower heart rate just before the end of the sleep period.
shows the contactless technology–derived vital sign data collected alongside PSG reference data during the laboratory visit for the participant depicted in . The PSG heart rate and breathing rate both show changes as the hypnogram shows transitions between different stages of sleep, with higher variability during wake and REM sleep and lower variability during non-REM sleep. The differences in the data resolution (WSA: 60 seconds; Emfit and Somnofy: 30 seconds) between the contactless technologies can be seen from the plots. The WSA vital signs estimates provided by the device are rounded to the nearest integers, leading to more discretized vital sign data. The trends in the heart rate and breathing rate data recorded by the PSG and the contactless technologies are more similar during the sleep periods compared with the wake periods. The Somnofy had a number of missing estimates of breathing rate during many of the wake epochs determined by the device, and these missing vital sign epochs also coincided with periods of higher activity, as detected by a wrist-worn activity device (Empatica E4). Snoring, as detected by the WSA, followed a pattern that closely followed the snoring signal detected by PSG, with some disagreement in the detection of the snoring event ().
Figure 1. Vital signs from a male participant aged 65 to 70 years collected at home and in the laboratory. (A) Raster plot showing the heart rate (bpm), and breathing rate (cpm) as detected by the Withings Sleep Analyzer along with device-detected sleep (or) wake period and sleep diary information. (B) Estimates of mean heart rate, mean breathing rate, and apnea-hypopnea index (AHI; depicted as circles adjacent to the time courses) were made during the night as determined by the device. 
Figure 2. Vital sign data simultaneously collected from 3 contactless technologies and polysomnography (PSG) in the laboratory for the male participant described in Figure 1. The PSG consensus hypnogram (top) is depicted at the top followed by the heart rate (red), and breathing rate (blue) from PSG and contactless devices, and activity data in arbitrary units are from the Empatica E4 device. The gray regions in all the plots correspond to the lights-off period. A: artifact; bpm: beats per minute; cpm: cycles per minute; N1: stage 1 of nonrapid eye movement (non-REM) sleep; N2: stage 2 of non-REM sleep; N3: stage 3 of non-REM sleep; R: REM sleep; W: wake; WSA: Withings Sleep Analyzer. Summary of Collected DataIn the laboratory study, all 35 PSGs (ground truth or reference data, cohort 1: n=18, 51%; cohort 2: n=17, 49%) were available. The total number of nights of data collected in the laboratory for each of the 3 contactless technologies were 35 for WSA, 16 for Emfit, and 17 for Somnofy. One night of data was lost due to a device malfunction for Emfit. At home, a total of 401 days of data were collected across the 35 participants, with 321 days of data available for WSA (cohort 1: n=10, 56%; cohort 2: n=17, 100%) and 228 days of data available for Emfit (cohort 2: n=17, 100%). At home, for WSA, portions of data from 8 participants were lost in cohort 1 due to deployment errors and Wi-Fi dropouts, with a data loss of 3.3% (11/332 days lost). For Emfit, the data loss was 4.2% (10/238 days lost).
In the laboratory, the range of the heart rate estimated (minimum to maximum) by the contactless technologies was 40 to 90 bpm for WSA and 40 to 135 bpm for Emfit, whereas for breathing rate, the ranges were 8 to 35 cpm for WSA, 6 to 30 cpm for Emfit, and 6 to 30 cpm for Somnofy. The undermattress device–generated vital sign (both heart rate and breathing rate) data for 100% of the in-bed periods. Somnofy bedside radar, by contrast, generated breathing rate data less continuously, resulting in data unavailability at 32.21% (54.41/169.14 hours) of the in-bed period (in-laboratory). Most of these missed breathing rate epochs were found to be in the Somnofy-predicted wake state (the total percentage of breathing rate epochs unavailable per label was follows: wake=34.35/54.41 hours, 62.39%, REM=6.96/54.41 hours, 12.73%; LS=12.19/54.41 hours, 23.1%, and DS=0.91/54.41 hours, 1.78%).
All-Night Vital SignsThe concordance between the nightly average heart rate and breathing rate estimates of the contactless technologies against PSG are shown in and . The WSA (MAPE 3.28%; ICC=0.87) had a lower level of agreement compared with Emfit (MAPE 1.83%; ICC=0.96), and this was due to an outlier participant with severe arrhythmia (see the outlier in ). When the outlier was removed WSA (WSA*: MAPE 1.87%; ICC=1.0) had an agreement with the PSG that was similar to that of Emfit. The MDC was higher for Emfit (3.25 bpm) compared with WSA* (1.17 bpm), which can be seen from the higher dispersion in the Emfit estimates (A). For the breathing rate, Somnofy (MAPE 4.64%; ICC=0.82) had a high agreement, followed by WSA (MAPE 6.29%; ICC=0.78) and Emfit (MAPE 5.46%; ICC=0.76). The MDC follows the agreement results, with Somnofy (1.98 cpm) having a somewhat lower value compared with WSA (2.08 cpm) and Emfit (2.21 cpm).
Figure 3. Association between vital signs estimated by 3 contactless devices and estimates from polysomnography (PSG) averaged across the night while sleeping in the sleep laboratory. (A) Heart rate (beats per minute [bpm]); (B) breathing rate (cycles per minute [cpm]). The error bars represent the SD of the estimate within participants. The PSG heart rate and breathing rate are derived from electrocardiogram (ECG) and respiratory inductance plethysmography thorax (RIP thorax), respectively. The number of participants (ie, nights) available for the devices are Withings Sleep Analyzer (WSA; n=34), Emfit (n=16), and Somnofy (n=17). Table 1. All-night average of vital signs and their agreement metricsa.Vital signsDevice, mean (SD)PSGb, mean (SD)Biasc (SD; 95% CI)LoAd (lower bound), (95% CI)LoA (upper bound), (95% CI)MDCeMAEf (95% CI)SADg (95% CI)MAPEh (95% CI)ICCi (95% CI)Heart rateaMetrics of agreement for overall heart rate of the devices against the electrocardiogram (ECG) estimates (included in the PSG). The number of participants contributing to each of these devices was as follows: WSA (n=35), WSA* (n=34), Emfit (n=16), and Somnofy (n=17).
bPSG: polysomnography.
cBias is the difference in measurement between the device and PSG ECG.
dLoA: limits of agreement.
eMDC: minimum detectable changes; smallest detectable change independent of measurement error (half of Bland-Altman agreement width).
fMAE: mean absolute error.
gSAD: standardized absolute difference; directionless version of Cohen d.
hMAPE: mean absolute percentage error.
iICC: intraclass correlation coefficient with 2-way random effects; measure of measurement reliability.
jWSA: Withings Sleep Analyzer.
kWSA* depicts the outlier-removed WSA data.
Vitals Signs During Different Vigilance StatesWe investigated the agreement of the vital signs estimated by the contactless technologies to the PSG reference during the different vigilance states of the consensus PSG hypnogram. The distribution of the vital signs for the different vigilance states is provided in Figure S1 in . The heart rate and breathing rate estimates of both the PSG and contactless devices were not normally distributed (via the Kolmogorov-Smirnov test). The difference between the mean heart rate estimated by the devices and PSG was <2.5 bpm across all sleep stages. The difference between the mean breathing rate estimated by the devices and the PSG was <1.5 cpm across all sleep stages. Overall, the concordance between the estimates of both heart and breathing rate provided by the contactless technologies and PSG was good across all vigilance states (Figure S2 and Tables S1 and S2 in ).
Time Course of Vital Signs During SleepA shows the time course of the heart rate and breathing rate estimated by the contactless devices and PSG over the PSG lights-off period in the laboratory. B shows these time courses for the sleep diary–defined lights-off periods recorded at home (B). The vital sign data were mean centered and averaged over the epochs during which participants were asleep as detected by the PSG hypnogram for the laboratory data and as detected by the sleep scoring algorithms of the devices for the at-home data. The data are plotted per hour starting from the onset of the lights-off period. The error bars represent the SD of the estimate and are shifted along the x-axis to improve visibility.
Both in the laboratory and at home, the heart rate started close to or above the mean, gradually decreased over the night, and reached the nightly minimum in the second part of the sleep period before a slight increase at the end of the sleep period. The similarity between the contactless technology and PSG heart rate hourly time series in the laboratory was determined using MAPE. The WSA (MAPE 14.42%) closely follows the PSG, while the Emfit (MAPE 144.34%) is less similar.
The breathing rate trends at home were similar to that of the heart rate, starting above the overnight mean and gradually decreasing to a lower value closer to the end of the night. In the laboratory, the breathing rate hourly estimates were fluctuating and did not show any clear trend. Both the WSA (MAPE 129.95%) and Somnofy (MAPE 130.01%) had the highest similarity with the PSG, while the Emfit had the lowest similarity (MAPE 295.76%).
Figure 4. Time course of vital signs during sleep in the laboratory (A and B) and at home (C and D). The number of participants available for each of the devices (measured in nights) in the laboratory (polysomnography [PSG]: n=35, 35 nights; Withings Sleep Analyzer [WSA]: n=34, 34 nights; Emfit: n=16, 16 nights; and Somnofy: n=17, 17 nights) and at home (WSA: n=27, 295 nights; Emfit: n=17, 213 nights). bpm: beats per minute; cpm: cycles per minute. Effect of Temporal Resolution on the Vital Signs AccuracyTo examine the effect of the length of the time period over which the vital signs are computed, we averaged the heart rate and breathing rate estimates over 60 minutes, 10 minutes, and 1 minute and computed the agreement with the corresponding PSG reference estimates ( and ). We examined the cumulative distribution function (CDF) of the MAE at these resolutions to better characterize the estimation error. The CDFs are depicted in , while more detailed scatter plots and associated agreement measures are provided in Figure S3 and Tables S2 and S3 in .
For all devices and both heart and breathing rates, the CDFs become steeper with increasing duration of the time window over which these variables are computed. For both heart rate and breathing rate, the agreement (measured by ICC) with the PSG reference estimates increased with decreasing temporal resolution (Figure S3 in ). On closer inspection of the CDFs, we find that for the heart rate estimates, the error at the 90th percentile is lower for WSA than for Emfit for the 1-minute and 60-minute estimations (overall error<4 bpm). For the Emfit, the median (50th percentile) error of the heart rate estimates became smaller with increasing duration of the time window, but the WSA median error was always close to 1. For the breathing rate, at all 3 resolutions, 50% of the estimates had an error of <1 cpm. When we inspected the 90th percentile error of breathing rate estimates at 1 minute, we found that Somnofy had the lowest error (2.56 cpm), followed by WSA (3.46 cpm) and Emfit (4.26 cpm). This trend was seen for lower resolutions as well. At the 50th percentile, we see the effect of discrete breathing rate output from WSA on the 50th percentile error, where the error of the other 2 devices falls below 0.5, while the WSA error does not. A detailed discussion of the effects of temporal resolution on the agreement between the device and PSG vital sign estimate is provided in .
Table 2. Effect of temporal resolution on reliability of estimates of heart ratea.Heart rate resolutionDevice, mean (SD)PSGb, mean (SD)Biasc (SD; 95% CI)LoAd (lower bound), (95% CI)LoA (upper bound), (95% CI)MDCeMAEf (95% CI)SADg (95% CI)MAPEh (95% CI)ICCi (95% CI)60 minutesaMetrics of agreement for overall heart rate estimates from the devices against PSG electrocardiogram (ECG) estimates (beats per minute) at various temporal resolutions.
bPSG: polysomnography.
cBias is the difference in measurement between the device and PSG ECG (device–PSG ECG).
dLoA: limits of agreement.
eMDC: minimum detectable changes; smallest detectable change independent of measurement error (half of Bland-Altman agreement width).
fMAE: mean absolute error.
gSAD: standardized absolute difference; a directionless version of Cohen d.
hMAPE: mean absolute percentage error.
iICC: intraclass correlation coefficient; with two-way random effects (measures of measurement reliability).
jWSA: Withings sleep analyzer.
Table 3. Effect of temporal resolution on the reliability of estimates of breathing ratea.Breathing rate resolutionDevice, mean (SD)PSGb, mean (SD)Biasc (SD; 95% CI)LoAd (lower bound), (95% CI)LoA (upper bound), (95% CI)MDCeMAEf (95% CI)SADg (95% CI)MAPEh (95% CI)ICCi (95% CI)60 minutesaMetrics of agreement for overall heart rate estimates from the devices against PSG electrocardiogram (ECG) estimates (beats per minute) at various temporal resolutions.
bPSG: polysomnography.
cBias is the difference in measurement between the device and PSG ECG (device–PSG ECG).
dLoA: limits of agreement.
eMDC: minimum detectable changes; smallest detectable change independent of measurement error (half of Bland-Altman agreement width).
fMAE: mean absolute error.
gSAD: standardized absolute difference; a directionless version of Cohen d.
hMAPE: mean absolute percentage error.
iICC: intraclass correlation coefficient; with two-way random effects (measures of measurement reliability).
jWSA: Withings sleep analyzer.
Figure 5. Effect of time window over which vital signs are estimated on device measurement error; (A) heart rate (beats per minute [bpm]) and (B) breathing rate (cycles per minute [cpm]). The cumulative density function of the absolute error is represented for each of the devices for the window lengths 1, 10, and 60 minutes. The median (50th percentile) and the 90th percentile are represented by horizontal lines. PSG: polysomnography; WSA: Withings Sleep Analyzer. Estimating Breathing Disturbances During SleepWSA SnoreThe results of the WSA snore analysis and an example of the overnight time course of the snore data are depicted , as well as in Figure S6 in . Out of the 35 participants, snore data from both PSG and WSA were available for 30 (86%) participants (PSG snore sensor data were not available for 2 participants, and WSA snore data were unavailable for 3 participants). Of the remaining 30 participants, it was determined from PSG snore sensor data that 8 (27%) participants did not have any form of snoring, whereas 22 (73%) participants snored. The WSA incorrectly determined that 5 (23%) of these 22 participants had no snoring, leading to a moderate performance with a Matthews correlation coefficient of 0.69 (C).
PSG determined that the 22 snorers had a nightly snoring duration ranging between 10 and 270 minutes (A). The concordance between the snore duration estimates from the WSA and PSG was high (r2=0.76; P<.001; n=30). When the snoring intensity determined by the PSG snore sensor was grouped based on the WSA snore labels (B), we found that, on average, the intensity of the PSG-detected snore events that were not detected by WSA was lower than the intensity of the snore events detected by both PSG and WSA, but there was a considerable overlap between the distributions.
Overall, the snore events were underestimated by the WSA compared with PSG. The distribution of the snoring events as determined by the PSG snore sensor showed that snoring was present across all sleep states, with >20% of all non-REM epochs having snore events, while for REM epochs, this was 17.48% (D). By contrast, when the distribution of snore events automatically identified by WSA was analyzed, the WSA-determined snore events were high during LS and DS and low during REM (E). The corresponding confusion matrix between the PSG consensus sleep stage and Withings sleep stage prediction during WSA-identified snore events is depicted in E, which shows that the WSA does not score wake when snore events are detected. WSA also scored more snore epochs as DS, followed by LS and REM sleep.
Figure 6. Withings Sleep Anal
Comments (0)