Abstract
Five spot-check patient monitors from four leading manufacturers were tested for their performance using three commonly used noninvasive blood pressure (NIBP) simulators. The test focused on how the monitors performed in terms of sensitivity, determination time, and measurement range between high and low systolic and diastolic pressures at low, normal and high pulse rates. For the purpose of this study, spot-check monitors are defined as those used for taking intermittent blood pressure readings in low-acuity applications such as in the wards and in the emergency department. Monitors from GE HealthCare, Mindray, Philips and Welch Allyn were evaluated. The test results demonstrated that GE HealthCare’s CARESCAPE™ VC150 monitor with DINAMAPTM SuperSTAT algorithm* delivered a high degree of sensitivity compared to the other tested devices when interrogated at high and very low pulse amplitudes. The measurement range of all five devices straddled the clinically relevant systolic and diastolic blood pressures, and GE HealthCare’s CARESCAPE VC150 monitor demonstrated the widest minimum-maximum range for both. Both Welch Allyn devices revealed sensitivity <90% in several pressure bands within clinically relevant ranges for both systolic and diastolic pressure. In this test, the greater sensitivity seen in the CARESCAPE VC150 monitor does not come at the expense of determination time, since the CARESCAPE VC150 monitor achieved the median value among the five devices and was 22 seconds quicker than the slowest of the five devices.
*GE HealthCare’s Portrait™ VSM vital signs monitor uses the same DINAMAP algorithm
DISCLAIMER: The data collection and data analysis for this study was conducted in 2019 by an independent third party. The data from this study is stored in GE HealthCare’s internal quality record system under reference DOC2452066. Data collection for all devices was done in accordance to a strict protocol to ensure fair comparison. The results from this study can only be interpreted in light of the specific test conditions and do not necessarily reflect usage in a standard clinical environment.
Purpose
This paper evaluates overall performance among spot-check NIBP measurement devices used in clinical settings. The study did not evaluate the clinical accuracy of the devices; instead, it focused on their range of performance. The parameters evaluated during the test were:
Sensitivity
This parameter refers to the percent of successful determinations. The test recorded successful determinations taken at 100% amplitude and at minimal 10% amplitude at varying pressure levels. One hundred percent amplitude means 1 ml filling of the simulator plunger. The actual pressure pulse in mmHg depends on the ratio of the volume pushed by the simulator plunger (1 ml), the volume of the filled cuff, and its pressure. One hundred percent is physiologically typical for adults and 50% is typical for neonates. Sensitivity in blood pressure measurement is important across the clinically relevant range but is of the utmost importance at thresholds for hypertension and hypotension. Sensitivity in blood pressure (BP) determinations at the extremes can impact the efficiency of the clinical team in identifying blood pressures that may require acute attention.
Range of diastolic and systolic readings
This refers to minimum and maximum diastolic and systolic pressure values each device was able to record, at low pulse rate (≤ 60 bpm), mid-range pulse rate (>60- 160 bpm), and high pulse rate (>160 bpm). The wider the range the better the performance as it is will enable the device to get a measurement in a wider variety of patient types and conditions.
Determination time
This refers to the time required for the system to run through the full cycle of the blood pressure measurement at given simulation settings. The primary benefit of a short determination time is patient comfort, but it also can contribute to clinician productivity.
The highest performing monitors overall are those that best combine performance against all three of these parameters. For example, a short determination time is desirable, but not to the detriment of sensitivity. Similarly, high sensitivity is beneficial, but not if at the expense of range of measurement and thus a limitation on the range of patient types and acuities the device can monitor effectively
Test setup and methods
A third-party clinical research organization conducted this analysis in accordance with a structured protocol in which each device was interrogated across a range of simulated blood pressures, heart rates and amplitudes of flow. Three commonly available simulators, a CuffLink, BP Pump 2 and ProSim 8, were used for the test. Five patient monitors, listed below, with NIBP measurement capability were tested and compared:
- GE HealthCare CARESCAPE VC150
- Mindray VS-900
- Philips IntelliVue MP5
- Welch Allyn 6000 Series
- Welch Allyn Connex
Each monitor was tested using all three simulators at systolic pressures, diastolic pressures and pulse rates covering the specified measuring range. Measurements were also taken at varying pulse amplitudes from 10% to 240%. The results were documented along with any error messages given by the monitors and with other clinically meaningful observations.
The percentages of successful determinations at each simulator setting of systolic and diastolic pressure, pulse rate, and pulse amplitude were also documented. When recording data, any measurement at which the monitor gave a reading was considered successful, even if the result was false. Where a test was done at a certain simulator setting, the percent number was copied from the next lower setting. In most cases, the settings are shown in steps of 10. As an exception, when many measurements were done at inbetween values, an extra row was added to the table. If only a few measurements were done at those values, the results were included in the next lower setting at the low end and next higher setting at the high end.
Results
Sensitivity
The most substantial difference in sensitivity occurred at the minimal 10% amplitude, where the two Welch Allyn monitors recorded lower values than the other three monitors evaluated. GE HealthCare’s CARESCAPE VC150 monitor at 100% amplitude showed the highest sensitivity (97%); the second highest in sensitivity was the Philips IntelliVue MP5 monitor (89%). The lowest sensitivity measures were from the Welch Allyn devices: the Connex recorded 79% sensitivity and the 6000 Series recorded 74%. At low amplitude (10%) there was great variability among the devices. Phillips IntelliVue MP5 (88%) and GE HealthCare’s CARESCAPE VC150 monitor (83%) had the highest sensitivity. In contrast, the Welch Allyn 6000 Series and Connex sensitivity recorded was 40% and 0%. Table 1 shows the percentage of successful determinations using all three simulators at different amplitudes and pulse rates as defined in the test protocol.
100% amplitude means 1 ml filling of the simulator plunger. The actual pressure pulse in mmHg depends on the ratio of the volume pushed by the simulator plunger (1 ml), the volume of the filled cuff, and its pressure.
An aggregate view of sensitivity performance stratified by systolic and diastole pressures is depicted below. Each cell in the tables represents the percentage of successful measurements done at different amplitudes and pulse rates as defined in the test protocol. For systolic pressure (Table 2), the data spans pressure range from 40 to 300 mmHg, while for diastolic pressure (Table 3), the range reported is from 40 to 200 mmHg. The majority of devices demonstrated 100% sensitivity throughout, with decay at the extremes of the range tested.
As stated earlier, sensitivity should not be confused with a measurement of accuracy; in fact, it is a measure of robustness in obtaining a recorded measure during a blood pressure measurement attempt. This could be extrapolated to patient care. For example, if a device’s sensitivity were 50%, then for every two patients measured, only one would have a valid BP reported. In the other patient there would be a need to either repeat the measurement with the same device, re-measure with a different device, or perform a manual measurement. Each of these options may require additional time in order to record a clinically valid and actionable result and may reduce the patient comfort.
Measurement range
Each monitor was tested to determine the lowest and highest pressure it was able to record at the given range of pulse rates. Where a device failed to record a measurement at the high or low end of its range, a second trial was performed to verify that the system would actually keep failing at that pressure.
When pulse rates were low (≤ 60 bpm), all devices demonstrated a measurement range for both diastolic and systolic pressure that included the clinically relevant threshold for hypertension (140 mmHg systolic and 90 mmHg diastolic). and the same was true for hypotension (90 mmHg systolic, 60 mmHg diastolic). Mindray’s VS900 monitor demonstrated the widest measurement range for diastolic pressure; GE HealthCare’s CARESCAPE VC150 monitor showed the widest range for systolic pressure. Low heart rate and hypertension is not a common occurrence in the absence of a Cushing reflex. A Cushing reflex is most commonly attributed to patients with intracranial hypertension due to brain lesions or injury (i.e. subarachnoid hemorrhage) and can be life-threatening if not readily identified and treated.
When pulse rates were between 60 and 160 bpm, all devices demonstrated a measurement range for both diastolic and systolic pressure that included the clinically relevant threshold for hypertension (140 mmHg systolic and 90 mmHg diastolic), and for hypotension (90 mmHg systolic, 60 mmHg diastolic). The Philips’ IntelliVue MP5 monitor demonstrated the widest measurement range for diastolic pressure; the GE HealthCare’s CARESCAPE VC150 monitor showed the widest range for systolic pressure. Of note is the upper limit of the Welch Allyn devices (Connex systolic upper limit 250, 6000 Series systolic upper limit 227). While rare, systolic blood pressure that exceeds 250 mmHg can occur, especially in patients with poor compliance to hypertension treatment. Sustained and profound elevations in systolic pressure can cause complications that include pulmonary edema, myocardial infarction and stroke. All of these conditions are time-sensitive and hinge on timely and reliable blood pressure recordings.
When pulse rates were high (>160 bpm), all devices demonstrated a measurement range for both diastolic and systolic pressure that included the clinically relevant threshold for hypertension (140 mmHg systolic and 90 mmHg diastolic) and for hypotension (90 mmHg systolic, 60 mmHg diastolic). GE HealthCare’s CARESCAPE VC150 monitor showed the greatest measurement range for both diastolic and systolic pressure. As in the previous section, the Welch Allyn devices showed upper limits of detection below 200 mmHg. This is not common clinically but can be found during periods of profound catecholamine release, where systolic pressure can exceed 200 mmHg. In the management of hypertensive crisis with tachycardia, time is of the essence in reducing the risk for development of cardiovascular complications that can include stroke and cardiac failure.1
Determination Time
Determination times were recorded for blood pressure 120/80 mmHg, pulse rates 60 to 80 bpm, and 100% pulse amplitude as measured with the three simulators. The values represent the typical low value or the average of the nearest typical low values. All repeats of long measurements at these settings were excluded. The Welch Allyn monitors showed the shortest determination times.
The mean and median determination times were 30 (± 5.9) and 26 seconds, respectively. The CARESCAPE VC150 monitor was within one standard deviation of the mean and defined the median (26 seconds), while the Mindray VS-900 (14 sec) and Philips IntelliVue MP5 (22 sec) were slower than the GE HealthCare device and beyond two standard deviations from the mean. The difference in determination time between the GE HealthCare and Phillips devices is in excess of 60 seconds for every three blood pressure measurements. In a busy clinical environment, this time savings can represent more than 33 minutes for every 100 patient assessments conducted.
Observations
In this benchmark study using five blood pressure measuring devices, all showed comparable measurement ranges that straddled clinically significant thresholds for both hypotension and hypertension. However, GE HealthCare’s CARESCAPE VC150 monitor demonstrated the widest measurement range. Furthermore, the sensitivity at 100% amplitude and 10% amplitude, which represents an exaggerated clinical worst-case scenario, revealed that CARESCAPE VC150 monitor was at or near the top when compared to the others. In contrast, both Welch Allyn devices revealed sensitivity <90% in several pressure bands within clinically relevant ranges for both systolic and diastolic pressure. Finally, the greater sensitivity seen in the CARESCAPE VC150 monitor does not come at the expense of determination time.
Discussion and conclusions
Blood pressure measurement often involves trade-offs between measurement speed, range and sensitivity. Frequently, the blood pressure device is used in a general medical floor where 15 to 30 patients will require vital sign measurement checks at a minimum of every eight hours. The time efficiency of the clinician to complete all vital sign measurements in this scenario can be enhanced by a shorter determination time and greater device sensitivity.
Beyond the scope of this analysis was an assessment of accuracy, for each device was assumed to be accurate within its respective dynamic range. However, sensitivity analysis was performed to determine the frequency with which each device would produce a valid BP measurement when assessed over a range of pressures and simulated flow conditions. When using devices with lower sensitivity, repeated measurements will be more likely to occur; requiring more clinician time (for example, a device with 74% sensitivity may require 26% more time due to repeated measurements). Therefore, efficiency from a more rapid determination time may be offset by poorer sensitivity. GE HealthCare’s CARESCAPE VC150 monitor performed at the upper range for sensitivity while having determination times at the median among all devices tested.
Blood pressure determination across the clinically relevant range is important, and each device was assumed to have sufficient accuracy to receive clearance by the U.S. FDA and other regulatory authorities. At the extremes is where BP detection for these devices may vary, notably at elevated systolic blood pressures. In cases such as hypertensive crisis, new onset headache suggestive of subarachnoid hemorrhage, or decompensated congestive heart failure, the systolic blood pressure in a small percentage of patients can be greater than 200 mmHg and can approach 300 mmHg. When the systolic pressure is elevated to this magnitude, there is an imminent risk for brain or cardiac complications to progress.2 Therefore, the performance of a blood pressure device becomes of increasing importance beyond workflow efficiency and patient comfort.
References
- Picone, D.S., et al., Accuracy of cuff-measured blood pressure: systematic reviews and meta-analyses. Journal of the American College of Cardiology, 2017. 70(5): p. 572-586.
- Vincent, Jean-Louis, et al. Textbook of Critical Care E-Book, https://www.worldcat.org/title/textbook-of-critical-care-e-book/ oclc/1017837997 (Accessed October 2020).