Background: Collecting personal health data through self-trackers, smartphones and other wearable technology can potentially impact a person’s health. Understanding how such data are captured and analyzed is important for patients who are using personalized data to monitor their health. While self-tracking has increased, few users understand the validity and reliability of the data they are tracking. Health applications vary on the information they provide, as well as on the reliability, validity, and accuracy of the data.
Purpose:The purpose of this study was two-fold. First, the study compared blood pressure measurements generated from a traditional automated blood pressure cuff and an Apple iOS application to examine the accuracy of a mobile health application Secondly, the study assessed nursing students’ challenges to using mobile health applications with their patients.
Design: This descriptive study compared blood pressure results from a traditional automated blood pressure cuff and Apple iOS health application results.
Results: Findings suggest that there are differences in the mean measures between the two methods for both systolic blood pressure (SBP) and diastolic blood pressure (DBP). This difference was significant at the 0.001 level for DBP and the 0.01 level for SBP. Further, the results indicate that the mobile method tends to overestimate both measures. Perceived and noted challenges to the use of health applications among this population included cost and lack of accuracy.
Keywords: wearables, health apps, blood pressure, mobile health
Healthcare data such as blood pressure, heart rate and oxygen saturation are now available via fitness trackers, smartphones, and other wearable devices. Added peripheral devices can track glucose, heart rhythms, and more. Smart devices have been embraced by both patients and those interested in tracking their own health parameters. Most of these devices automate data collection, analyze data through predetermined algorithms, and show results with recommendations. Not all data from these devices, however, are equal and understanding how health information is captured and presented is important when using these devices to monitor and manage health.
Background and Significance
Consumer health wearables and applications can impact how patients manage their care and potentially improve outcomes (Low et al., 2018). A number of studies support the potential healthcare value of data from mobile healthcare applications. Burnham and colleagues (2018) found through their literature review that wearable technology could help predict outcomes. These smart wearables can also improve outcomes in heart failure patients monitored at home (Mlakar, 2018). Capturing step counts and heart rate together have a role in the diagnosis of cardiac hypertrophy in healthy people (Lim et al., 2018). A small study of general practitioners in Ireland found that they could see value in wearable blood pressure monitoring for their patients (Morrissey, 2018).
While self-tracking has increased, few users consider or understand the validity and reliability of the data they are tracking. Mobile health applications vary on the information they provide, as well as the reliability, validity, and accuracy of the data (Marston et al., 2017). For example, devices worn on the hip are the most accurate for counting steps, while devices worn on the wrist are not as accurate (Cadmus-Bertram 2017, Evenson et al., 2015). Each type of wearable device is less accurate for measuring energy expenditure than they are for measuring steps (Evenson et al., 2015). Devices measuring heart rate and blood pressure (BP) also have issues with accuracy (Bouts et al. 2018, Plant et al. 2016). According to Jamaladin and colleagues (2018), the majority of applications used to log and monitor BP were of poor quality. They did not, however, assess applications that directly measured blood pressure.
The lack of accuracy of these devices presents a conundrum. If the data is not accurate, then it should not be used to direct care or diagnose symptoms. Device users, however, often believe the data they provide is factual. Nurses and other care givers can help educate consumers, i.e. their patients, about how to best interpret and use the data these devices collect.
Mobile Application Blood Pressure Measurement
The oscillometric principle is the basis of most automatic BP measurement pressure devices (Alpert et al., 2014). This principle can be applied to mobile BP monitoring with the user as the actuator (instead of the cuff) and the phone as the sensor (rather than the cuff) to measure oscillations and pressure (Chandrasekhar et al., 2018). To measure blood pressure using the smartphone, the user presses their finger against the phone, and the phone camera uses photoplethysmography (PPG) to read oscillations in the underlying artery. The PPG method can independently measure SBP and DBP.
The purpose of this research was to compare the accuracy of a traditional automated blood pressure cuff to an Apple iOS mobile health application for blood pressure, and by doing so, raise awareness of the benefits as well as the potential issues with these applications.
This descriptive study compared blood pressure results from a traditional automated blood pressure cuff and Apple iOS health application results. The study questions were:
After receiving approval by the university Institutional Review Board (#20-0073), a convenience sample of senior (final semester or 4-year program) pre-licensure nursing student volunteers were recruited through faculty announcements in classes. Interested students who met the inclusion criteria (over 18, willing to have their blood pressure assessed, and willing to participate in the study) signed a consent form and were oriented to the study.
Setting, Procedure, and Instruments
Blood pressures were assessed with a traditional automated blood pressure cuff (Lovia® Intelligent Type Digital Blood Pressure Monitor), and an Apple® iPhone 6s application (BP Checker®). The manufacturer of the Lovia® device states that readings are equivalent to those obtained by a trained observer using the cuff and stethoscope auscultation method. The manufacturer reported accuracy pressure within±3mmHg (0.4kPa) (User Manual, n.d.). There is no equivalent data about reliability and accuracy from the mobile application developers.
After signing the consent form, participants answered two initial questions: student ID number (used as study number) and if they had ever been diagnosed with hypertension. Blood pressure readings from each device were recorded in a password protected Excel spreadsheet by the research assistant. After having their blood pressure measured with both devices, participants completed an electronic survey, which included questions about their gender and experience with wearables. The survey also included 10 usability and 10 ease-of-use questions from Davis’s Technology Acceptance Model and a fill-in question asking them to describe any challenges or barriers they had when using mobile apps for health care (Davis, 1989). After completing an online questionnaire targeting their perceived challenges with the mobile health application, participants were given a $10 gift card for their participation.
Data Analysis Plan
The statistical methods used in this work included several types of t tests for comparing the mean measures of blood pressure in the sample data. A paired t test compared the mean blood pressure results from the traditional automated blood pressure cuff and mobile methods of measurement. The paired t test was appropriate because of the dependent nature of measurements from the same individual. This dependent nature also lent itself to examining the data via correlation coefficients (notated as ) and scatterplots. Independent t tests were also used to compare mean blood pressure measurements between genders. These means were independent since they arise from different subsets of individuals. All analyses were performed using R statistical software.
Fifty-one (51) senior nursing students participated in this study. Two students reported a diagnosis of hypertension and were thus excluded from the study. The majority (86%, n=42) of participants were between the ages of 20 and 22 with 8% (n=4) over the age of 30. Females made up 90% (n=44) of the student participants. About half (51%, n=25) reported using wearable technology; of those, 72% (n=18) wore an Apple® Smartwatch, 4% each wore a FitBit® (n=1) or Garmin® (n=1), and 20% (n=5) reported wearing a smartwatch (no brand listed). Of the students using a wearable device, most self-reported being competent (36%, n=9) or proficient (56%, n=14)) in its use. Ninety-eight percent (n=42) of participants reported being competent (12%, n=6), proficient (69%, n=34) or expert (16%, n=8) in the general use of technology.
Table 1 displays results from performing one-sided paired t tests for comparing the mean levels of blood pressure measures from the traditional and mobile methods of measurement. These results suggest that there are differences in the mean measures between the two methods for both SBP and DBP. This difference was significant at the 0.001 level for DBP and the 0.01 level for SBP. Further, the results indicate that the mobile method tended to overestimate and have less variance associated with both measures since both the means and standard deviations for the mobile measures are higher than those for the traditional measures. If the mobile measures appropriately represented the true blood pressure (as measured with the traditional method), we would not expect to see p-values indicating a significant difference in the two measures.
Table 1: One-sided paired t tests to compare mean measures from the traditional and mobile methods of measurement
Figure 1 offers two scatter plots to demonstrate the association between the traditional and mobile methods of measurement. These plots show little to no association for both blood pressure measures between the traditional and mobile methods of measurement (SBP: , DBP: ). Further, they also confirm the over-estimation noted via the t tests in Table 1, especially for DBP since the measures congregate on the upper end of the x axis while they are generally on the lower end of the y axis. The lack of correlation also suggest that the mobile method is not able to represent the traditional method. We would expect to see a strong positive correlation (data scattering from the bottom left to the top right of the plot) if the mobile data represented the actual blood pressure well. Rather, we observed mobile data clustered around values that are considered normal range (120 for SBP and 80 for DBP) when the traditional method indicated more variation in the true blood pressure measures.
Figure 1: Scatterplots comparing the traditional and mobile methods of measurement
Study participants identified challenges using mobile health applications for direct blood pressure measurements and questioned the accuracy of the data. Others challenged the utility of mobile health applications and reported that the cost, lack of knowledge, and technical difficulties associated with using such applications may prevent their adoption by patients. Some participants listed more than one challenge. See Table 2 for a list of reported challenges.
Table 2: Reported Challenges to using Health Applications
Gender differences were also evaluated with the foundational understanding that male blood pressures typically measure higher than females (Dorans et al. 2018). This was also seen in this study where SBP for male participants were higher than females when measured with the traditional automated blood pressure cuff (data not shown). However, this difference was not seen when comparing the male and female SBP when measured with the mobile device. This further suggests that the mobile application lacks accuracy.
According to a Pew research report, 21% of American adults report wearing either a smartwatch or fitness tracker (Vogels, 2020). Twenty-eight percent of people aged 18 to 34 reported they wear a tracker and 25% reported they use health applications (McCarthy, 2019). Carroll and colleagues (2017) found that as of 2015, nearly two-thirds (64%) of the American public owned a mobile phone, an increase from 35% in 2011. Those who used health applications (compared with those who either did not have applications or did not have the necessary equipment) were more likely to be younger, live in metropolitan areas, have more education, have higher income, and report excellent health (Carroll et al., 2017). Of those who were using health applications, most (92%) reported them as helpful.
While many of the most widely used health applications focus on fitness, there are others more focused on general health, and few that directly measure blood pressure. The general population, however, does not often consider the accuracy of the data these devices provide. Most applications in the Apple App store that directly track blood pressure are categorized as entertainment rather than medical applications. Some are now listed as beta applications so that developers can continue to gather information to improve their applications. Luo and colleagues (2019) tested a smartphone-based application using transdermal optical imaging technology to measure blood pressure and found that it predicted blood pressure with a clinically acceptable accuracy. According to Alessa and colleagues (2019), few applications for hypertension in the Apple and Google Play stores included evidence of their effectiveness or usability, and most did not meet data security and privacy standards. The ECRI Institute listed mobile health as one of its 2019 top safety concerns (2019). Risks the institute identified included a lack of regulations of devices, barriers preventing providers from obtaining and using the data, and not using the devices correctly (ECRI, 2019). ECRI recommended addressing usability concerns and developing ways to help providers identify patients most likely to use the devices and when patients were using the devices correctly.
Final semester senior nursing students were selected for this study to help them critically evaluate health measurements from a mobile device. The study participants were not aware that mobile BP measurements were available, and they reported concerns about the accuracy, as well as the ease of using the applications. Participant-reported barriers were similar to the risks identified by the ECRI Institute. Since participants reported that the data could be questionable, this lack of accuracy could prevent them from using available applications or recommending them to their patients. Teaching students about the availability of these applications, the potential risks of their use, and their pros and cons will better prepare them to advise others.
Study participants were collected by convenience, which led to a sample that was mostly female and under the age of 30. This population is not among the most prone to hypertension, with none of the participants reporting a previous hypertension diagnosis. Since the application used for this study is no longer available, duplication of this study is not possible until new applications are released.
Suggestions for Future Research
While more health applications are being developed, an accurate application for direct measurement of blood pressure is not yet available without the need for additional peripherals. Thus, in the interim, patients should be advised to seek other convenient means of measuring their blood pressure. Nurses and other clinicians should look for opportunities to discuss the use of mobile health applications with their patients. While applications such as heart rate and step trackers are fairly consistent and accurate, other applications such as direct BP monitoring should be used with caution. Future research could compare new applications as they are developed, or expand to other measurements such as heart rate or oxygen saturation. Informatics nurses can help nurses and patients understand how to critically assess mobile health applications to select valid and reliable options.
The views and opinions expressed in this blog or by commenters are those of the author and do not necessarily reflect the official policy or position of HIMSS or its affiliates.
Powered by the HIMSS Foundation and the HIMSS Nursing Informatics Community, the Online Journal of Nursing Informatics is a free, international, peer reviewed publication that is published three times a year and supports all functional areas of nursing informatics.
References & Bios
Alessa, T., Hawley, M., Hock, E., & Witte, L. d. (2019). Smartphone Apps to Support Self-Management of Hypertension. JMIR Mhealth and Uhealth, 7(5). https://doi.org/10.2196/13645
Alpert, B., Quinn, D., Gallick, D. (2014). Oscillometric blood pressure: A review for clinicians. Journal of the American Society of Hypertension, 8, 930–938 (2014). https://doi.org/10.1016/j.jash.2014.08.014
Boutes, A., Brackman, L., Martin, E., Subasic, M., & Potkanswicz, E. (2018). The accuracy and validity of iOS-based heart rate apps during moderate to high intensity exercise. International Journal of Exercise Science, 11 (7), 533-540. https://digitalcommons.wku.edu/ijes/vol11/iss7/4
Burnham, J. P., Lu, C., Yaeger, L. H., Bailey, T. C., & Kollef, M. H. (2018). Using wearable technology to predict health outcomes: A literature review. Journal of the American Medical Informatics Association: JAMIA, 25(9), 1221-1227. https://doi.org/10.1093/jamia/ocy082
Cadmus-Bertram, L. (2017). Using fitness trackers in clinical research: What nurse practitioners need to know. The Journal for Nurse Practitioners, 13(1), 34-40. https://doi.org/10.1016/j.nurpra.2016.10.012
Carroll, J., Moorhead, A., Bond, R., LeBlanc, W., Petrella, R., & Fiscella. (2017). Who uses mobile phone health apps and does use matter? A secondary data analytics approach. Journal of Medical Internet Research, 19(4): e125. https://doi.org/10.2196/jmir.5604
Chandrasekhar, A., Kim, C.-S., Naji, M., Natarajan, K., Hahn, J.-O., & Mukkamala, R. (2018). Smartphone-based blood pressure monitoring via the oscillometric finger-pressing method. Science Translational Medicine, 10(431), eaap8674. https://doi.org/10.1126/scitranslmed.aap8674
Davis, F. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13 (9), 319-340. https://doi.org/10.2307/249008
ECRI Institute. (2019). 2019 Top 10 Patient Safety Concerns. Executive Brief. https://www.ecri.org/landing-top-10-patient-safety-concerns-2019
Evenson, K., Goto, M., & Furberg, R. (2015). Systematic review of the validity and reliability of consumer-wearable activity trackers. International Journal of Behavioral Nutrition and Physical Activity, 12, 159 – 181. https://doi.org/10.1186/s12966-015-0314-1
Jamaladin, H. J. H., Belt, T. H. v. d., Luijpers, L. C. H., Graaff, F. R. d., Bredie, S. J. H., Roeleveld, N., & Gelder, M. M. H. J. v. (2018). Mobile apps for blood pressure monitoring: Systematic search in app stores and content analysis. JMIR mHealth and uHealth, 6(11), e187. https://doi.org/10.2196/mhealth.9888
Lim, W. K., Davila, S., Teo, J. X., Yang, C., Pua, C. J., Blöcker, C., Lim, J., Ching, J., Yap, J., Tan, S., Sahlén, A., Chin, C., The, B., Rozen, S., Cook, S., Yeo, K., & Tan, P. (2018). Beyond fitness tracking: The use of consumer-grade wearable data from normal volunteers in cardiovascular and lipidomics research. PLoS Biology, 16(2), e2004285. https://doi.org/10.1371/journal.pbio.2004285
Low, C. A., Bovbjerg, D. H., Ahrendt, S., Choudry, M. H., Holtzman, M., Jones, H. L., Pingpank, J., Ramalingam, L., Zeh, H., Zureikat,A., & Bartlett, D. (2018). Fitbit step counts during inpatient recovery from cancer surgery as a predictor of readmission. Annals of Behavioral Medicine, 52(1), 88–92. https://doi.org/10.1093/abm/kax022.
Luo, H. , Yang, D. , Barszczyk, A. , Vempala, N. , Wei, J. , Wu, S. J. , Zheng, P. P. , Fu, G. , Lee, K. & Feng, Z. (2019). Smartphone-Based Blood Pressure Measurement Using Transdermal Optical Imaging Technology. Circulation: Cardiovascular Imaging, 12(8), e008857. doi: 10.1161/CIRCIMAGING.119.008857.
Marston, H., Freeman, S., & Musselwhite, C. (Eds). (2017). Mobile e-Health. Springer International Publishing.
McCarthy, J. (December 11, 2019). One in Five U.S. Adults Use Health Apps, Wearable Trackers. Gallup Polls. https://news.gallup.com/poll/269096/one-five-adults-health-apps-wearable-trackers.aspx
Mlakar, M., Puddu, P. E., Somrak, M., Bonfiglio, S., Luštrek, M., Chiron and HeartMan research projects, & on behalf of the Chiron and HeartMan research projects. (2018). Mining telemonitored physiological data and patient-reported outcomes of congestive heart failure patients. PloS One, 13(3), e0190323. https://doi.org/10.1371/journal.pone.0190323
Morrissey, E. C., Glynn, L. G., Casey, M., Walsh, J. C., & Molloy, G. J. (2018). New self-management technologies for the treatment of hypertension: General practitioners' perspectives. Family Practice, 35(3), 318-322. https://doi.org/10.1093/fampra/cmx100
Plante, T., Urrea, B., MacFarlane, Z., Blumenthal, R., Miller, E., Appel, L., & Martin, S. (2018). Validation of the instant blood pressure smartphone app. JAMA Internal Medicine, 176(5), 700–702. https://doi.org/10.1001/jamainternmed.2016.0157.
User Manual Blood Pressure Monitor EBP 095 (n.d). https://images-na.ssl-images-amazon.com/images/I/911kKv3naKL.pdf
Vogels, E. (January 9, 2020). About one-in-five Americans use a smart watch or fitness tracker. Pew Research. https://www.pewresearch.org/fact-tank/2020/01/09/about-one-in-five-americans-use-a-smart-watch-or-fitness-tracker/
Lisa Anne Bove, DNP, RN- BC
Dr. Lisa Anne Bove has worked in healthcare informatics for over 25 years in a variety of positions and has been certified by the American Nurses Credentialing Center in nursing informatics since 1996. Her work has focused on improving health through the use of electronic information. Dr. Bove has been an interim chief nursing informatics officer of a major multi-system hospital system, program manager for large clinical design and implementation projects, and educator for both continuing education and college level classes. Her field of study is around ease of use and usefulness of mobile health care technology to advance practice through the use of data. Her teaching is focused on informatics, as well as on project management, EHR implementation and leadership.
Rachel Carroll, PhD
From Dr. Rachel Carroll’s training as a biostatistician in a public health department to her current role as a professor at a university charged with serving the region in which it is situated, community focus and engagement has always been a motivator for her research and study. Dr. Carroll has published over 30 peer-reviewed articles throughout the course of her career in high-impact journals. Topics have included statistics, biostatistics, spatial statistics, time series analysis, environmental research, oncological research, epidemiology, and social sciences.