Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/ou_press/stopwatch-the-preliminary-evaluation-of-a-smartwatch-based-system-for-maLd7xBcgR
- Publisher
- Oxford University Press
- Copyright
- Copyright © 2022 Society for Research on Nicotine and Tobacco
- ISSN
- 1462-2203
- eISSN
- 1469-994X
- DOI
- 10.1093/ntr/nty008
- Publisher site
- See Article on Publisher Site
Abstract
Introduction: Recent developments in smoking cessation support systems and interventions have highlighted the requirement for unobtrusive, passive ways to measure smoking behavior. A number of systems have been developed for this that either use bespoke sensing technology, or expensive combinations of wearables and smartphones. Here, we present StopWatch, a system for passive detection of cigarette smoking that runs on a low-cost smartwatch and does not require additional sensing or a connected smartphone. Methods: Our system uses motion data from the accelerometer and gyroscope in an Android smartwatch to detect the signature hand movements of cigarette smoking. It uses machine learn- ing techniques to transform raw motion data into motion features, and in turn into individual drags and instances of smoking. These processes run on the smartwatch, and do not require a smartphone. Results: We conducted preliminary validations of the system in daily smokers (n = 13) in laboratory and free-living conditions running on an Android LG G-Watch. In free-living conditions, over a 24-h period, the system achieved precision of 86% and recall of 71%. Conclusions: StopWatch is a system for passive measurement of cigarette smoking that runs en- tirely on a commercially available Android smartwatch. It requires no smartphone so the cost is low, and needs no bespoke sensing equipment so participant burden is also low. Performance is currently lower than other more expensive and complex systems, though adequate for some applications. Future developments will focus on enhancing performance, validation on a range of smartwatches, and detection of electronic cigarette use. Implications: We present a low-cost, smartwatch-based system for passive detection of cigarette smoking. It uses data from the motion sensors in the watch to identify the signature hand move- ments of cigarette smoking. The system will provide the detailed measures of individual smoking behavior needed for context-triggered just-in-time smoking cessation support systems, and to enable just-in-time adaptive interventions. More broadly, the system will enable researchers to ob- tain detailed measures of individual smoking behavior in free-living conditions that are free from the recall errors and reporting biases associated with self-report of smoking. © The Author(s) 2018. Published by Oxford University Press on behalf of the Society for Research on Nicotine and Tobacco. 257 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. Downloaded from https://academic.oup.com/ntr/article/21/2/257/4823697 by DeepDyve user on 12 July 2022 258 Nicotine & Tobacco Research, 2019, Vol. 21, No. 2 While SmokeBeat improves on RisQ in not requiring additional, Introduction bespoke sensing hardware, in its current form as an integrated In a recent commentary, Naughton described the current knowledge system for smoking behavior change, it requires wireless connection on the potential for using mobile phones to deliver just-in-time (JIT) to a smartphone to measure smoking behavior. While smartphone support for smokers attempting to quit. He detailed three types of ownership continues to increase world-wide, ownership is still much JIT support: user-triggered support, in which the delivery of sup- lower among individuals with lower incomes (eg, in the United port is initiated by a request from the user, server-triggered support, States, 64% for individuals earning <$30k, compared with 83% for which is initiated automatically on the basis of a set of pre-deter- those earning >$50k ). Low income is associated with higher preva- mined rules, and context-triggered support, which is delivered on lence of smoking globally, meaning that interventions aiming to the basis of detailed, dynamic information about the user, including reach sections of society with higher rates of smoking cannot make their patterns of specific behaviors, location, and physiological state. assumptions about smartphone ownership. Furthermore, those indi- Naughton described how using this rich context data can make sup- viduals with smartphones will typically only be within close prox- port systems robust to within and between individual differences, imity (and therefore wireless network range) of their smartphone and how the ultimate context-based system could be considered approximately 90% of the time. This means that smoking detection to be one that captures the necessary data for this unobtrusively, systems relying on smartphones for detection will miss some aspects without the need for self-report. of smoking behavior (eg, leaving a smartphone inside when going One key item of data for a context-based smoking cessation sup- outside to smoke). port system will be a detailed measure of an individual’s smoking StopWatch is a system that uses data from the motion sensors in behavior. Naughton points out that the act of smoking can already a commercially available smartwatch, and identifies smoking events be detected automatically using a wrist-worn accelerometer, and that by applying machine learning methods that run entirely on the watch this may soon be possible with off-the-shelf smartwatches. Here, we itself. Unlike RisQ it does not require the user to wear any bespoke present a smartwatch system, called StopWatch, that does exactly this. sensing devices, and unlike RisQ and SmokeBeat, there is no need It uses data from the motion sensors on a low-cost smartwatch to de- for a wirelessly connected smartphone. It is subject independent, and tect the signature hand movements of cigarette smoking, and does does not need to be trained to recognize an individual’s smoking so without the need for a smartphone or data network connection. gestures. It provides the potential for development of a number of A number of previous studies have explored different ways to smartwatch-based systems (that do not require smartphones), for use technology to passively detect cigarette smoking. One approach helping smokers to quit, including a system enabling smokers to re- has been to use on-body sensors to measure respiratory rates, and view their smoking behavior over time to better understand their to look for the patterns of changes in respiratory rate associated patterns of smoking behavior, and JIT smoking interventions that with cigarette smoking. This technique has been combined with use information about an individual’s patterns of smoking behavior the use of proximity detectors that measure hand-to-mouth move- to target more effective smoking behavior change interventions. ments to give increased sensitivity and specificity of smoking detec - Here, we describe the implementation of the StopWatch system, tion. Reliable measurement of respiratory rate, however, requires and preliminary validation of the system in laboratory and free-living use of cumbersome thoracic sensor bands. These may be acceptable conditions. for use in short-term measurement sessions, but are not suitable for longer-term use in free-living conditions. As motion sensors have become more commonplace in mobile Methods and wearable digital devices, researchers began using these to de- System Overview tect the signature hand movements associated with cigarette smok- The StopWatch system comprises software that resides entirely ing. Parate et al. demonstrated this using their RisQ system, which within a low-cost, commercially available smartwatch. For develop- comprised a bespoke motion-sensor equipped wristband wirelessly ment and validation, we used a G model watch from manufacturer connected to a smartphone. The wristband contained an integrated LG, running the Android Wear v1.5 operation system. This device inertia measurement unit (IMU) that fused linear motion data from provided a good balance between battery life, comfort, and usability an accelerometer and angular motion data from a gyroscope with (essential for longitudinal use), an open development environment, orientation data from a compass to provide three-dimensional trajec- and ease of access to sensor data (further details of smartwatch se- tory data describing hand movements. These data were transferred lection criteria are included in Supplementary Material). The acceler- to the smartphone by Bluetooth, and machine learning techniques ometer and gyroscope motion data were produced by an InvenSense were applied to classify instances of cigarette smoking. MPU6516 IMU running in normal mode on the watch. These were The RisQ system achieves high level of sensitivity and specificity sampled at a rate of 100Hz. The user interface for the StopWatch in laboratory and free-living tests, but it has the limitation that it system is detailed in Supplementary Material. requires a bespoke sensing wristband. Using this will be less arduous than a thoracic band respiratory sensor, but it is still an additional Analysis Pipeline sensing device that places burden on the user. This issue has recently been addressed by a smoking behavior change system that uses the Following the approach adopted by Parate et al., a multi-stage ana- motion sensors in commercially available smartwatches and activity lysis machine-learning pipeline was used to detect instances of cig- monitors to measure smoking behavior. SmokeBeat uses acceler- arette smoking from raw motion sensor data. Unlike the Parate and ometer and gyroscope data from a smartwatch or activity monitor, SmokeBeat systems, our analysis pipeline runs entirely on a smart- a smartphone application, and cloud-based analytics to detect the watch, and not on a smartphone. We used a three-stage analysis hand movements associated with cigarette smoking, identify pat- pipeline, illustrated in Figure 1. Step 1. Raw motion data are sub- terns in their smoking behavior, and engage the user with behavior jected to binning and threshold (gyroscope data), and applied to an change techniques (eg, goal setting). initial decision tree classifier (accelerometer data) to identify when Downloaded from https://academic.oup.com/ntr/article/21/2/257/4823697 by DeepDyve user on 12 July 2022 Nicotine & Tobacco Research, 2019, Vol. 21, No. 2 259 a hand movement corresponds to one of a number of motion fea- percentage of true positives among all the events identified by the tures relevant to smoking, which include “hand raising to mouth,” system as smoking) of 75%, recall (the percentage of true positives “hand stationary at mouth,” “hand moving away from mouth.” Step among the actual smoking events) of 92%, and accuracy (the per- 2. Motion features are presented to a second decision tree classifier, centage of true positives and true negatives among the total num- which looks for features of particular values, happening in a specific ber of events) of 90%. Further details of the laboratory validation pattern, to identify a single drag of a cigarette. Specifically, the deci - results, and the methods used to compute the performance metrics, sion tree looks for—hand raise to mouth motion that lasts between are included in Supplementary Material. 0.3 and 0.7 seconds, followed by—hand stationary at mouth for Participants subsequently took the system away and wore it between 0.4 and 8.0 seconds, followed by—movement of the hand in free-living conditions for a period of 24 hours. In this second away from the mouth. Step 3. The number of drags and time be- phase, an adapted version of the application used previously to label tween drags is analyzed to look for a reliable instance of smoking a motion data when identifying the analysis pipeline parameters, was cigarette. When six drags are detected, with a duration of <80 sec- used to record self-report data. With this, if the system failed to de- onds between drags, this is designated as an instance of smoking a tect an instance of smoking, the participant could easily record this cigarette. false negative with a button press on the smartwatch. Similarly, if The procedures for determining specific analysis pipeline param - the system detected an instance of smoking when the participant eters (based on laboratory and free-living smoking data collected was not smoking, the participant could log this as a false positive from 38 participants), together with details of data formats and pro- with a single button press. (Note: this application was also running cedures for downloading smoking data from the StopWatch system in the laboratory validation session to ensure no differences between are described in Supplementary Material. the systems under test.) Participants also completed a paper diary of smoking events, recording the time and date of every cigarette smoked, and every false positive and false negative. In line with Validation and Results established techniques for testing classification system performance Validation was performed using a set of 14 new smoking partici- in extended free-living conditions, true negatives were not recorded, pants not previously involved in determining the parameters for the as these can artificially inflate performance statistics. This means analysis pipeline (eligibility criteria are described in Supplementary accuracy cannot be determined, and system performance is instead Material). One participant was excluded from our validation data as characterized by recall and precision. it transpired that, contrary to the instructions provided to all partici- A summary of the results of the free-living validation is shown pants, this participant had worn the smartwatch on their nondomi- in Figure 2. Overall, in the free-living validation phase the system nant hand. The remaining 13 participants (6 female, all right-handed, performed with a precision of 86% (95% CI: 78% to 93%) and a mean age 21 years, SD 3 years) completed two stages of verification. recall of 71% (95% CI: 63% to 78%). As can be seen from Figure 2, Firstly, system performance was assessed in a laboratory setting, there was considerable inter-participant variation in precision and with participants completing a number of tasks that included smok- recall. Performance data for the RisQ system (the only comparable ing a cigarette, drinking from a glass, and eating with hands and cut- system with detailed performance data available at this time) also lery. All tasks were performed sitting down. Participants were first shows notable variation in performance between participants. The provided with detailed printed instructions, and a demonstration of variation in the StopWatch performance data is different to that the StopWatch system (they took the instructions with them after observed in the RisQ data (more variation in recall performance the laboratory session for reference in the free-living phase). The with StopWatch and more variation in precision with RisQ), but this experimenter then moved to behind a two-way mirror to observe is to be expected as RisQ uses different machine learning methods. and record system performance during the different tasks. Overall, To explore the level of agreement between the data from StopWatch in this validation phase the system performed with a precision (the and the paper diaries, we calculated Cohen’s Kappa for each par- ticipant, which indicated substantial agreement between StopWatch data and diary data (mean 0.73, 95% CI: 0.67 to 0.79). Discussion StopWatch is a system for passive detection of cigarette smoking. It uses data from the accelerometer and gyroscope motion sensors in a low- cost, Android smartwatch, and applies an analysis pipeline running on the watch to automatically detect and log instances of cigarette smok- ing. Preliminary validation of the system was performed using an LG G-Watch, running the Android Wear 1.5 operating system. In free-living conditions, the system achieved precision of 86% and recall of 71%. We envisage a number of applications for the system. Because it detects smoking passively, requiring no input from the user, the system can provide detailed measurements of smoking behavior that are free from the recall errors and reporting biases associated with 9–12 self-report of smoking. The system will therefore provide new opportunities for any researchers interested in measuring detailed patterns of smoking behaviors in individuals in free-living condi- Figure 1. Analysis pipeline for detection of drags and instances of smoking. tions, with minimal user burden, and at low cost. Downloaded from https://academic.oup.com/ntr/article/21/2/257/4823697 by DeepDyve user on 12 July 2022 260 Nicotine & Tobacco Research, 2019, Vol. 21, No. 2 Figure 2. Free-living validation results. For some applications, the current level of performance may not be an Returning to the JIT smoking cessation support Naughton issue. Having modest recall means the system may miss some instances described, the StopWatch system provides the capability to unobtru- of smoking, but the high precision means when it does label an event sively capture smoking behavior data for context-triggered JIT sup- as smoking, there is a good level of certainty the event was an instance port systems. Indeed, by gathering detailed smoking behavior data of smoking. In the future, the processing power of smartwatches will for individuals, StopWatch could enable more advanced forms of JIT increase, and it will be possible to run increasingly powerful classifi - support, such as Just-In-Time Adaptive Interventions. cation algorithms on the watch, increasing both recall and precision Other systems, like SmokeBeat and RisQ, also provide capability performance. Another limitation of the system is that, while it will run for passive detection of smoking. What sets StopWatch apart from on any Android smartwatch equipped with an accelerometer and gyro- these other systems is that it just uses a low-cost smartwatch, and scope, it has currently only been validated running on an LG G-Watch. does not require bespoke sensing hardware, a smartphone, or data Future work on the StopWatch system will include validating network connectivity. This has several benefits: (1) The system will the system on a range of different smartwatches, exploring ways work as long as the smartwatch is worn and has power, and will not to increase the performance of the system, testing the feasibility of stop working if the watch is out of range from a paired smartphone or using the system in a variety of smoking behavior change interven- if it loses data network connectivity. (2) Using a commercially avail- tions, and exploring the feasibility of using the system to passively able smartwatch means we leverage the manufacturer’s investment measure use of electronic cigarettes and distinguish between cigar- in usability and design. This is important because for measurement ette smoking and electronic cigarette use in dual use individuals. and intervention systems that need to be worn and used for extended periods of time, user experience is an important consideration. (3) Using just a smartwatch keeps the cost of the system low. The watch Funding we used is currently available for less than $100, and this (excluding This work was supported by the Medical Research Council Integrative the need to perform a brief set-up to load the application software Epidemiology Unit at the University of Bristol which is supported by onto the smartwatch), is the total cost of the StopWatch system. the Medical Research Council and the University of Bristol (grants MC_ Looking to the future, recent forecasts from wearable market UU_12013/6 and MC_UU_12013/7). experts indicate mobile network (cellular) connectivity will be one of the key new features that will see the market for smartwatches Declarations of Interest grow strongly in the next few years. Indeed, the wearable market is already seeing significant changes, with sales in basic activity moni - The authors are listed as inventors on patent application PCT/ tors that cannot run third party applications declining, and sales of GB2017/050110 “Method and Device for Detecting a Smoking smartwatches showing substantial growth. The inclusion of mobile Gesture” network connectivity is important, as it will increase the number of apps that can run on a smartwatch without the necessity to be paired Acknowledgments with a smartphone. This is likely to shift the way smartwatches are MRM is a member of the United Kingdom Centre for Tobacco and Alcohol used in the future, with users increasingly expecting a smartwatch app Studies, a UKCRC Public Health Research: Centre of Excellence which to be a standalone experience, free from the need for a smartphone. receives funding from the British Heart Foundation, Cancer Research UK, In its current form, the StopWatch system has a number of weak- Economic and Social Research Council, Medical Research Council, and the nesses. The performance is not as high as other passive detection sys- National Institute for Health Research, under the auspices of the UK Clinical tems that use smartphone-based analysis pipelines (eg, Parate et al. ). Research Collaboration. Downloaded from https://academic.oup.com/ntr/article/21/2/257/4823697 by DeepDyve user on 12 July 2022 Nicotine & Tobacco Research, 2019, Vol. 21, No. 2 261 8. Dey AK, Wac K, Ferreira D, Tassini K, Hong J, Ramos J. Getting closer: References an empirical investigation of the proximity of user to their smart phones. 1. Naughton F. Delivering “Just-In-Time” smoking cessation support via UbiCom. 2011;2011:163–172. mobile phones: current knowledge and future directions. Nicotine Tob 9. Hatziandreu EJ, Pierce JP, Fiore MC, Grise V, Novotny TE, Davis RM. The Res. 2016;19(3):379–383. reliability of self-reported cigarette consumption in the United States. Am 2. Parate A, Chiu MC, Chadowitz C, Ganesan D, Kalogerakis E. RisQ: rec- J Public Health. 1989;79(8):1020–1023. ognizing smoking gestures with inertial sensors on a wristband. MobiSys. 10. Krall EA, Valadian I, Dwyer JT, Gardner J. Accuracy of recalled smoking 2014;2014:149–161. data. Am J Public Health. 1989;79(2):200–202. 3. Ali A, Hossain S, Hovsepian K, Rahman M, Plarre K, Kumar S. mPuff: 11. Klesges RC, Debon M, Ray JW. Are self-reports of smoking rate biased? automated detection of cigarette smoking puffs from respiration measure- Evidence from the second national health and nutrition examination sur- ments. ISPN. 2012;2012:269–280. vey. J Clin Epidemiol. 1995;48(10):1225–1233. 4. Lopez-Meyer P, Tiffany S, Sazonov E. Identification of cigarette 12. Sillett RW, Wilson MB, Malcolm RE, Ball KP. Deception among smokers. smoke inhalations from wearable sensor data using a support vector Br Med J. 1978;2(6146):1185–1186. machine classifier. Conf Proc IEEE Eng Med Biol Soc. 2012;2012: 13. Spruijt-Metz D, Nilsen W. Dynamic models of behavior for just-in-time 4050–4053. adaptive interventions. Pervasive Comput. 2014;13(3):13–17. 5. Smokebeat. http://somatix.com/solutions/. Accessed July 25, 2017. 14. Piwek L, Ellis DA, Andrews S, Joinson A. The rise of consumer health 6. Pew Research Centre. Mobile Fact Sheet. www.pewinternet.org/fact-sheet/ wearables: promises and barriers. PLoS Med. 2016;13(2):e1001953. mobile/. Accessed November 13, 2017. 15. IDC. Worldwide Quarterly Wearable Device Tracker. www.idc.com/get- 7. Casetta B, Videla AJ, Bardach A, et al. Association between cigarette smok- doc.jsp?containerId=prUS42342317. Accessed July 25, 2017. ing prevalence and income level: a systematic review and meta-analysis. 16. IDC. Worldwide Quarterly Wearable Device Tracker. www.idc.com/get- Nicotine Tob Res. 2017;19(12):1401–1407. doc.jsp?containerId=prUS43015117. Accessed November 11, 2017.
Journal
Nicotine and Tobacco Research – Oxford University Press
Published: Jan 4, 2019
Keywords: smoking; cell phones; cigarette smoking; smartwatches; sensor; mental recall; accelerometers; precision; machine learning; social support; self-report; smoking cessation; smokers; electronic cigarettes