Wearable data analysis, visualisation and recommendations on the go using android middleware

Wearable data analysis, visualisation and recommendations on the go using android middleware Multimed Tools Appl (2018) 77:26397–26448 https://doi.org/10.1007/s11042-018-5867-y Wearable data analysis, visualisation and recommendations on the go using android middleware 1 1 Marios C. Angelides & Lissette Andrea Cabello Wilson & Paola Liliana Burneo Echeverría Received: 2 October 2017 /Revised: 1 March 2018 /Accepted: 4 March 2018 / Published online: 4 June 2018 The Author(s) 2018 Abstract Wearable technology comes with the promise of improving one’s lifestyles thru data mining of their physiological condition. The potential to generate a change in daily or routine habits thru these devices leaves little doubt. Whilst the hardware capabilities of wearables have evolved rapidly, software apps that interpret and present the physiological data and make recommendations in a simple, clear and meaningful way have not followed a similar pattern of evolution. Existing fitness apps provide routinely some information to the wearer by mining personal data but the subsequent analysis is limited to supporting ad hoc personal goals. The information and recom- mendations presented are often either not entirely relevant or incomplete and often not easy to interpret by the wearer. The primary motivation behind this research is to address this wearable technology software challenge by developing a middleware mobile app that is supported by data analytics and machine learning to assist with interpretation of wearer data and with making of personal lifestyle improvement recommendations on the go which may then be used to feedback to the wearer’s daily goals and activities. The secondary motivation is to correlate and compare with trends in the wearer’s peer community. . . Keywords Wearable technology Visual information systems Machine learning * Marios C. Angelides marios.angelides@brunel.ac.uk Lissette Andrea Cabello Wilson lcabello@fiec.espol.edu.ec Department of Electronic and Computer Engineering, College of Engineering Design and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK 26398 Multimed Tools Appl (2018) 77:26397–26448 1 Introduction Google’s Verily Life Sciences have recently unveiled an ordinary looking health tracking Bstudy watch^ to unobtrusively and continuously collect physiological data from the wearer, such as their heart rate, electrocardiograms (ECG), movement data, their skin’s electrical conductance as well as ambient light and sound. The Google watch does not currently allow the wearer to see their health data, only the date, time, and some instructions. The long-term vision of Verily is to unravel biomarkers through tracking thousands of healthy people especially when they fall ill. Although the watch features a processor that manages and encrypts its wearer’s data and its software is updatable over-the-air, Verily is not currently marketing the watch as a medical device, for which they will need FDA approval, but as a clinical and observational tool for scalable collection of rich and complex data sets. The watch’s battery lasts for up to a week without a charge and it stores Braw^ data produced over the same amount of time, so it does not have to be synched as frequently as other watches. The inclusion of an ECG which Verily regards as their biggest technical novelty can reveal heart abnormalities. Such measurements are normally taken in hospitals with several stick-on electrodes but the Google watch picks up a lower-resolution signal from just two electrical contacts when the wearer grasps the metal bezel with their other hand. The watch is being deployed in a study in Europe whose aim is to track the progression of Parkinson’sdisease among patients diagnosed with the disease. Uninterrupted long-term sleep and heart readings are invaluable in monitoring the progression of Parkinson’s and cannot be monitored in hospitals, especially the patients’ sleep patterns. Verily argues that the watch’s scope can be extended through the inclusion of additional sensors that will help generate additional bio- feedback thus shaping its place in the Internet of Things (IoT) landscape [8, 11, 12]. Having continuous and uninterrupted access to such personal data in real time may help with the diagnosis of underlying conditions on the go, as maybe the case with Google’s watch, and making personal recommendations on life style improvements, otherwise, by identifying personal preferences and behaviour including food habits, sleep patterns, and daily activity schedules. It is widely acknowledged that all these devices can motivate individuals to change habits towards a better health or lifestyle. For instance, they are being considered in the workplace to monitor workers’ activities and control schedules and as they hold consumer’s health data, these devices are also a point of interest for physicians and health insurers [1, 10, 21]. Samsung, Fitbit, Apple and Sony have long joined the race in wearable technology hardware but according to Verily, the next but necessary stage in the evolution is the development of software algorithms that can Binterpret^ the wearer’s raw data, thus making accurate diagnoses and personalised recommendations, where possible, often by comparing the wearer’s data against the data of other wearers in the same peer group [3, 14]. As these devices cover a broad range of wearers of different ages, physical conditions and lifestyles, it is important that the presented information is as useful to all wearers as the wearable hardware with which it has been recorded. This would in turn inform the development of personalised immersive and alternative reality environments. The focus of this paper is to address the wearable technology software challenge by developing such a personal immersive environment that collects, analyses and visualises personal wearer data mined with a wearable device and which integrates a machine learning approach to making personal recommendations for lifestyle im- provements on the go in close consideration of trends in the wearer’s peer community. These lifestyle improvement recommendations may then be used to feedback to the Multimed Tools Appl (2018) 77:26397–26448 26399 Table 1 Issues with wearables Ethical/Legal Economic Social Technological Intellectual Property √ Data Protection √ Privacy √ Safety √ Regulation √ Innovation √ Price √ Choice √ Connectivity √ Integration √ Inclusion √ Networking √ Systems Interoperability √ Technical Standards √ Wireless Connectivity √ Data Security √ Application areas √√ Usability √ Real-time √ wearer’s daily goals and activities. As access to Google watch’s wearer data is currently restricted, the authors showcase their approach with a Samsung Gear Fit and Fitbit. The environment is developed as a mobile middleware app aimed at offering personal recommendations on the go for improving personal lifestyles. To show how this aim has been achieved, the rest of the paper is organised as follows: Section 2 presents a related research review, section 3 details the mobile apps developed, including our data analysis and machine learning approaches, and section 4 presents our experimentation. Section 5 concludes. Fig. 1 Social connectivity [34] 26400 Multimed Tools Appl (2018) 77:26397–26448 Decision Making 9 7 Analysed Data & Goal Recommendaon Synchronisaon Visualisaon 1 6 Analysed Data & Configuraon Recommendaon & Goal Setup Forwarding 2 5 Raw Data Analysis Tracking & Recommendaon Acvity Generaon Data Raw Data Synchronisaon Forwarding Fig. 2 A flowchart of wearable data analysis, visualisation and recommendation 2 Related research review Table 1 shows the range of issues reported in literature and arising from our research review [1–14, 16, 18–27, 31–36]. The findings have been grouped under four categories, namely ethical/legal, economic, social and technological. However, the boundaries between these categories are fuzzy since discussion of these issues has been from various perspectives. This Fig. 3 A flowchart of comparative analysis during steps 5 and 6 Multimed Tools Appl (2018) 77:26397–26448 26401 Chromosome GA Inial Populaon Generaon Chromosome Selecon (e.g. Roulee) Chromosome Crossover & Mutaon Chromosome Chromosome Verificaon Full Decision Vector Limit Constraint Vector Optimisation Constraint Vector Gene Value Limit Value Optimisation Value Chromosome PO Non- & Dominated Idenficaon Sleep Duration g1 480 431 L1 Sleep Calories g2 390 O1 300 Chromosome Non- & Dominated Fitness Value Exercise Duration g3 O2 30 Exercise Distance g4 3 O3 3 Chromosome Exercise Calories g5 318 O4 300 Non-Dominated Step Count g6 14373 O5 10000 Steps Distance g7 10 O6 10 Step Calories g8 392 O7 300 Fig. 4 A flowchart of GA and PO deployment during steps 5 and 6 and a typical chromosome Fig. 5 FitNalyser opening screen 26402 Multimed Tools Appl (2018) 77:26397–26448 Fig. 6 FitNalyser Profile screen is the result of this subject area being very multidisciplinary in nature whereby contributions have been made from different fields. Whilst the focus of our research review has been primarily software apps that support the wearer in their daily use of the wearables, which reportedly is now at the core of wearable software evolution, what has been reported so far in literature in relation to our objective are software apps that support and fine-tune the functionality of the wearable technology, e.g. energy consumption or display resolution. An Argus Insights Report [2] argues that whilst wearable hardware has been improving to an evolving standard, the same cannot be said for wearable software apps that interpret the data such devices collect. The report suggests that a clear gap exists between devices and such software apps that may enhance an aspect of the wearer’s lifestyle such as responsive learning or interactive gaming. Despite that, social connectivity through wearables has been on the rise where there is a common goal such as fitness or gaming. Figure 1 illustrates an Android wearable device linking up with a smartphone through Bluetooth and communicating with the cloud and other wearers who do the same. Therefore, we have had to consider such software app developments in closely related subjects, i.e. for analysing and presenting data collected from single or multiple sensors like those that define the wearable technology functionality; for example, optical sensors such as those for heart rate monitoring through measurement Multimed Tools Appl (2018) 77:26397–26448 26403 Fig. 7 FitNalyser My Data screen of blood flow changes, chemical sensors such as those for hydration monitoring through real-time measurement of sodium and potassium levels in body sweat, temperature sensors for monitoring the ambient temperature or impact sensors for monitoring vibration. Liu et al. [15] paint a landscape of human activity recognition sourced from past landscapes that includes sensor-based approaches that shed light on current progress which is still slow and not reaching far enough in satisfying the wearer needs. Shoaib et al. [28] present an android smartphone app that collects data using a Samsung Galaxy S2 smartphone’s accelerometer, gyroscope and magnetometer sensors to collect data on physical activities such as walking, running, sitting, standing, walking upstairs or down- stairs and then analysing this data using WEKA’s machine learning tool, with mixed success. The app typifies the approach most authors report, i.e. the use of one or more sensors to collect data and then correlate the data to one or more physical activities. Liu et al. [15]surveyseveral approaches and apps that follow this approach to collect data and then use a variety of techniques to interpret the data such as Neural Nets, Bayesian Networks, Decision tables and trees, etc. Several limitations are reported and arise with existing approaches and android apps that are deployed to support the wearer in revisiting their fitness goals, for example. Wearers often perform multiple activities at the same time or may change, 26404 Multimed Tools Appl (2018) 77:26397–26448 Fig. 8 FitNalyser step count screen planned or unplanned, from one activity to another; for instance, part of a wearer’s running course may involve going up or down a set of stairs. How is data correlated to each activity in such a case? As discussed earlier wearables are increasingly used as social connectivity tools, thus wearers often set their goals and compare their performance against that of their peer group. Current approaches and android apps only serve the wearer. How can wearer initial or revised fitness goals and perfor- mance data, for example, be set and compared to that of their peer group? Current approaches and android apps focus primarily on interpreting past performance and presenting the results to the wearer. How can past wearer and peer group performance be used to make on-the-go recommendations to the wearer? In the next section, we present an android app that addresses the above shortcomings. 3 Wearable data analysis, visualisation and recommendation We have used Mobincube to develop two versions of the android middleware app for Samsung smartphones [17]. One version is entitled FitNalyser and the other Bru-Fit. We have integrated JMP PRO with MatLab’s Genetic Algorithms (GA) and Pareto Optimality (PO) to analyse and Multimed Tools Appl (2018) 77:26397–26448 26405 Fig. 9 FitNalyser exercise screen compare the wearer’s data to peer community data with the former and to use the result to generate recommendations that may support the wearer with monitoring their daily goals and activities with the latter. Figure 2 shows a flowchart of the use of a wearable device, the analysis and visualisation of the wearer activity data it records and the recommendations generated using a machine learning technique to support the wearer with their monitoring of their daily goals and activities. The wearer configures their wearable and sets initial goals (1), the wearable sensors track the wearer’s daily activities and record data (2), the wearable synchronises with a smart device to transfer the wearable data recorded (3), the raw data is forwarded by the smart phone to the wearable data server (4) from which it is retrieved and analysed in comparison to community data to generate personal recommendations (5), the results of the analysis and the recommendations are forwarded to the smart device (6) where they are visualised and offered to the wearer (7) so that they may assist them with monitoring their daily goals and activities (8). The smart phone synchronises with the wearable to auto-update any revised goals (9). Iterating through steps 1 to 4, then 8 and 9 is the standard process with almost all wearable devices when paired with a smart phone. In our work, we introduce three additional steps, i.e. 5, 6 and 7, that address the shortcomings highlighted at the end of the previous section 26406 Multimed Tools Appl (2018) 77:26397–26448 Fig. 10 FitNalyser sleep screen and represent our novel contribution to research and development. JMP PRO and MatLab’s GA and PO are deployed at steps 5 and 6, and either FitNalyser or Bru-Fit are deployed at step 7. 3.1 Steps 5 and 6: Comparative analysis and recommendation generation Figure 3 shows a flowchart of the process of comparative analysis with JMP PRO during steps 5 and 6. JMP Pro has been selected for raw wearable data analysis and comparison to peer data for many reasons. First, it enables predictive modelling through the set of techniques it deploys, e.g. decision trees, bootstrap forest, Naive Bayes and neural networks. Stats emerge with these techniques, e.g. the average influence of additional hours of sleeping on calorie burning, residual variation of number of steps mid-week and at weekends, all of which help with model prediction accuracy even with missing or incomplete data. Second, it enables cross-validation through the set of techniques it uses, e.g. data partitioning or holdback. Cross- validation is not only invaluable when comparing wearer data to peer data but also building a predictive model that is not based on a single wearer sample which in turn avoids over-fitting. Third, it allows model comparison through common quality measures, e.g. R2, ROC, AUC, etc. This helps with selecting a model with the best goodness of fit, parsimony and cross- Multimed Tools Appl (2018) 77:26397–26448 26407 Fig. 11 FitNalyser Community Data screen Fig. 12 FitNalyser Community Data stats for step count, exercise and sleep 26408 Multimed Tools Appl (2018) 77:26397–26448 Fig. 13 FitNalyser Analysis screen validation as input to the GA. Fourth, it uses generalized regression through a set of regularization techniques such as Ridge, Lasso, adaptive Lasso, Elastic Net, adaptive Elastic Net that help overcome the biases that arise with strongly correlated predictors, e.g. number of hours sleeping and calorie burning which may result in over-fitting. This approach helps with building a diverse predictive model that may include data with many outliers, or skewed data. Figure 4 shows a flowchart of MatLab’s GA and PO deployment during steps 5 and 6. Generating an optimal recommendation D for each wearer becomes a multi-optimisation problem (MOP) which seeks to optimise objectives ⋃ O ðÞ D ≤0 subject to limit constraints d¼1 ! ! ! ! ⋃ L ðÞ D ≤0: Fx ¼ f x ……f x where x is an n-dimensional decision vari- d 1 k d¼1 able vector x ¼ x ;:…; x . The objectives and limit constraints are sourced from the best 1 n goodness of fit wearer model produced by JMP PRO in comparison to peer community data, e.g. steps per day, hours sleeping per night, hours exercising per day. Sofokleous and Angelides [29] suggest the use of GAs and Pareto Optimality (PO). The GA uses a fitness-based selection operator, such as roulette, to select chromosomes for reproduction from the evolved population and uses crossover and mutation to reproduce new offspring chromosomes. Once a termination criterion s met, the GA delivers one or more chromosomes as best-fit recommendations, D ,that satisfy all limit constraints, e.g. min number of sleeping hours. The PO evaluates each Multimed Tools Appl (2018) 77:26397–26448 26409 Fig. 14 FitNalyser Recommendations screen Fig. 15 Bru-Fit’s top section of the opening screen 26410 Multimed Tools Appl (2018) 77:26397–26448 Fig. 16 Bru-Fit’s bottom section of the opening screen D with reference to optimisation objectives, O , e.g. max number of steps per day, to distinguish i i between dominated and non-dominated chromosomes and then assigns a strength value to each non-dominated chromosome that is proportional to the number of the dominated chromosomes it weakly dominates. Figure 4 also includes an example of a typical chromosome as a Full Decision Vector along with its Limit and Optimisation Constraints Vectors. The chromosome population (P) is expressed as P = P ∪ P,where P and P contain the non-dominated and E I E I dominated chromosomes respectively. The strength value of a non-dominated C is expressed as j C jC ∈P &C ≺C j fg j j I i j SðÞ C ¼ , where C ∈ P . The PO then calculates a fitness value for each i E jP jþ1 dominated chromosome, C as the sum of the strength values of the non-dominated chromosomes j, jP j SCðÞ i ; if C ≺C i j that weakly dominate C,i.e. FC ¼ 1 þ ∑ ,where C ∈ P . A dominates j j i E ; otherwise i¼1 Fig. 17 Bru-Fit’s general information screen Multimed Tools Appl (2018) 77:26397–26448 26411 Fig. 18 Bru-Fit’s Active Min screen B, A ≺ B,iff O (A) ≤ O (B), ∀i =1, …, D and O (A)< O (B)forsome i and A weakly dominates B, i i i i A≺B,iff O (A) ≤ O (B), ∀i=1, …, D,where D is the number of optimisation constraints. A Pareto i i 0 0 Front of optimal D (C ) that satisfy: f : P ¼fg C ; C ; …; C ⟶C ∈P is produced where n i j 1 2 n j E E is the number of chromosomes. Game theorists argue that the distribution of optimal solutions on a Pareto Front suggests the existence of a Bbest-fit^ optimal recommendation D . Sofokleous and Angelides [30] calculates Euclidean Distances (ED) between each chromosome, i.e. a qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi similarity measure between two chromosomes: ED C ; C ¼ O C −O ðÞ C þ … j i 1 j 1 i þ O C −O ðÞ C ,where O (C ) is the result of evaluation of O using chromosome C m j m i k z k z values. The closer C is to C , the more likely is that C will result in the same satisfaction as C . j i j i The fitness value of each chromosome is then re-calculated and the one with the highest f(C ) value is selected as the Bbest-fit^ optimal recommendation D and recommended to the wearer: 0 1 UserRating 0 @ A fCðÞ¼ 10 þ ∑ 1 þ ED C ; C = 1 þ P i j i C ∈P E 26412 Multimed Tools Appl (2018) 77:26397–26448 Fig. 19 Bru-Fit’s Active Min screen weekly average The models produced by JMP PRO and the recommendations generated by the GA and PO are then forwarded to either FitNalyser or Bru-Fit for visualisation during step 7. 3.2 Step 7: FitNalyser This android middleware app runs on a Samsung smartphone paired with a Samsung Gear Fit hybrid smartwatch and fitness tracker which may be worn 24/7 since it is an always-on activity tracker that measures Bsteps^, Bexercise^, Bsleep^ and Bheart rate^. The 9-step process of Fig. 2 that includes FitNalyser at step 7 unfolds as follows: 1. Configuration and Goal Setup: The wearer populates their wearable with personal data, such as name, gender, age, weight, and height, to enable the device to collect data correctly and sets up their daily goals, e.g. number of steps, number of hours spent walking or sleeping. 2. Tracking Activity: The wearable’s sensors track the wearer’s daily activities and sleep patterns without inhibiting any of their senses. Multimed Tools Appl (2018) 77:26397–26448 26413 Fig. 20 Bru-Fit’sActive Min screen monthly average 3. Data Synchronisation: The wearable establishes a Bluetooth connection to, and synchro- nizes with, the smartphone to transfer collected data. 4. Raw Data Forwarding: Data Viewer, a third-party software tool running on the smartphone, receives the dataset and packages this for transfer to a laptop via the tracker’s cloud servers. 5. Raw Data Analysis and Recommendation Generation: JMP PRO performs general- ised regression analysis of the wearer dataset in comparison to the peer community dataset and produces a model that is used by the GA and PO to generate recommendations. 6. Analysed Data and Recommendation Forwarding: The laptop transfers the model pro- duced by JMP PRO and the recommendations generated by the GA and PO back to the smartphone via the tracker’s cloud servers. 7. Analysed Data and Recommendation Visualisation: The analysed data model and recom- mendations are visualised on FitNalyser to support the wearer with monitoring their daily goals and activities. 8. Decision Making: The wearer reconsiders his current daily goals and activities, and may set up new activity goals, e.g. increase the number of hours spent sleeping, reduce the number of hours spent walking. 9. Goal Synchronisation: The smart phone synchronises with the wearable to auto-update any revised goals. 26414 Multimed Tools Appl (2018) 77:26397–26448 Fig. 21 Bru-Fit’s Steps screen daily Figures 5 through to 14 showcase the FitNalyser android app at step 7. Figure 5 depicts the opening screen of the mobile app which has been divided into 4 parts: profile, my data, community data and analysis. Figure 6 shows the wearer profile following configuration: Gender, date of birth, weight and height. Figure 7 shows My Data screen which offers the choices of step count, exercise or sleep to the wearer. On Fig. 8 the step count performance is visualised, i.e. the daily average of number of steps, burned calories and daily distance and the monthly total of number of steps, burned calories and distance following tracking. Similarly, on Fig. 9 the exercise performance is visualised, i.e. the daily average of burned calories, duration, and distance as well as the monthly totals. Likewise, on Fig. 10 the nightly and monthly sleep average is visualised. Figure 11 mirrors the choices on offer on My Data screen for Community Data but in addition it reveals the number and age range of wearers, differentiating between males and females, and their weight. On Fig. 12 the daily averages and monthly totals for Community Data for both genders are visualised. Figure 13 relays personal against community performance data following analysis wearer own data and comparison against community data, with a gender bias. This helps develop personal and community fitness trends in relation to all activities. On this screen, the wearer may choose to consider recommendations generated by the GA and GO during step 5. On Fig. 14 recommendations are visualised in relation to diet, hydration and excercising. The wearer may use these recommendations to review current daily goals and activities and set up new ones. Multimed Tools Appl (2018) 77:26397–26448 26415 Fig. 22 Bru-Fit’sSteps screen weekly and monthly 3.3 Step 7: Bru-fit This android middleware app runs on a Samsung smartphone paired with a Fitbit® Surge hybrid smartwatch and fitness tracker that may be worn 24/7 as it is an always-on activity tracker that measures Bsteps^, Bexercise^, Bsleep^ and Bheart rate^. Its sensors include GPS, 3- axis accelerometer, 3-axis gyroscope, digital compass, optical heart rate monitor, altimeter, ambient light sensor and vibration motor. The 9-step process of Fig. 2 that includes BruFit at step 7 unfolds as follows: 1-3. Configuration and Goal Setup, Tracking Activity, Data Synchronisation: as with FitNalyser. 4. Raw Data Forwarding: A Fitbit open script executes the transfer of the raw dataset to a laptop via Fitbit’s cloud servers. 5. Raw Data Analysis and Recommendation Generation: as with FitNalyser. 6. Analysed Data and Recommendation Forwarding: A Fitbit open script executes the transfer of the models produced by JMP PRO and the recommendations generated by the GA and PO back to the smartphone via Fitbit’s servers. 7-9. Analysed Data and Recommendation Visualisation, Decision Making, Goal Synchronisation: as with FitNalyser. 26416 Multimed Tools Appl (2018) 77:26397–26448 Fig. 23 Bru-Fit’s Sleep screen daily Figures 15 through to 29 showcase the Bru-Fit android app at step 7. Figure 15 shows the top section of the opening screen of BruFit. Figure 16 shows the bottom section of the opening screen of BruFit, which has been divided into 5 parts: Active Min, Steps, Sleep, Calories, Analysis. Figure 17 gives an example of the general information that may be visualised on request. It shows the wearer’s profile which includes their activity level and the daily goals set following configuration and goal setup. The wearer may revisit these goals and set up new goals after considering the recommendation. More detailed stats are available in the Activity Frequency section which shows personal bests in relation to Steps, Very Active Minutes, Calories, km covered, Floors, and Hours Asleep. Figure 18 shows the Active Minutes screen which includes a daily average of steps, active minutes, calories, km covered, floors, and hours asleep and a comparison to the daily goals. Two detailed graphs show floor climbing and daily activity during the reporting period. Figure 19 reports the weekly activity and averages for floor climbing and distance whereas Fig. 20 reports the average monthly activity with emphasis on floor climbing. Figure 21 shows the Steps screen on which it compares the daily steps to the daily goal as well as a daily and hourly distribution of steps whereas Fig. 22 reports the weekly and monthly distribution of steps. Figure 23 shows the Sleep screen and reports daily sleep patterns and efficiency, actual sleep time, bed and rise time whereas Fig. 24 reports Multimed Tools Appl (2018) 77:26397–26448 26417 Fig. 24 Bru-Fit’s Sleep screen weekly and monthly the weekly and monthly sleep data. Figure 25 shows the Calories screen that details daily, weekly and monthly calories spent whereas Fig. 26 visualises a comparison between personal and community data with a gender bias on calories spent using bar and line graphs and boxplots. Likewise, Figs. 27 and 28 visualise comparisons between personal and community data, also with a gender bias, in relation to sleep data and steps respectively. Figure 29 shows Bru-Fit’s Recommendations screen whose content is generated by the GA and GO during step 5. This screen visualises recommendations in relation to sleep or lack of it, bed and rise times, calorific intake and spent, and activity or lack of it. The wearer may consider these recom- mendations to revisit current daily goals and activities and set up new ones. 4 Validation: a month in the process This section presents the application of the process on Fig. 2 over the period of one month with each of the two applications that we have developed. The purpose of this is to validate our approach and, in particular, the novelty in steps 5, 6 and 7. 26418 Multimed Tools Appl (2018) 77:26397–26448 Fig. 25 Bru-Fit’s Clories screen daily, weekly and monthly 4.1 FitNalyser At Step 1, the wearer configures their wearable and sets their initial goals of daily step count, exercise and sleep. During Step 2, the Samsung Gear Fit tracks the daily step count, exercise and sleep durations and monitors the heart rate without inhibiting any of the wearer’ssenses. At Step 3, the wearable synchronizes with the smartphone using a Bluetooth connection and transfers the daily dataset to the Samsung smartphone. During Step 4, Data viewer which has been installed on the smartphone, receives the dataset, packages that as a cvs file and forwards this file via the tracker’s cloud servers to a laptop that runs JMP PRO and the GA with PO. Table 2 shows a screenshot of the raw data file on step count. Data collected include start and end time, calories and distance. During Step 5, once both the personal daily and the community datasets have been received on the laptop, JMP PRO is used first to analyse and compare the personal against the public community dataset and build a general regression model. The first calculation made is the Body Mass Index (BMI) which uses measures of weight and height to determine whether the wearer is under, normal or overweight. This will inform the values of the Limit and Optimisation Constraint Vectors. This is followed by the Basal Metabolic Rate (BMR) which estimates how many calories are spent even without any activity during Multimed Tools Appl (2018) 77:26397–26448 26419 Fig. 26 Bru-Fit’s Analysis screen comparing personal to community data in relation to calories a 24-h period. This represents the minimum amount of energy required to keep the body functioning. The Harris-Benedict and Mifflin equations are commonly used to calculate BMR because of their accuracy. The BMR for men differs to that of women mostly due to their body weight: For men: (10 × weight)+ (6.25 × height) − (5 × age)+5. For women: (10 × weight) + (6.25 × height) − (5 × age) − 161 Table 3 gives the BMI and BMR calculated using parameters extracted from the personal dataset. These values are carried forward to the comparative analysis stage. In addition to the BMI and BMR calculation, calculation of the calories burned during all measured activities with the wearable, i.e. walking, exercising, and sleeping are presented on Fig. 30.Figure 30 reveals that at the peak of walking the number of calories burned are less than the calculated BMR since the amount of energy required to support vital body functions is significantly higher. A sample of a monthly personal dataset is presented on Table 4. The figure shows the total values of the variables relating to the three measured activities and their daily average. Moreover, a detailed distribution for each day of the week, regarding the main activities is shown on Table 5 which reveals the daily effort on a weekly basis with Tuesdays and Fridays peaking at 16%. Exercise is frequently occurring more on Sundays at 20% and sleep duration is at its peak on Wednesdays with 16%. 26420 Multimed Tools Appl (2018) 77:26397–26448 Fig. 27 Bru-Fit’s Analysis screen comparing personal to community data in relation to sleep Figure 31 reveals that when the number of the calories from all activities are added together, Tuesdays and Fridays are the days during which most calories are burned; this resembles the step count distribution. The community dataset has been compiled from 16 different wearers, 8 women and 8 men between the ages of 23 and 31 and it comprise of both qualitative and quantitative data: height (cm), weight (kg), age, gender, step_count, step_calories, steps_distance (km), exercise_calories, exercise_distance (km), exercise_duration (min), sleep_calories and sleep_duration (min). Table 6 shows the BMI and BMR of each wearer. Figures 32 and 33 present the community’s monthly distribution of spent calories during the measured activities with a gender bias. The two figures reveal a similar pattern for males and females. The range of calories spent whilst waking or sleeping is higher than whilst exercising with the maximum calories burned by women being around 16,000 whilst for men 20,000. The number of calories burned whilst sleeping is higher than daily exercising or walking because sleep duration does not vary as much as dailywalking andexercisingdoasFig. 34 shows. Figure 35 reveals that males spend more calories than females whilst sleeping and walking but less whilst exercising. Multimed Tools Appl (2018) 77:26397–26448 26421 Fig. 28 Bru-Fit’s Analysis screen comparing personal to community data in relation to steps Figures 36, 37, 38, 39, 40, 41, 42 and 43 present detailed data analytics produced by JMP PRO. Figure 36 shows the daily mean for walking being 10,622.917 and reaching a maximum of 45,408 and dropping to a minimum of 308. Figure 37 presents the daily mean of calories spent during walking being 387.89 and reaching a maximum value of 1530 and dropping to a minimum of 11. Figure 38 illustrates the daily mean for distance covered ranging between a maximum of 34.5 and a minimum of 0.25. The boxplots indicate that there may be upper or lower limit outliers and the last minimum might be one of these. All three graphs show a positive skewness distribution, which indicate that the lower limits for steps, calories, and distance is higher than the higher limits. The results show that most datasets report between 5000 and 15,000 steps daily. Thus, the number of calories spent fluctuate between 200 and 600. On Fig. 39 the distribution is slightly flatter than a standard normal whereas Figs. 40 and 41 show kurtosis values higher than 3 which indicate a high number of outliers. This number of outliers is a consequence of either forgetting to register the daily exercise activity or not exercising. Figure 42 shows a negative kurtosis of −0.14 indicating a flatter distribution with only one outlier and a low skewness of 0.26 making it symmetrical than the others. Calories burned whilst sleeping are directly proportional to the wearer’s weight, 50 kg to 79 kg. Thus, the distribution is most likely normal. Figure 43 26422 Multimed Tools Appl (2018) 77:26397–26448 Fig. 29 Bru-fit’s Recommendations screen shows a negative skewness with a high kurtosis value since sleep duration does not follow the expected pattern. Correlational analyses of the walking, exercising and sleeping dataset variables are presented on Figs. 44, 45 and 46. Whilst with exercising the correlation is similar to walking, with sleeping the correlation is not as high since the sleeping duration does not normally have a specific pattern, as wearers who exercise more daily are normally expected to sleep for longer periods. However, this is not always the case because a wearer may either have developed a regular sleeping pattern regardless or are unable to sleep for the required amount of time. Table 7 presents a full table of correlations between each variable of the dataset. Table 8 presents a summary of daily activity over the period of one week. For instance, step counts usually rises at weekends whereas exercising rises mid-week and on Sundays and average sleep duration is higher at weekends in comparison to weekdays with 3 additional hours of sleep. The highest daily calorie consumption seems to take place on Fridays as Fig. 47 shows. Figures 48, 49 and 50 map the personal against the community dataset. Figure 48 reveals that the monthly personal step count is higher than the community’s average for females whereas Figs. 49 and 50 reveal that personal exercise and sleep periods are lower. Figure 51 provides a comparison of the calories expended for the monitored activities and it reveals that the calories expended reflects either the effort and or the activity duration. Multimed Tools Appl (2018) 77:26397–26448 26423 Table 2 Step count’s cvs data file com.samsung.health.step_count time_offset end_time speed pkg_name start_time sample count calories distance datauuid deviceuuid update_time create_time 3600000 6/17/16 23:20 1.0366666 com.sec.android.app.shealth 6/17/16 23:19 14 0.51 07.81 c597e372-5b2 n651IFUG7o 6/17/16 23:19 6/17/16 23:19 3600000 6/17/16 23:26 2.4166667 com.sec.android.app.shealth 6/17/16 23:25 16 3.09 11.67 4a104a25-457 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:28 5.0833335 com.sec.android.app.shealth 6/17/16 23:27 27 1.14 31.01 edee5f62-265 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:29 2.9166667 com.sec.android.app.shealth 6/17/16 23:28 35 1.24 33.51 4a2d4896-5c2 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:30 3.9722223 com.sec.android.app.shealth 6/17/16 23:29 23 0.85 23.11 1a47aeef-179 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:31 1.9298451 com.sec.android.app.shealth 6/17/16 23:30 30 1.0099999 24.70 89f9a721-807 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:32 2.1912923 com.sec.android.app.shealth 6/17/16 23:31 19 0.5899999 16.90 a5ec49fd-ba4 n651IFUG7o 6/17/16 23:31 6/17/16 23:31 3600000 6/17/16 23:34 2.8055556 com.sec.android.app.shealth 6/17/16 23:33 6 0.28 06.13 a583f27e-a49 n651IFUG7o 6/17/16 23:33 6/17/16 23:33 3600000 6/17/16 23:35 1.4919225 com.sec.android.app.shealth 6/17/16 23:34 10 0.29 07.38 777c3953-a54 n651IFUG7o 6/17/16 23:34 6/17/16 23:34 3600000 6/17/16 23:36 5.4722223 com.sec.android.app.shealth 6/17/16 23:35 6 0.93 08.10 b675040f-aa2 n651IFUG7o 6/17/16 23:37 6/17/16 23:37 3600000 6/17/16 23:38 3.2222223 com.sec.android.app.shealth 6/17/16 23:37 25 1.85 27.44 99439287-0d1 n651IFUG7o 6/17/16 23:37 6/17/16 23:37 3600000 6/17/16 23:40 1.3333334 com.sec.android.app.shealth 6/17/16 23:39 17 0.58 11.38 7340ab3a-381 n651IFUG7o 6/17/16 23:42 6/17/16 23:42 3600000 6/17/16 23:45 2.5555556 com.sec.android.app.shealth 6/17/16 23:44 54 3.27 48.34 1be95e90-e23 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:46 5.888886 com.sec.android.app.shealth 6/17/16 23:45 8 0.66 11.71 b75e2467-547 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:47 2.1388888 com.sec.android.app.shealth 6/17/16 23:46 38 1.52 34.56 70e3f7b9-8d2 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:48 2.9444444 com.sec.android.app.shealth 6/17/16 23:47 18 0.61 17.74 37da7c16-002 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:49 5.0555553 com.sec.android.app.shealth 6/17/16 23:48 9 1.08 12.95 abb5b8dc-571 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:50 3.0277777 com.sec.android.app.shealth 6/17/16 23:49 23 0.7 20.99 259bb8bd-034 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:51 3.1111112 com.sec.android.app.shealth 6/17/16 23:50 47 1.76 46.13 008c5fe1-068 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:52 2.75 com.sec.android.app.shealth 6/17/16 23:51 32 1.22 30.51 a691ebd0-293 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 26424 Multimed Tools Appl (2018) 77:26397–26448 Table 3 Personal BMI and BMR calculation Height Weight Age Gender BMI BMR 154 54.4 28 female 22.94 1205.5 1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031 Days step_calories exercise_calories sleep_calories Fig. 30 Personal monthly distribution of spent calories during the measured activities Table 4 Personal monthly and average daily stats Step count Step Steps Exercise Exercise Exercise Sleep Sleep calories distance calories distance duration calories duration Per month 445566 12164 290.47 1184 28.88 349 12092 13362 Daily Ave 14373.10 392 9.37 118.40 2.89 34.91 390.05 431.03 Table 5 Personal daily summary on a weekly basis Step Count Step calories Exercise calories Exercise duration Sleep calories Sleep duration Monday 14547 390 120 37 342 378 Tuesday 15776 434 108 31 400 442 Wednesday 10731 294 127 36 421 466 Thursday 14834 401 79 26 362 401 Friday 15771 437 97 27 421 465 Saturday 13305 369 143 43 392 433 Sunday 15328 412 163 48 384 425 Calories Multimed Tools Appl (2018) 77:26397–26448 26425 Fig. 31 Personal weekly pie of calories spent Table 6 Community BMI and BMR Wearer Height Weight Age Gender BMI BMR 1 154 55 28 female 23.19 1211.5 2 159 50 28 female 19.78 1192.75 3 154 55 28 female 23.19 1211.5 4 160 53 30 female 20.70 1219 5 166 60 29 female 21.77 1331.5 6 155 55 28 female 22.89 1217.75 7 160 57 30 female 22.27 1259 8 157 56 29 female 22.72 1235.25 9 177 62 29 male 19.79 1586.25 10 178 80 31 male 25.25 1762.5 11 182 80 28 male 23.89 1808.75 12 180 79 28 male 24.38 1780 13 170 70 23 male 24.15 1802.5 14 174 70.3 30 male 23.22 1645.5 15 176 79 31 male 25.50 1740 16 169 76 30 male 26.61 1671.25 Steps 7795 11929 11447 9271 16148 15038 12445 8020 Exercise 4079 3040 6973 3806 5067 5613 4758 5067 Sleep 12813 12332 13095 12782 15513 12501 14996 13443 Subject Fig. 32 Community monthly distribution of spent calories during the measured activities for females Calories 26426 Multimed Tools Appl (2018) 77:26397–26448 Steps 13927 9098 7721 11825 18675 13153 16828 18910 Exercise 1179 2254 7565 5572 0 2782 0 8368 Sleep 15688 20038 17250 17201 18755 16977 13685 16936 Subject Fig. 33 Community monthly distribution of spent calories during the measured activities for males 123456789 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Days Exercise Sleep Fig. 34 Community monthly duration of exercising and sleep for women Fig. 35 Community daily calories spent: males versus females Fig. 36 Daily data analytics for step count Duraon Calories Multimed Tools Appl (2018) 77:26397–26448 26427 Fig. 37 Daily data analytics for calories burnt during walking Fig. 38 Daily data analytics for distance covered whilst walking Figure 52 which summarises the monitored activities, reveals a personal pattern that matches on some days only that of the community and may also suggest a sedentary lifestyle. Table 9 summarises the personal results and Figs. 53, 54 and 55 provide a comparison to the relevant community dataset and trends. The data analytics models on Figs. 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46 and Table 7 are used by GA and PO during the second part of Step 5 to generate an initial population of Fig. 39 Daily data analytics for calories burnt during exercising 26428 Multimed Tools Appl (2018) 77:26397–26448 Fig. 40 Daily data analytics for distance covered whilst exercising Fig. 41 Daily data analytics for exercise duration chromosomes and to source limit and optimisation constraints. The recommendations to the wearer will be sourced from the non-dominated chromosome that evolves, like the one which appears on Fig. 4. During Step 6 the laptop transfers the daily models produced by JMP PRO and the recommendation generated by the GA and PO to the smartphone via the tracker’s cloud servers. During Step 7, the data models and recommendation are visualised using Fig. 42 Daily data analytics for calories burnt during sleeping Multimed Tools Appl (2018) 77:26397–26448 26429 Fig. 43 Daily data analytics for sleeping duration FitNalyser to support the wearer with monitoring their daily goals and activities and their weekly and monthly performance. The analysis of the personal dataset collated on the wearable device has picked up that the daily average of expended calories is approximately Fig. 44 Correlation between walking variables 26430 Multimed Tools Appl (2018) 77:26397–26448 25% less than the daily average a person with that BMI is expected to burn which in the long- term will result in weight gain and the chromosome and subsequent recommendation generated addresses that. There is clearly room for improvement in relation to calorie burning and the generated recommendation visualised on Fig. 14 reflects that. The wearer may wish to consider as much evidence behind this recommendation as possible and may also wish to consider their performance in relation to that of their peers. Figures 8 through to 13 provide all the daily and weekly evidence the wearer may seek. During Step 8, the wearer, having considered the evidence and recommendation, may revisit their current daily goals and activities, and set up new activity goals, e.g. reduce the number of hours spent sleeping, increase the number of hours spent walking, increase the step count, to move from sedentary to lightly active or active. During Step 9, the smart phone synchronises with the wearable to auto-update any revised goals. Not all wearables facilitate that but both wearables used in our experiment allow that. Fig. 45 Correlation between exercising variables Multimed Tools Appl (2018) 77:26397–26448 26431 4.2 Bru-Fit Steps 1, 2 and 3 are like FitNalyser’s. Fitbit tracks sleep through movement using one of two modes, normal and sensitive. It records, as when awake, significant movements only in normal mode and nearly all movements in sensitive mode: t ¼ t −t asleep Total tracked awake asleep Sleep Efficiency ¼ 100 t þ t þ t asleep restless awoken during sleep Fitbit tracks the Heart rate through blood volume variation using LED lights onto the skin. It records the intensity of exercising as a percentage of: Fig. 46 Correlation between sleeping variables 26432 Multimed Tools Appl (2018) 77:26397–26448 Table 7 Full table of correlations Correlations step_count step_calories steps_distance exercise_calories exercise_distance exercise_duration sleep_calories sleep_duration step_count 1.0000 0.9180 0.9878 0.3004 0.3302 0.3482 −0.1174 0.0700 step_calories 0.9180 1.0000 0.9296 0.2953 0.2963 0.3193 0.0015 0.0228 steps_distance 0.9878 0.9296 1.0000 0.2999 0.3240 0.3369 −0.0921 0.0589 exercise_calories 0.3004 0.2953 0.2999 1.0000 0.9780 0.9373 −0.0908 −0.1317 exercise_distance 0.3302 0.2963 0.3240 0.9780 1.0000 0.9746 −0.1692 −0.0989 exercise_duration 0.3482 0.3193 0.3369 0.9373 0.9746 1.0000 −0.1652 −0.0834 sleep_calories −0.1174 0.0015 −0.0921 −0.0908 −0.1692 −0.1652 1.0000 0.5124 sleep_duration 0.0700 0.0228 0.0589 −0.1317 −0.0989 −0.0834 0.5124 1.0000 Multimed Tools Appl (2018) 77:26397–26448 26433 Table 8 Dailyactivitytrends Step Step Steps Exercise Exercise Exercise Sleep Sleep count calories distance calories distance (km) duration calories Duration (km) (min) (min) Monday 8580 315 6.44 193 3.80 47 382 359 Tuesday 8261 317 6.29 210 4.19 52 377 359 Wednesday 8572 317 6.38 164 3.16 40 391 370 Thursday 8555 308 6.30 184 3.48 45 403 377 Friday 13731 502 10.11 118 2.09 27 632 595 Saturday 13169 475 9.76 186 3.02 43 618 576 Sunday 12968 459 9.64 216 4.60 55 624 579 Fig. 47 Daily calories burned 400000 355264 300000 women men main subject step_count Fig. 48 Monthly step count 26434 Multimed Tools Appl (2018) 77:26397–26448 women men main subject exercise_duraon (min) Fig. 49 Monthly exercise period 14000 women men main subject sleep_duraon (min) Fig. 50 Monthly sleep period sleep_calories exercise_calories step_calories 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 main subject men women Fig. 51 Monthly comparison of calories burnt Multimed Tools Appl (2018) 77:26397–26448 26435 Step count Exercise duraon Sleep duraon Days Main Subject Dataset Main Subject Dataset Main Subject Dataset 8580 47 359 Monday 14547 37 378 8261 52 359 Tuesday 15776 31 442 Wednesday 10731 8572 36 40 466 370 Thursday 14834 8555 26 45 401 377 13731 27 595 Friday 15771 27 465 13169 43 576 Saturday 13305 43 433 12968 55 579 Sunday 15328 48 425 Fig. 52 Personal against community trends Table 9 Personal results Gender Female Age 28 years old Weight 54.4 kg Height 154 cm BMI 22.94 (Healthy weight) Daily calories burnt 1173.5 Activity level Sedentary Daily average step count 14373 Daily average exercise duration 35 min (2 days per week) Daily average sleep duration 431 min Personal trends in relation to community step count = normal; exercise duration = poor; sleep duration = group normal Maximum rate ¼ 220−age This rate decreases with age, thus seeking to acheve the same level of intensity across different age groups requires different amounts of effort. Table 10 shows Fitbit’s five intensity zones as a percentage of Heart rate. Fitbit tracks activity and calories burned through exercise intensity using Steps, Floors and Heart Rate along with Metabolic Equivalent (MET), BMR, weight and height. BMR values 445566 14373 step count step count 355264 11460 0 200000 400000 0 5000 10000 15000 20000 main subject women main subject women Fig. 53 Personal against community step count 26436 Multimed Tools Appl (2018) 77:26397–26448 349 35 exercise duraon exercise duraon (min) (min) 1379 58 0 500 1000 1500 020 40 60 80 main subject women main subject women Fig. 54 Personal against community exercise duration 13362 431 sleep duraon (min) sleep duraon (min) 14664 473 12500 13500 14500 400 420 440 460 480 main subject women main subject women Fig. 55 Personal against community sleep duration Table 10 Fitbit heart rate zones Effort Effect Peak zone 85% of maximum rate High-intensity exercise zone Cardio 70 to 84% of maximum rate Medium-to-high intensity exercise zone Fat burn zone 50 to 69% of maximum rate Low-to-medium intensity exercise zone Out of zone Below 50% of maximum rate Not enough to be considered exercise Resting 60-80 beats per minute Awake, calm, comfortable, not recently exerted yourself Table 11 Fitbit parameters tracked Parameter Variable Sleep Time in bed Actual Sleep Time Activities Minutes Sedentary Minutes Lightly Active Minutes Fairly Active MinutesVeryActive Steps Number of Steps Floors Number of Floors Calories Number of Calories burned Multimed Tools Appl (2018) 77:26397–26448 26437 Fig. 56 Participants by age usually account for at least half of the calories burnt in a day and this is estimated based on gender, age, height, and weight. Table 11 shows the Fitbit parameters tracked. A significant difference between Bru-Fit and FitNalyser takes place during Step 4, during which a Fitbit open script executes the transfer to a laptop via Fitbit’s cloud servers of the raw dataset which hasbeenpackagedintoacvsfile. Step 5 is like FitNalyser but the wearer and the peer group comprise different wearers. The community group comprise of 14 different wearers, 6 women and 8 men and their dataset includes both qualitative and quantitative data: Age, gender, height, weight, BMI, calorie stats (maximum, minimum, average, daily, weekly, monthly), activity stats (level, frequency), step stats, floor stats, distance, sleep stats (daily patterns, efficiency). Figures 56 and 57 classify participants by age and gender respectively. Figure 58 maps the daily average of number of calories spent across the community. The figure reveals a mixture of distribution across the Fig. 57 Participants by gender 26438 Multimed Tools Appl (2018) 77:26397–26448 Fig. 58 Daily average of number of calories spent across the community Fig. 59 Parameter correlation Fig. 60 Daily calories spent: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26439 Fig. 61 Daily calories spent: Personal vs community females Fig. 62 Daily calories spent: Personal vs Lightly Active community members Fig. 63 Daily calories spent: Personal vs Lightly Active community females 26440 Multimed Tools Appl (2018) 77:26397–26448 Fig. 64 Daily calories spent: Personal vs community Fig. 65 Daily average calories spent: Personal vs Active community Fig. 66 Daily sleep: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26441 Fig. 67 Daily sleep: Personal vs community females Fig. 68 Daily sleep: Personal vs Lightly Active community members Fig. 69 Daily sleep: Personal vs Lightly Active community females 26442 Multimed Tools Appl (2018) 77:26397–26448 Fig. 70 Daily sleep: Personal vs community Fig. 71 Daily average sleep: Personal vs Active community Fig. 72 Daily steps: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26443 Fig. 73 Daily steps: Personal vs Lightly Active community members Fig. 74 Daily steps: Personal vs community females Fig. 75 Daily steps: Personal vs Lightly Active community females 26444 Multimed Tools Appl (2018) 77:26397–26448 Fig. 76 Daily steps: Personal vs community community, positively skewed because of low number of calories spent and normal. This splits the community into sedentary or lightly active. Figure 59 shows parameter correlation. Red highlights indicate no positive correlation has been established whereas blue highlights indicate positive correlation. The correlation becomes stronger as the shade becomes darker. Unlike with FitNalyser, the correlation map does not indicate a correlation between daily activity and sleep. Figures 60, 61, 62, 63, 64 and 65 compare the number of calories spent between the wearer and the community datasets. The results are attributed to females normally having a higher MBR than males because of having, in general, a higher percentage of fat to muscle in comparison to males who have, in general, a higher percentage of muscle to fat. Figures 66, 67, 68, 69, 70 and 71 show the outcome of tracking sleep activity. The figures reveal a higher sleep activity in males than in females. Figures 72, 73, 74, 75, 76 and 77 show the outcome of tracking daily steps through walking. These figures reveal a higher walking activity, and thus steps, by males than females in sharp contrast to past studies which report the reverse. A second significant difference between Bru-Fit and FitNalyser takes place during Step 6, at which a Fitbit open script executes the transfer back to the smartphone via Fitbit’s servers of Fig. 77 Daily average steps: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26445 the models produced by JMP PRO and the recommendations generated by the GA and PO. At Step 7, the analysis of the personal dataset collated on the wearable device has picked up that that the wearer has exceeded their daily goals with midweek being the most active days and Sundays the least active with walking being the most popular activity. Daily tracking of sleep efficiency has picked up that this rises during weekdays but declines at weekends despite a higher sleep activity during weekends. There were also significant variations in sleep efficien- cy across community members belonging to the same age group. This may be attributed to circadian rhythm sleep disorder CRSD which is common among younger generations. The results also reveal that males are more likely to be overweight and fall into the sedentary range than females are. Unlike FitNalyser the figures do not correlate sleep and daily activity. There is clearly room for improvement in relation to physical activity and sleeping at weekends and the generated recommendation visualised on Fig. 29 reflects that. The wearer may also wish to consider as much evidence behind this recommendation as possible and may also wish to consider their performance in relation to that of their peers. Figures 17 through to 28 provide all the daily and weekly evidence the wearer may seek. During Step 8, the wearer, having considered the evidence and recommendation, may revisit their current daily goals and activities, and set up new activity goals, e.g. reduce the number of hours spent sleeping and increase the number of hours spent walking. During Step 9, the smart phone synchronises with the wearable to auto-update any revised goals. Figure 17 shows how goals set on this wearable are visualised with Bru-Fit. 5 Concluding discussion The paper demonstrates how wearable technology may be used beyond the hype to improve personal lifestyles by suggesting areas where generating a change in daily or routine habits may lead to a healthier lifestyle. Currently, such cycles appear to be driven by irrelevant or incomplete information which may not be easy to interpret but under peer pressure and the fear of exclusion, wearers adopt ad hoc change regardless. Our research and development work demonstrates that a planned approach to data collection, analysis and visualisation coupled with a machine learning recommender approach may transpose wearer decisions to be informed and that the use of peer group data is for generating correlated recommendations not for applying social pressure. A mobile app might not be able to support the entire cycle and consequently backbone computing support is required. The analysis has also thrown some accepted truths into dispute, i.e. the correlation between exercising and sleeping is very low suggesting the two are more independent than once thought. This might be the result of exogenous factors such as workstyle or underlining health conditions such as cardiovascular diseases. Areas for further research and development naturally include, in addition to the rising need for standardisation, the addition of more sensors to track further personal fitness and alarms to warn of the possibility of ambulatory incidents, an approach recently put on trial by the National Health Service (NHS) in the UK. This would raise the significance of the personal recommendations, especially in the health sector, but this would require a more extensive clinical trial as it would have serious repercussions to health and fitness. Finally, suggestions of automatic readjustment of personal goals following a recommendation, i.e. without wearer intervention, has divided the community after clinicians recommend that this should only be encouraged under clinical supervision. 26446 Multimed Tools Appl (2018) 77:26397–26448 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and repro- duction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. References 1. Anaya LHS, Alsadoon A, Costadopoulos N, Prasad PWC (2017) Ethical Implications of User Perceptions of Wearable Devices, Sci Eng Ethics, online first 2. Argus Insights Inc. (2016) Wearables demand report. Available at: http://www.argusinsights.com/wearables- demand-14-15 3. ARM Ltd (2014) Advanced Architectures and Technologies for the Development of Wearable Devices. White Paper, pp. 11 4. Bandodkar AJ, Jeerapan I, Wang J (2016) Wearable chemical sensors: present challenges and future prospects. ACS Sensors 1(5):464–482 5. Chen M, Ma Y, Song J, Lai C, Hu B (2016) Smart clothing: connecting human with clouds and big data for sustainable health monitoring. Mobile Networks and Applications 21:1–21 6. Delgado-Gonzalo R, Parak J, Tarniceriu A, Renevey P, Bertschi, M, Korhonen I (2015) Evaluation of accuracy and reliability of PulseOn optical heart rate monitoring device. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Milan, pp 430-433 7. Demir E, Demir K, Odabaşı S, Odabaşı F (2016) A challenge for higher education: Wearable technology for fashion design departments. World Journal on Educational Technology 8(1):69–77 8. Guesmi C, Mabrouk IB, Had L, Talb L, Gharsallah A (2016) G-Shaped Dual Band Wearable Button Antenna for ISM Band Applications. International Journal on Communications Antenna and Propagations 6(1):39–43 9. Hill C (2015) Wearables–the future of biometric technology? Biometric Technology Today 2015(8):5–9 10. IDC Research Inc. (2016) IDC Forecasts Wearables Shipments to Reach 213.6 Million Units Worldwide in 2020 with Watches and Wristbands Driving Volume While Clothing and Eyewear Gain Traction. Available at: http://www.idc.com/getdoc.jsp?containerId=prUS41530816 11. Kalinauckas A (2015a) Wearable technology. Eng Technol 10(4):36–43 12. Kalinauckas A (2015b) Wearable technology: smarten up your outfit - E & T Magazine. Available at: http://eandt.theiet.org/magazine/2015/04/wearable-technology.cfm 13. Kolodzey L, Grantcharov PD, Rivas H, Schijven MP, Grantcharov TP (2017) Wearable technology in the operating room: a systematic review. BMJ Innovations Journal 3:55–63 14. Ledger D (2016) Inside Wearables – Part 1,2,3. Endeavour Partners. Available at: http://endeavourpartners. net/inside-wearables-part-3 15. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115 16. Matzeu G, Florea L, Diamond D (2015) Advances in wearable chemical sensor design for monitoring biological fluids. Sensors Actuators B Chem 211:403–418 17. News.samsung.com (2016) Samsung Expands Industry-Leading Wearable Line with Samsung Gear Fit. Available at: https://news.samsung.com/global/samsung-expands-industry-leading-wearable-line-with-samsung-gear-fit 18. Parak J, Tarniceriu A, Renevey P, Bertschi M, Delgado-Gonzalo R, Korhonen I (2015) Evaluation of the beat-to-beat detection accuracy of PulseOn wearable optical heart rate monitor. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Milan, pp 8099-8102 19. Park S, Chung K, Jayaraman S (2014) Wearables: Fundamentals, advancements, and a roadmap for the future. Wearable Sensors: Fundamentals, Implementation and Applications. Encyclopedia of Information Science and Technology, pp. 1-23 20. Patel MS, Asch DA, Volpp KG (2015) Wearable devices as facilitators, not drivers, of health behavior change. JAMA 313(5):459–460 21. Rettberg JW (2016) Seeing ourselves through technology: How we use selfies, blogs and wearable devices to see and shape ourselves. Palgrave Macmillan 22. Richter F (2016) Infographic: The Global Wearables Market in 2015. Statista Infographics. Available at: https://www.statista.com/chart/4423/wearable-device-shipments-2015 23. Robson K, Pitt L, Kietzmann J, Halvorson W, Wallstrom A (2016) Wearable technology: trends and opportunities for organizations. In: Celebrating America’s Pastimes: Baseball, Hot Dogs, Apple Pie and Marketing? Springer, pp. 801-801 Multimed Tools Appl (2018) 77:26397–26448 26447 24. Salah H, Maclntosh E, Rajakulendran N (2015) Wearable Tech: Leveraging Canadian Innovation to Improve Health. MaRS Market Insights 25. Schüll ND (2016) Data for life: Wearable technology and the design of self-care, BioSocieties 26. Shelgikar AV, Anderson P, Stephens MR (2016) Sleep tracking, wearable technology, and opportunities for research and clinical care. CHEST Journal 50(3):732–743 27. Shih PC, Han K, Poole ES, Rosson MB, Carroll JM (2015) Use and adoption challenges of wearable activity trackers, iConference 2015 Proceedings, p. 12 28. Shoaib M, Scholten H, Havinga PJM (2013) Towards physical activity recognition using smartphone sensors. 2013 IEEE 10th International Conference on Ubiquitous Intelligence & Computing and 2013 IEEE 10th International Conference on Autonomic & Trusted Computing, Sorrento, pp 80–87 29. Sofokleous AA, Angelides MC (2008) DCAF: an MPEG-21 dynamic content adaptation framework, Multimedia Tools and Applications 40:151–182 30. Sofokleous AA, Angelides MC (2009) Dynamic selection of a video content adaptation strategy from a Pareto front, The Computer Journal 52:413–428 31. Stoppa M, Chiolerio A (2014) Wearable electronics and smart textiles: a critical review. Sensors 14(7): 11957–11992 32. Thierer AD (2015) The internet of things and wearable technology: Addressing privacy and security concerns without derailing innovation. Adam Thierer, The Internet of Things and Wearable Technology: Addressing Privacy and Security Concerns without Derailing Innovation, pp. 21 33. Viscusi S (2016) The Current and Future State of Wearables: Trends, Attitudes, and Total Domination. Available at: http://www.wearabletechworld.com/topics/wearable-tech/articles/417581-current-future-state- wearables-trends-attitudes-total-domination 34. Woolley M (2015) Woolley’s Wearables – Android & Bluetooth - Bluetooth Blog. Bluetooth Blog. Available at: http://blog.bluetooth.com/woolleys-wearables-android-bluetooth 35. Yang J, Wei D, Tang L, Song X, Luo W, Chu J, Gao T, Shi H, Du C (2015) Wearable temperature sensor based on graphene nanowalls. RSC Adv 5(32):25609–25615 36. Zachariah T, Klugman N, Campbell B, Adkins J, Jackson N, Dutta P (2015) The internet of things has a gateway problem. In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, Santa Fe, pp 27–32. https://doi.org/10.1145/2699343.2699344 Marios C. Angelides holds a Chair in Computing at Brunel University London. He is a Chartered Engineer (CEng) and a Chartered Fellow of the British Computer Society (FBCS CITP). He holds a BSc and a PhD both from the London School of Economics (LSE) where he also began his academic career as a lecturer more than 25 years ago. For over two decades, he has been researching multimedia communications with a focus on the application of MPEG standards and game theory through evolutionary computation. In 1995, Kluwer published his first book, BMultimedia Information Systems^. In 2011, Wiley published his edited book on BMPEG Applications^ and in 2014 IEEE/Wiley published his edited book on BDigital Games^. 26448 Multimed Tools Appl (2018) 77:26397–26448 Lissette Andrea Cabello Wilson is a lecturer in the Department of Computer Engineering at Escuela Superior Politecnica del Litoral (ESPOL) in Ecuador. She holds a BSc in Telematics Engineering from ESPOL and an MSc in Computer Communication Networks from Brunel University London where she is currently a research associate in the Electronics and Computer Engineering Department. She is also an associate lecturer in the Universidad de Guayaquil where she has participated in the accreditation process to obtain the CEAACES certification. Her research interests include network security and virtualization. Paola Burneo Echeverría works in industry specializing in VOIP networks and applications. She holds a BSc in Electronics Engineering (Telecommunications) from the Universidad Politécnica Salesiana in Cuenca in Ecuador and an MSc in Computer Communication Networks from Brunel University London where she is currently a research associate in the Electronics and Computer Engineering Department. Her research interests include data mining and software defined networks. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Multimedia Tools and Applications Springer Journals

Wearable data analysis, visualisation and recommendations on the go using android middleware

Free
52 pages

Loading next page...
 
/lp/springer_journal/wearable-data-analysis-visualisation-and-recommendations-on-the-go-072Yz0CcOX
Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s)
Subject
Computer Science; Multimedia Information Systems; Computer Communication Networks; Data Structures, Cryptology and Information Theory; Special Purpose and Application-Based Systems
ISSN
1380-7501
eISSN
1573-7721
D.O.I.
10.1007/s11042-018-5867-y
Publisher site
See Article on Publisher Site

Abstract

Multimed Tools Appl (2018) 77:26397–26448 https://doi.org/10.1007/s11042-018-5867-y Wearable data analysis, visualisation and recommendations on the go using android middleware 1 1 Marios C. Angelides & Lissette Andrea Cabello Wilson & Paola Liliana Burneo Echeverría Received: 2 October 2017 /Revised: 1 March 2018 /Accepted: 4 March 2018 / Published online: 4 June 2018 The Author(s) 2018 Abstract Wearable technology comes with the promise of improving one’s lifestyles thru data mining of their physiological condition. The potential to generate a change in daily or routine habits thru these devices leaves little doubt. Whilst the hardware capabilities of wearables have evolved rapidly, software apps that interpret and present the physiological data and make recommendations in a simple, clear and meaningful way have not followed a similar pattern of evolution. Existing fitness apps provide routinely some information to the wearer by mining personal data but the subsequent analysis is limited to supporting ad hoc personal goals. The information and recom- mendations presented are often either not entirely relevant or incomplete and often not easy to interpret by the wearer. The primary motivation behind this research is to address this wearable technology software challenge by developing a middleware mobile app that is supported by data analytics and machine learning to assist with interpretation of wearer data and with making of personal lifestyle improvement recommendations on the go which may then be used to feedback to the wearer’s daily goals and activities. The secondary motivation is to correlate and compare with trends in the wearer’s peer community. . . Keywords Wearable technology Visual information systems Machine learning * Marios C. Angelides marios.angelides@brunel.ac.uk Lissette Andrea Cabello Wilson lcabello@fiec.espol.edu.ec Department of Electronic and Computer Engineering, College of Engineering Design and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK 26398 Multimed Tools Appl (2018) 77:26397–26448 1 Introduction Google’s Verily Life Sciences have recently unveiled an ordinary looking health tracking Bstudy watch^ to unobtrusively and continuously collect physiological data from the wearer, such as their heart rate, electrocardiograms (ECG), movement data, their skin’s electrical conductance as well as ambient light and sound. The Google watch does not currently allow the wearer to see their health data, only the date, time, and some instructions. The long-term vision of Verily is to unravel biomarkers through tracking thousands of healthy people especially when they fall ill. Although the watch features a processor that manages and encrypts its wearer’s data and its software is updatable over-the-air, Verily is not currently marketing the watch as a medical device, for which they will need FDA approval, but as a clinical and observational tool for scalable collection of rich and complex data sets. The watch’s battery lasts for up to a week without a charge and it stores Braw^ data produced over the same amount of time, so it does not have to be synched as frequently as other watches. The inclusion of an ECG which Verily regards as their biggest technical novelty can reveal heart abnormalities. Such measurements are normally taken in hospitals with several stick-on electrodes but the Google watch picks up a lower-resolution signal from just two electrical contacts when the wearer grasps the metal bezel with their other hand. The watch is being deployed in a study in Europe whose aim is to track the progression of Parkinson’sdisease among patients diagnosed with the disease. Uninterrupted long-term sleep and heart readings are invaluable in monitoring the progression of Parkinson’s and cannot be monitored in hospitals, especially the patients’ sleep patterns. Verily argues that the watch’s scope can be extended through the inclusion of additional sensors that will help generate additional bio- feedback thus shaping its place in the Internet of Things (IoT) landscape [8, 11, 12]. Having continuous and uninterrupted access to such personal data in real time may help with the diagnosis of underlying conditions on the go, as maybe the case with Google’s watch, and making personal recommendations on life style improvements, otherwise, by identifying personal preferences and behaviour including food habits, sleep patterns, and daily activity schedules. It is widely acknowledged that all these devices can motivate individuals to change habits towards a better health or lifestyle. For instance, they are being considered in the workplace to monitor workers’ activities and control schedules and as they hold consumer’s health data, these devices are also a point of interest for physicians and health insurers [1, 10, 21]. Samsung, Fitbit, Apple and Sony have long joined the race in wearable technology hardware but according to Verily, the next but necessary stage in the evolution is the development of software algorithms that can Binterpret^ the wearer’s raw data, thus making accurate diagnoses and personalised recommendations, where possible, often by comparing the wearer’s data against the data of other wearers in the same peer group [3, 14]. As these devices cover a broad range of wearers of different ages, physical conditions and lifestyles, it is important that the presented information is as useful to all wearers as the wearable hardware with which it has been recorded. This would in turn inform the development of personalised immersive and alternative reality environments. The focus of this paper is to address the wearable technology software challenge by developing such a personal immersive environment that collects, analyses and visualises personal wearer data mined with a wearable device and which integrates a machine learning approach to making personal recommendations for lifestyle im- provements on the go in close consideration of trends in the wearer’s peer community. These lifestyle improvement recommendations may then be used to feedback to the Multimed Tools Appl (2018) 77:26397–26448 26399 Table 1 Issues with wearables Ethical/Legal Economic Social Technological Intellectual Property √ Data Protection √ Privacy √ Safety √ Regulation √ Innovation √ Price √ Choice √ Connectivity √ Integration √ Inclusion √ Networking √ Systems Interoperability √ Technical Standards √ Wireless Connectivity √ Data Security √ Application areas √√ Usability √ Real-time √ wearer’s daily goals and activities. As access to Google watch’s wearer data is currently restricted, the authors showcase their approach with a Samsung Gear Fit and Fitbit. The environment is developed as a mobile middleware app aimed at offering personal recommendations on the go for improving personal lifestyles. To show how this aim has been achieved, the rest of the paper is organised as follows: Section 2 presents a related research review, section 3 details the mobile apps developed, including our data analysis and machine learning approaches, and section 4 presents our experimentation. Section 5 concludes. Fig. 1 Social connectivity [34] 26400 Multimed Tools Appl (2018) 77:26397–26448 Decision Making 9 7 Analysed Data & Goal Recommendaon Synchronisaon Visualisaon 1 6 Analysed Data & Configuraon Recommendaon & Goal Setup Forwarding 2 5 Raw Data Analysis Tracking & Recommendaon Acvity Generaon Data Raw Data Synchronisaon Forwarding Fig. 2 A flowchart of wearable data analysis, visualisation and recommendation 2 Related research review Table 1 shows the range of issues reported in literature and arising from our research review [1–14, 16, 18–27, 31–36]. The findings have been grouped under four categories, namely ethical/legal, economic, social and technological. However, the boundaries between these categories are fuzzy since discussion of these issues has been from various perspectives. This Fig. 3 A flowchart of comparative analysis during steps 5 and 6 Multimed Tools Appl (2018) 77:26397–26448 26401 Chromosome GA Inial Populaon Generaon Chromosome Selecon (e.g. Roulee) Chromosome Crossover & Mutaon Chromosome Chromosome Verificaon Full Decision Vector Limit Constraint Vector Optimisation Constraint Vector Gene Value Limit Value Optimisation Value Chromosome PO Non- & Dominated Idenficaon Sleep Duration g1 480 431 L1 Sleep Calories g2 390 O1 300 Chromosome Non- & Dominated Fitness Value Exercise Duration g3 O2 30 Exercise Distance g4 3 O3 3 Chromosome Exercise Calories g5 318 O4 300 Non-Dominated Step Count g6 14373 O5 10000 Steps Distance g7 10 O6 10 Step Calories g8 392 O7 300 Fig. 4 A flowchart of GA and PO deployment during steps 5 and 6 and a typical chromosome Fig. 5 FitNalyser opening screen 26402 Multimed Tools Appl (2018) 77:26397–26448 Fig. 6 FitNalyser Profile screen is the result of this subject area being very multidisciplinary in nature whereby contributions have been made from different fields. Whilst the focus of our research review has been primarily software apps that support the wearer in their daily use of the wearables, which reportedly is now at the core of wearable software evolution, what has been reported so far in literature in relation to our objective are software apps that support and fine-tune the functionality of the wearable technology, e.g. energy consumption or display resolution. An Argus Insights Report [2] argues that whilst wearable hardware has been improving to an evolving standard, the same cannot be said for wearable software apps that interpret the data such devices collect. The report suggests that a clear gap exists between devices and such software apps that may enhance an aspect of the wearer’s lifestyle such as responsive learning or interactive gaming. Despite that, social connectivity through wearables has been on the rise where there is a common goal such as fitness or gaming. Figure 1 illustrates an Android wearable device linking up with a smartphone through Bluetooth and communicating with the cloud and other wearers who do the same. Therefore, we have had to consider such software app developments in closely related subjects, i.e. for analysing and presenting data collected from single or multiple sensors like those that define the wearable technology functionality; for example, optical sensors such as those for heart rate monitoring through measurement Multimed Tools Appl (2018) 77:26397–26448 26403 Fig. 7 FitNalyser My Data screen of blood flow changes, chemical sensors such as those for hydration monitoring through real-time measurement of sodium and potassium levels in body sweat, temperature sensors for monitoring the ambient temperature or impact sensors for monitoring vibration. Liu et al. [15] paint a landscape of human activity recognition sourced from past landscapes that includes sensor-based approaches that shed light on current progress which is still slow and not reaching far enough in satisfying the wearer needs. Shoaib et al. [28] present an android smartphone app that collects data using a Samsung Galaxy S2 smartphone’s accelerometer, gyroscope and magnetometer sensors to collect data on physical activities such as walking, running, sitting, standing, walking upstairs or down- stairs and then analysing this data using WEKA’s machine learning tool, with mixed success. The app typifies the approach most authors report, i.e. the use of one or more sensors to collect data and then correlate the data to one or more physical activities. Liu et al. [15]surveyseveral approaches and apps that follow this approach to collect data and then use a variety of techniques to interpret the data such as Neural Nets, Bayesian Networks, Decision tables and trees, etc. Several limitations are reported and arise with existing approaches and android apps that are deployed to support the wearer in revisiting their fitness goals, for example. Wearers often perform multiple activities at the same time or may change, 26404 Multimed Tools Appl (2018) 77:26397–26448 Fig. 8 FitNalyser step count screen planned or unplanned, from one activity to another; for instance, part of a wearer’s running course may involve going up or down a set of stairs. How is data correlated to each activity in such a case? As discussed earlier wearables are increasingly used as social connectivity tools, thus wearers often set their goals and compare their performance against that of their peer group. Current approaches and android apps only serve the wearer. How can wearer initial or revised fitness goals and perfor- mance data, for example, be set and compared to that of their peer group? Current approaches and android apps focus primarily on interpreting past performance and presenting the results to the wearer. How can past wearer and peer group performance be used to make on-the-go recommendations to the wearer? In the next section, we present an android app that addresses the above shortcomings. 3 Wearable data analysis, visualisation and recommendation We have used Mobincube to develop two versions of the android middleware app for Samsung smartphones [17]. One version is entitled FitNalyser and the other Bru-Fit. We have integrated JMP PRO with MatLab’s Genetic Algorithms (GA) and Pareto Optimality (PO) to analyse and Multimed Tools Appl (2018) 77:26397–26448 26405 Fig. 9 FitNalyser exercise screen compare the wearer’s data to peer community data with the former and to use the result to generate recommendations that may support the wearer with monitoring their daily goals and activities with the latter. Figure 2 shows a flowchart of the use of a wearable device, the analysis and visualisation of the wearer activity data it records and the recommendations generated using a machine learning technique to support the wearer with their monitoring of their daily goals and activities. The wearer configures their wearable and sets initial goals (1), the wearable sensors track the wearer’s daily activities and record data (2), the wearable synchronises with a smart device to transfer the wearable data recorded (3), the raw data is forwarded by the smart phone to the wearable data server (4) from which it is retrieved and analysed in comparison to community data to generate personal recommendations (5), the results of the analysis and the recommendations are forwarded to the smart device (6) where they are visualised and offered to the wearer (7) so that they may assist them with monitoring their daily goals and activities (8). The smart phone synchronises with the wearable to auto-update any revised goals (9). Iterating through steps 1 to 4, then 8 and 9 is the standard process with almost all wearable devices when paired with a smart phone. In our work, we introduce three additional steps, i.e. 5, 6 and 7, that address the shortcomings highlighted at the end of the previous section 26406 Multimed Tools Appl (2018) 77:26397–26448 Fig. 10 FitNalyser sleep screen and represent our novel contribution to research and development. JMP PRO and MatLab’s GA and PO are deployed at steps 5 and 6, and either FitNalyser or Bru-Fit are deployed at step 7. 3.1 Steps 5 and 6: Comparative analysis and recommendation generation Figure 3 shows a flowchart of the process of comparative analysis with JMP PRO during steps 5 and 6. JMP Pro has been selected for raw wearable data analysis and comparison to peer data for many reasons. First, it enables predictive modelling through the set of techniques it deploys, e.g. decision trees, bootstrap forest, Naive Bayes and neural networks. Stats emerge with these techniques, e.g. the average influence of additional hours of sleeping on calorie burning, residual variation of number of steps mid-week and at weekends, all of which help with model prediction accuracy even with missing or incomplete data. Second, it enables cross-validation through the set of techniques it uses, e.g. data partitioning or holdback. Cross- validation is not only invaluable when comparing wearer data to peer data but also building a predictive model that is not based on a single wearer sample which in turn avoids over-fitting. Third, it allows model comparison through common quality measures, e.g. R2, ROC, AUC, etc. This helps with selecting a model with the best goodness of fit, parsimony and cross- Multimed Tools Appl (2018) 77:26397–26448 26407 Fig. 11 FitNalyser Community Data screen Fig. 12 FitNalyser Community Data stats for step count, exercise and sleep 26408 Multimed Tools Appl (2018) 77:26397–26448 Fig. 13 FitNalyser Analysis screen validation as input to the GA. Fourth, it uses generalized regression through a set of regularization techniques such as Ridge, Lasso, adaptive Lasso, Elastic Net, adaptive Elastic Net that help overcome the biases that arise with strongly correlated predictors, e.g. number of hours sleeping and calorie burning which may result in over-fitting. This approach helps with building a diverse predictive model that may include data with many outliers, or skewed data. Figure 4 shows a flowchart of MatLab’s GA and PO deployment during steps 5 and 6. Generating an optimal recommendation D for each wearer becomes a multi-optimisation problem (MOP) which seeks to optimise objectives ⋃ O ðÞ D ≤0 subject to limit constraints d¼1 ! ! ! ! ⋃ L ðÞ D ≤0: Fx ¼ f x ……f x where x is an n-dimensional decision vari- d 1 k d¼1 able vector x ¼ x ;:…; x . The objectives and limit constraints are sourced from the best 1 n goodness of fit wearer model produced by JMP PRO in comparison to peer community data, e.g. steps per day, hours sleeping per night, hours exercising per day. Sofokleous and Angelides [29] suggest the use of GAs and Pareto Optimality (PO). The GA uses a fitness-based selection operator, such as roulette, to select chromosomes for reproduction from the evolved population and uses crossover and mutation to reproduce new offspring chromosomes. Once a termination criterion s met, the GA delivers one or more chromosomes as best-fit recommendations, D ,that satisfy all limit constraints, e.g. min number of sleeping hours. The PO evaluates each Multimed Tools Appl (2018) 77:26397–26448 26409 Fig. 14 FitNalyser Recommendations screen Fig. 15 Bru-Fit’s top section of the opening screen 26410 Multimed Tools Appl (2018) 77:26397–26448 Fig. 16 Bru-Fit’s bottom section of the opening screen D with reference to optimisation objectives, O , e.g. max number of steps per day, to distinguish i i between dominated and non-dominated chromosomes and then assigns a strength value to each non-dominated chromosome that is proportional to the number of the dominated chromosomes it weakly dominates. Figure 4 also includes an example of a typical chromosome as a Full Decision Vector along with its Limit and Optimisation Constraints Vectors. The chromosome population (P) is expressed as P = P ∪ P,where P and P contain the non-dominated and E I E I dominated chromosomes respectively. The strength value of a non-dominated C is expressed as j C jC ∈P &C ≺C j fg j j I i j SðÞ C ¼ , where C ∈ P . The PO then calculates a fitness value for each i E jP jþ1 dominated chromosome, C as the sum of the strength values of the non-dominated chromosomes j, jP j SCðÞ i ; if C ≺C i j that weakly dominate C,i.e. FC ¼ 1 þ ∑ ,where C ∈ P . A dominates j j i E ; otherwise i¼1 Fig. 17 Bru-Fit’s general information screen Multimed Tools Appl (2018) 77:26397–26448 26411 Fig. 18 Bru-Fit’s Active Min screen B, A ≺ B,iff O (A) ≤ O (B), ∀i =1, …, D and O (A)< O (B)forsome i and A weakly dominates B, i i i i A≺B,iff O (A) ≤ O (B), ∀i=1, …, D,where D is the number of optimisation constraints. A Pareto i i 0 0 Front of optimal D (C ) that satisfy: f : P ¼fg C ; C ; …; C ⟶C ∈P is produced where n i j 1 2 n j E E is the number of chromosomes. Game theorists argue that the distribution of optimal solutions on a Pareto Front suggests the existence of a Bbest-fit^ optimal recommendation D . Sofokleous and Angelides [30] calculates Euclidean Distances (ED) between each chromosome, i.e. a qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi similarity measure between two chromosomes: ED C ; C ¼ O C −O ðÞ C þ … j i 1 j 1 i þ O C −O ðÞ C ,where O (C ) is the result of evaluation of O using chromosome C m j m i k z k z values. The closer C is to C , the more likely is that C will result in the same satisfaction as C . j i j i The fitness value of each chromosome is then re-calculated and the one with the highest f(C ) value is selected as the Bbest-fit^ optimal recommendation D and recommended to the wearer: 0 1 UserRating 0 @ A fCðÞ¼ 10 þ ∑ 1 þ ED C ; C = 1 þ P i j i C ∈P E 26412 Multimed Tools Appl (2018) 77:26397–26448 Fig. 19 Bru-Fit’s Active Min screen weekly average The models produced by JMP PRO and the recommendations generated by the GA and PO are then forwarded to either FitNalyser or Bru-Fit for visualisation during step 7. 3.2 Step 7: FitNalyser This android middleware app runs on a Samsung smartphone paired with a Samsung Gear Fit hybrid smartwatch and fitness tracker which may be worn 24/7 since it is an always-on activity tracker that measures Bsteps^, Bexercise^, Bsleep^ and Bheart rate^. The 9-step process of Fig. 2 that includes FitNalyser at step 7 unfolds as follows: 1. Configuration and Goal Setup: The wearer populates their wearable with personal data, such as name, gender, age, weight, and height, to enable the device to collect data correctly and sets up their daily goals, e.g. number of steps, number of hours spent walking or sleeping. 2. Tracking Activity: The wearable’s sensors track the wearer’s daily activities and sleep patterns without inhibiting any of their senses. Multimed Tools Appl (2018) 77:26397–26448 26413 Fig. 20 Bru-Fit’sActive Min screen monthly average 3. Data Synchronisation: The wearable establishes a Bluetooth connection to, and synchro- nizes with, the smartphone to transfer collected data. 4. Raw Data Forwarding: Data Viewer, a third-party software tool running on the smartphone, receives the dataset and packages this for transfer to a laptop via the tracker’s cloud servers. 5. Raw Data Analysis and Recommendation Generation: JMP PRO performs general- ised regression analysis of the wearer dataset in comparison to the peer community dataset and produces a model that is used by the GA and PO to generate recommendations. 6. Analysed Data and Recommendation Forwarding: The laptop transfers the model pro- duced by JMP PRO and the recommendations generated by the GA and PO back to the smartphone via the tracker’s cloud servers. 7. Analysed Data and Recommendation Visualisation: The analysed data model and recom- mendations are visualised on FitNalyser to support the wearer with monitoring their daily goals and activities. 8. Decision Making: The wearer reconsiders his current daily goals and activities, and may set up new activity goals, e.g. increase the number of hours spent sleeping, reduce the number of hours spent walking. 9. Goal Synchronisation: The smart phone synchronises with the wearable to auto-update any revised goals. 26414 Multimed Tools Appl (2018) 77:26397–26448 Fig. 21 Bru-Fit’s Steps screen daily Figures 5 through to 14 showcase the FitNalyser android app at step 7. Figure 5 depicts the opening screen of the mobile app which has been divided into 4 parts: profile, my data, community data and analysis. Figure 6 shows the wearer profile following configuration: Gender, date of birth, weight and height. Figure 7 shows My Data screen which offers the choices of step count, exercise or sleep to the wearer. On Fig. 8 the step count performance is visualised, i.e. the daily average of number of steps, burned calories and daily distance and the monthly total of number of steps, burned calories and distance following tracking. Similarly, on Fig. 9 the exercise performance is visualised, i.e. the daily average of burned calories, duration, and distance as well as the monthly totals. Likewise, on Fig. 10 the nightly and monthly sleep average is visualised. Figure 11 mirrors the choices on offer on My Data screen for Community Data but in addition it reveals the number and age range of wearers, differentiating between males and females, and their weight. On Fig. 12 the daily averages and monthly totals for Community Data for both genders are visualised. Figure 13 relays personal against community performance data following analysis wearer own data and comparison against community data, with a gender bias. This helps develop personal and community fitness trends in relation to all activities. On this screen, the wearer may choose to consider recommendations generated by the GA and GO during step 5. On Fig. 14 recommendations are visualised in relation to diet, hydration and excercising. The wearer may use these recommendations to review current daily goals and activities and set up new ones. Multimed Tools Appl (2018) 77:26397–26448 26415 Fig. 22 Bru-Fit’sSteps screen weekly and monthly 3.3 Step 7: Bru-fit This android middleware app runs on a Samsung smartphone paired with a Fitbit® Surge hybrid smartwatch and fitness tracker that may be worn 24/7 as it is an always-on activity tracker that measures Bsteps^, Bexercise^, Bsleep^ and Bheart rate^. Its sensors include GPS, 3- axis accelerometer, 3-axis gyroscope, digital compass, optical heart rate monitor, altimeter, ambient light sensor and vibration motor. The 9-step process of Fig. 2 that includes BruFit at step 7 unfolds as follows: 1-3. Configuration and Goal Setup, Tracking Activity, Data Synchronisation: as with FitNalyser. 4. Raw Data Forwarding: A Fitbit open script executes the transfer of the raw dataset to a laptop via Fitbit’s cloud servers. 5. Raw Data Analysis and Recommendation Generation: as with FitNalyser. 6. Analysed Data and Recommendation Forwarding: A Fitbit open script executes the transfer of the models produced by JMP PRO and the recommendations generated by the GA and PO back to the smartphone via Fitbit’s servers. 7-9. Analysed Data and Recommendation Visualisation, Decision Making, Goal Synchronisation: as with FitNalyser. 26416 Multimed Tools Appl (2018) 77:26397–26448 Fig. 23 Bru-Fit’s Sleep screen daily Figures 15 through to 29 showcase the Bru-Fit android app at step 7. Figure 15 shows the top section of the opening screen of BruFit. Figure 16 shows the bottom section of the opening screen of BruFit, which has been divided into 5 parts: Active Min, Steps, Sleep, Calories, Analysis. Figure 17 gives an example of the general information that may be visualised on request. It shows the wearer’s profile which includes their activity level and the daily goals set following configuration and goal setup. The wearer may revisit these goals and set up new goals after considering the recommendation. More detailed stats are available in the Activity Frequency section which shows personal bests in relation to Steps, Very Active Minutes, Calories, km covered, Floors, and Hours Asleep. Figure 18 shows the Active Minutes screen which includes a daily average of steps, active minutes, calories, km covered, floors, and hours asleep and a comparison to the daily goals. Two detailed graphs show floor climbing and daily activity during the reporting period. Figure 19 reports the weekly activity and averages for floor climbing and distance whereas Fig. 20 reports the average monthly activity with emphasis on floor climbing. Figure 21 shows the Steps screen on which it compares the daily steps to the daily goal as well as a daily and hourly distribution of steps whereas Fig. 22 reports the weekly and monthly distribution of steps. Figure 23 shows the Sleep screen and reports daily sleep patterns and efficiency, actual sleep time, bed and rise time whereas Fig. 24 reports Multimed Tools Appl (2018) 77:26397–26448 26417 Fig. 24 Bru-Fit’s Sleep screen weekly and monthly the weekly and monthly sleep data. Figure 25 shows the Calories screen that details daily, weekly and monthly calories spent whereas Fig. 26 visualises a comparison between personal and community data with a gender bias on calories spent using bar and line graphs and boxplots. Likewise, Figs. 27 and 28 visualise comparisons between personal and community data, also with a gender bias, in relation to sleep data and steps respectively. Figure 29 shows Bru-Fit’s Recommendations screen whose content is generated by the GA and GO during step 5. This screen visualises recommendations in relation to sleep or lack of it, bed and rise times, calorific intake and spent, and activity or lack of it. The wearer may consider these recom- mendations to revisit current daily goals and activities and set up new ones. 4 Validation: a month in the process This section presents the application of the process on Fig. 2 over the period of one month with each of the two applications that we have developed. The purpose of this is to validate our approach and, in particular, the novelty in steps 5, 6 and 7. 26418 Multimed Tools Appl (2018) 77:26397–26448 Fig. 25 Bru-Fit’s Clories screen daily, weekly and monthly 4.1 FitNalyser At Step 1, the wearer configures their wearable and sets their initial goals of daily step count, exercise and sleep. During Step 2, the Samsung Gear Fit tracks the daily step count, exercise and sleep durations and monitors the heart rate without inhibiting any of the wearer’ssenses. At Step 3, the wearable synchronizes with the smartphone using a Bluetooth connection and transfers the daily dataset to the Samsung smartphone. During Step 4, Data viewer which has been installed on the smartphone, receives the dataset, packages that as a cvs file and forwards this file via the tracker’s cloud servers to a laptop that runs JMP PRO and the GA with PO. Table 2 shows a screenshot of the raw data file on step count. Data collected include start and end time, calories and distance. During Step 5, once both the personal daily and the community datasets have been received on the laptop, JMP PRO is used first to analyse and compare the personal against the public community dataset and build a general regression model. The first calculation made is the Body Mass Index (BMI) which uses measures of weight and height to determine whether the wearer is under, normal or overweight. This will inform the values of the Limit and Optimisation Constraint Vectors. This is followed by the Basal Metabolic Rate (BMR) which estimates how many calories are spent even without any activity during Multimed Tools Appl (2018) 77:26397–26448 26419 Fig. 26 Bru-Fit’s Analysis screen comparing personal to community data in relation to calories a 24-h period. This represents the minimum amount of energy required to keep the body functioning. The Harris-Benedict and Mifflin equations are commonly used to calculate BMR because of their accuracy. The BMR for men differs to that of women mostly due to their body weight: For men: (10 × weight)+ (6.25 × height) − (5 × age)+5. For women: (10 × weight) + (6.25 × height) − (5 × age) − 161 Table 3 gives the BMI and BMR calculated using parameters extracted from the personal dataset. These values are carried forward to the comparative analysis stage. In addition to the BMI and BMR calculation, calculation of the calories burned during all measured activities with the wearable, i.e. walking, exercising, and sleeping are presented on Fig. 30.Figure 30 reveals that at the peak of walking the number of calories burned are less than the calculated BMR since the amount of energy required to support vital body functions is significantly higher. A sample of a monthly personal dataset is presented on Table 4. The figure shows the total values of the variables relating to the three measured activities and their daily average. Moreover, a detailed distribution for each day of the week, regarding the main activities is shown on Table 5 which reveals the daily effort on a weekly basis with Tuesdays and Fridays peaking at 16%. Exercise is frequently occurring more on Sundays at 20% and sleep duration is at its peak on Wednesdays with 16%. 26420 Multimed Tools Appl (2018) 77:26397–26448 Fig. 27 Bru-Fit’s Analysis screen comparing personal to community data in relation to sleep Figure 31 reveals that when the number of the calories from all activities are added together, Tuesdays and Fridays are the days during which most calories are burned; this resembles the step count distribution. The community dataset has been compiled from 16 different wearers, 8 women and 8 men between the ages of 23 and 31 and it comprise of both qualitative and quantitative data: height (cm), weight (kg), age, gender, step_count, step_calories, steps_distance (km), exercise_calories, exercise_distance (km), exercise_duration (min), sleep_calories and sleep_duration (min). Table 6 shows the BMI and BMR of each wearer. Figures 32 and 33 present the community’s monthly distribution of spent calories during the measured activities with a gender bias. The two figures reveal a similar pattern for males and females. The range of calories spent whilst waking or sleeping is higher than whilst exercising with the maximum calories burned by women being around 16,000 whilst for men 20,000. The number of calories burned whilst sleeping is higher than daily exercising or walking because sleep duration does not vary as much as dailywalking andexercisingdoasFig. 34 shows. Figure 35 reveals that males spend more calories than females whilst sleeping and walking but less whilst exercising. Multimed Tools Appl (2018) 77:26397–26448 26421 Fig. 28 Bru-Fit’s Analysis screen comparing personal to community data in relation to steps Figures 36, 37, 38, 39, 40, 41, 42 and 43 present detailed data analytics produced by JMP PRO. Figure 36 shows the daily mean for walking being 10,622.917 and reaching a maximum of 45,408 and dropping to a minimum of 308. Figure 37 presents the daily mean of calories spent during walking being 387.89 and reaching a maximum value of 1530 and dropping to a minimum of 11. Figure 38 illustrates the daily mean for distance covered ranging between a maximum of 34.5 and a minimum of 0.25. The boxplots indicate that there may be upper or lower limit outliers and the last minimum might be one of these. All three graphs show a positive skewness distribution, which indicate that the lower limits for steps, calories, and distance is higher than the higher limits. The results show that most datasets report between 5000 and 15,000 steps daily. Thus, the number of calories spent fluctuate between 200 and 600. On Fig. 39 the distribution is slightly flatter than a standard normal whereas Figs. 40 and 41 show kurtosis values higher than 3 which indicate a high number of outliers. This number of outliers is a consequence of either forgetting to register the daily exercise activity or not exercising. Figure 42 shows a negative kurtosis of −0.14 indicating a flatter distribution with only one outlier and a low skewness of 0.26 making it symmetrical than the others. Calories burned whilst sleeping are directly proportional to the wearer’s weight, 50 kg to 79 kg. Thus, the distribution is most likely normal. Figure 43 26422 Multimed Tools Appl (2018) 77:26397–26448 Fig. 29 Bru-fit’s Recommendations screen shows a negative skewness with a high kurtosis value since sleep duration does not follow the expected pattern. Correlational analyses of the walking, exercising and sleeping dataset variables are presented on Figs. 44, 45 and 46. Whilst with exercising the correlation is similar to walking, with sleeping the correlation is not as high since the sleeping duration does not normally have a specific pattern, as wearers who exercise more daily are normally expected to sleep for longer periods. However, this is not always the case because a wearer may either have developed a regular sleeping pattern regardless or are unable to sleep for the required amount of time. Table 7 presents a full table of correlations between each variable of the dataset. Table 8 presents a summary of daily activity over the period of one week. For instance, step counts usually rises at weekends whereas exercising rises mid-week and on Sundays and average sleep duration is higher at weekends in comparison to weekdays with 3 additional hours of sleep. The highest daily calorie consumption seems to take place on Fridays as Fig. 47 shows. Figures 48, 49 and 50 map the personal against the community dataset. Figure 48 reveals that the monthly personal step count is higher than the community’s average for females whereas Figs. 49 and 50 reveal that personal exercise and sleep periods are lower. Figure 51 provides a comparison of the calories expended for the monitored activities and it reveals that the calories expended reflects either the effort and or the activity duration. Multimed Tools Appl (2018) 77:26397–26448 26423 Table 2 Step count’s cvs data file com.samsung.health.step_count time_offset end_time speed pkg_name start_time sample count calories distance datauuid deviceuuid update_time create_time 3600000 6/17/16 23:20 1.0366666 com.sec.android.app.shealth 6/17/16 23:19 14 0.51 07.81 c597e372-5b2 n651IFUG7o 6/17/16 23:19 6/17/16 23:19 3600000 6/17/16 23:26 2.4166667 com.sec.android.app.shealth 6/17/16 23:25 16 3.09 11.67 4a104a25-457 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:28 5.0833335 com.sec.android.app.shealth 6/17/16 23:27 27 1.14 31.01 edee5f62-265 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:29 2.9166667 com.sec.android.app.shealth 6/17/16 23:28 35 1.24 33.51 4a2d4896-5c2 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:30 3.9722223 com.sec.android.app.shealth 6/17/16 23:29 23 0.85 23.11 1a47aeef-179 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:31 1.9298451 com.sec.android.app.shealth 6/17/16 23:30 30 1.0099999 24.70 89f9a721-807 n651IFUG7o 6/17/16 23:30 6/17/16 23:30 3600000 6/17/16 23:32 2.1912923 com.sec.android.app.shealth 6/17/16 23:31 19 0.5899999 16.90 a5ec49fd-ba4 n651IFUG7o 6/17/16 23:31 6/17/16 23:31 3600000 6/17/16 23:34 2.8055556 com.sec.android.app.shealth 6/17/16 23:33 6 0.28 06.13 a583f27e-a49 n651IFUG7o 6/17/16 23:33 6/17/16 23:33 3600000 6/17/16 23:35 1.4919225 com.sec.android.app.shealth 6/17/16 23:34 10 0.29 07.38 777c3953-a54 n651IFUG7o 6/17/16 23:34 6/17/16 23:34 3600000 6/17/16 23:36 5.4722223 com.sec.android.app.shealth 6/17/16 23:35 6 0.93 08.10 b675040f-aa2 n651IFUG7o 6/17/16 23:37 6/17/16 23:37 3600000 6/17/16 23:38 3.2222223 com.sec.android.app.shealth 6/17/16 23:37 25 1.85 27.44 99439287-0d1 n651IFUG7o 6/17/16 23:37 6/17/16 23:37 3600000 6/17/16 23:40 1.3333334 com.sec.android.app.shealth 6/17/16 23:39 17 0.58 11.38 7340ab3a-381 n651IFUG7o 6/17/16 23:42 6/17/16 23:42 3600000 6/17/16 23:45 2.5555556 com.sec.android.app.shealth 6/17/16 23:44 54 3.27 48.34 1be95e90-e23 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:46 5.888886 com.sec.android.app.shealth 6/17/16 23:45 8 0.66 11.71 b75e2467-547 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:47 2.1388888 com.sec.android.app.shealth 6/17/16 23:46 38 1.52 34.56 70e3f7b9-8d2 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:48 2.9444444 com.sec.android.app.shealth 6/17/16 23:47 18 0.61 17.74 37da7c16-002 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:49 5.0555553 com.sec.android.app.shealth 6/17/16 23:48 9 1.08 12.95 abb5b8dc-571 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:50 3.0277777 com.sec.android.app.shealth 6/17/16 23:49 23 0.7 20.99 259bb8bd-034 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:51 3.1111112 com.sec.android.app.shealth 6/17/16 23:50 47 1.76 46.13 008c5fe1-068 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 3600000 6/17/16 23:52 2.75 com.sec.android.app.shealth 6/17/16 23:51 32 1.22 30.51 a691ebd0-293 n651IFUG7o 6/17/16 23:55 6/17/16 23:55 26424 Multimed Tools Appl (2018) 77:26397–26448 Table 3 Personal BMI and BMR calculation Height Weight Age Gender BMI BMR 154 54.4 28 female 22.94 1205.5 1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031 Days step_calories exercise_calories sleep_calories Fig. 30 Personal monthly distribution of spent calories during the measured activities Table 4 Personal monthly and average daily stats Step count Step Steps Exercise Exercise Exercise Sleep Sleep calories distance calories distance duration calories duration Per month 445566 12164 290.47 1184 28.88 349 12092 13362 Daily Ave 14373.10 392 9.37 118.40 2.89 34.91 390.05 431.03 Table 5 Personal daily summary on a weekly basis Step Count Step calories Exercise calories Exercise duration Sleep calories Sleep duration Monday 14547 390 120 37 342 378 Tuesday 15776 434 108 31 400 442 Wednesday 10731 294 127 36 421 466 Thursday 14834 401 79 26 362 401 Friday 15771 437 97 27 421 465 Saturday 13305 369 143 43 392 433 Sunday 15328 412 163 48 384 425 Calories Multimed Tools Appl (2018) 77:26397–26448 26425 Fig. 31 Personal weekly pie of calories spent Table 6 Community BMI and BMR Wearer Height Weight Age Gender BMI BMR 1 154 55 28 female 23.19 1211.5 2 159 50 28 female 19.78 1192.75 3 154 55 28 female 23.19 1211.5 4 160 53 30 female 20.70 1219 5 166 60 29 female 21.77 1331.5 6 155 55 28 female 22.89 1217.75 7 160 57 30 female 22.27 1259 8 157 56 29 female 22.72 1235.25 9 177 62 29 male 19.79 1586.25 10 178 80 31 male 25.25 1762.5 11 182 80 28 male 23.89 1808.75 12 180 79 28 male 24.38 1780 13 170 70 23 male 24.15 1802.5 14 174 70.3 30 male 23.22 1645.5 15 176 79 31 male 25.50 1740 16 169 76 30 male 26.61 1671.25 Steps 7795 11929 11447 9271 16148 15038 12445 8020 Exercise 4079 3040 6973 3806 5067 5613 4758 5067 Sleep 12813 12332 13095 12782 15513 12501 14996 13443 Subject Fig. 32 Community monthly distribution of spent calories during the measured activities for females Calories 26426 Multimed Tools Appl (2018) 77:26397–26448 Steps 13927 9098 7721 11825 18675 13153 16828 18910 Exercise 1179 2254 7565 5572 0 2782 0 8368 Sleep 15688 20038 17250 17201 18755 16977 13685 16936 Subject Fig. 33 Community monthly distribution of spent calories during the measured activities for males 123456789 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Days Exercise Sleep Fig. 34 Community monthly duration of exercising and sleep for women Fig. 35 Community daily calories spent: males versus females Fig. 36 Daily data analytics for step count Duraon Calories Multimed Tools Appl (2018) 77:26397–26448 26427 Fig. 37 Daily data analytics for calories burnt during walking Fig. 38 Daily data analytics for distance covered whilst walking Figure 52 which summarises the monitored activities, reveals a personal pattern that matches on some days only that of the community and may also suggest a sedentary lifestyle. Table 9 summarises the personal results and Figs. 53, 54 and 55 provide a comparison to the relevant community dataset and trends. The data analytics models on Figs. 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46 and Table 7 are used by GA and PO during the second part of Step 5 to generate an initial population of Fig. 39 Daily data analytics for calories burnt during exercising 26428 Multimed Tools Appl (2018) 77:26397–26448 Fig. 40 Daily data analytics for distance covered whilst exercising Fig. 41 Daily data analytics for exercise duration chromosomes and to source limit and optimisation constraints. The recommendations to the wearer will be sourced from the non-dominated chromosome that evolves, like the one which appears on Fig. 4. During Step 6 the laptop transfers the daily models produced by JMP PRO and the recommendation generated by the GA and PO to the smartphone via the tracker’s cloud servers. During Step 7, the data models and recommendation are visualised using Fig. 42 Daily data analytics for calories burnt during sleeping Multimed Tools Appl (2018) 77:26397–26448 26429 Fig. 43 Daily data analytics for sleeping duration FitNalyser to support the wearer with monitoring their daily goals and activities and their weekly and monthly performance. The analysis of the personal dataset collated on the wearable device has picked up that the daily average of expended calories is approximately Fig. 44 Correlation between walking variables 26430 Multimed Tools Appl (2018) 77:26397–26448 25% less than the daily average a person with that BMI is expected to burn which in the long- term will result in weight gain and the chromosome and subsequent recommendation generated addresses that. There is clearly room for improvement in relation to calorie burning and the generated recommendation visualised on Fig. 14 reflects that. The wearer may wish to consider as much evidence behind this recommendation as possible and may also wish to consider their performance in relation to that of their peers. Figures 8 through to 13 provide all the daily and weekly evidence the wearer may seek. During Step 8, the wearer, having considered the evidence and recommendation, may revisit their current daily goals and activities, and set up new activity goals, e.g. reduce the number of hours spent sleeping, increase the number of hours spent walking, increase the step count, to move from sedentary to lightly active or active. During Step 9, the smart phone synchronises with the wearable to auto-update any revised goals. Not all wearables facilitate that but both wearables used in our experiment allow that. Fig. 45 Correlation between exercising variables Multimed Tools Appl (2018) 77:26397–26448 26431 4.2 Bru-Fit Steps 1, 2 and 3 are like FitNalyser’s. Fitbit tracks sleep through movement using one of two modes, normal and sensitive. It records, as when awake, significant movements only in normal mode and nearly all movements in sensitive mode: t ¼ t −t asleep Total tracked awake asleep Sleep Efficiency ¼ 100 t þ t þ t asleep restless awoken during sleep Fitbit tracks the Heart rate through blood volume variation using LED lights onto the skin. It records the intensity of exercising as a percentage of: Fig. 46 Correlation between sleeping variables 26432 Multimed Tools Appl (2018) 77:26397–26448 Table 7 Full table of correlations Correlations step_count step_calories steps_distance exercise_calories exercise_distance exercise_duration sleep_calories sleep_duration step_count 1.0000 0.9180 0.9878 0.3004 0.3302 0.3482 −0.1174 0.0700 step_calories 0.9180 1.0000 0.9296 0.2953 0.2963 0.3193 0.0015 0.0228 steps_distance 0.9878 0.9296 1.0000 0.2999 0.3240 0.3369 −0.0921 0.0589 exercise_calories 0.3004 0.2953 0.2999 1.0000 0.9780 0.9373 −0.0908 −0.1317 exercise_distance 0.3302 0.2963 0.3240 0.9780 1.0000 0.9746 −0.1692 −0.0989 exercise_duration 0.3482 0.3193 0.3369 0.9373 0.9746 1.0000 −0.1652 −0.0834 sleep_calories −0.1174 0.0015 −0.0921 −0.0908 −0.1692 −0.1652 1.0000 0.5124 sleep_duration 0.0700 0.0228 0.0589 −0.1317 −0.0989 −0.0834 0.5124 1.0000 Multimed Tools Appl (2018) 77:26397–26448 26433 Table 8 Dailyactivitytrends Step Step Steps Exercise Exercise Exercise Sleep Sleep count calories distance calories distance (km) duration calories Duration (km) (min) (min) Monday 8580 315 6.44 193 3.80 47 382 359 Tuesday 8261 317 6.29 210 4.19 52 377 359 Wednesday 8572 317 6.38 164 3.16 40 391 370 Thursday 8555 308 6.30 184 3.48 45 403 377 Friday 13731 502 10.11 118 2.09 27 632 595 Saturday 13169 475 9.76 186 3.02 43 618 576 Sunday 12968 459 9.64 216 4.60 55 624 579 Fig. 47 Daily calories burned 400000 355264 300000 women men main subject step_count Fig. 48 Monthly step count 26434 Multimed Tools Appl (2018) 77:26397–26448 women men main subject exercise_duraon (min) Fig. 49 Monthly exercise period 14000 women men main subject sleep_duraon (min) Fig. 50 Monthly sleep period sleep_calories exercise_calories step_calories 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 main subject men women Fig. 51 Monthly comparison of calories burnt Multimed Tools Appl (2018) 77:26397–26448 26435 Step count Exercise duraon Sleep duraon Days Main Subject Dataset Main Subject Dataset Main Subject Dataset 8580 47 359 Monday 14547 37 378 8261 52 359 Tuesday 15776 31 442 Wednesday 10731 8572 36 40 466 370 Thursday 14834 8555 26 45 401 377 13731 27 595 Friday 15771 27 465 13169 43 576 Saturday 13305 43 433 12968 55 579 Sunday 15328 48 425 Fig. 52 Personal against community trends Table 9 Personal results Gender Female Age 28 years old Weight 54.4 kg Height 154 cm BMI 22.94 (Healthy weight) Daily calories burnt 1173.5 Activity level Sedentary Daily average step count 14373 Daily average exercise duration 35 min (2 days per week) Daily average sleep duration 431 min Personal trends in relation to community step count = normal; exercise duration = poor; sleep duration = group normal Maximum rate ¼ 220−age This rate decreases with age, thus seeking to acheve the same level of intensity across different age groups requires different amounts of effort. Table 10 shows Fitbit’s five intensity zones as a percentage of Heart rate. Fitbit tracks activity and calories burned through exercise intensity using Steps, Floors and Heart Rate along with Metabolic Equivalent (MET), BMR, weight and height. BMR values 445566 14373 step count step count 355264 11460 0 200000 400000 0 5000 10000 15000 20000 main subject women main subject women Fig. 53 Personal against community step count 26436 Multimed Tools Appl (2018) 77:26397–26448 349 35 exercise duraon exercise duraon (min) (min) 1379 58 0 500 1000 1500 020 40 60 80 main subject women main subject women Fig. 54 Personal against community exercise duration 13362 431 sleep duraon (min) sleep duraon (min) 14664 473 12500 13500 14500 400 420 440 460 480 main subject women main subject women Fig. 55 Personal against community sleep duration Table 10 Fitbit heart rate zones Effort Effect Peak zone 85% of maximum rate High-intensity exercise zone Cardio 70 to 84% of maximum rate Medium-to-high intensity exercise zone Fat burn zone 50 to 69% of maximum rate Low-to-medium intensity exercise zone Out of zone Below 50% of maximum rate Not enough to be considered exercise Resting 60-80 beats per minute Awake, calm, comfortable, not recently exerted yourself Table 11 Fitbit parameters tracked Parameter Variable Sleep Time in bed Actual Sleep Time Activities Minutes Sedentary Minutes Lightly Active Minutes Fairly Active MinutesVeryActive Steps Number of Steps Floors Number of Floors Calories Number of Calories burned Multimed Tools Appl (2018) 77:26397–26448 26437 Fig. 56 Participants by age usually account for at least half of the calories burnt in a day and this is estimated based on gender, age, height, and weight. Table 11 shows the Fitbit parameters tracked. A significant difference between Bru-Fit and FitNalyser takes place during Step 4, during which a Fitbit open script executes the transfer to a laptop via Fitbit’s cloud servers of the raw dataset which hasbeenpackagedintoacvsfile. Step 5 is like FitNalyser but the wearer and the peer group comprise different wearers. The community group comprise of 14 different wearers, 6 women and 8 men and their dataset includes both qualitative and quantitative data: Age, gender, height, weight, BMI, calorie stats (maximum, minimum, average, daily, weekly, monthly), activity stats (level, frequency), step stats, floor stats, distance, sleep stats (daily patterns, efficiency). Figures 56 and 57 classify participants by age and gender respectively. Figure 58 maps the daily average of number of calories spent across the community. The figure reveals a mixture of distribution across the Fig. 57 Participants by gender 26438 Multimed Tools Appl (2018) 77:26397–26448 Fig. 58 Daily average of number of calories spent across the community Fig. 59 Parameter correlation Fig. 60 Daily calories spent: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26439 Fig. 61 Daily calories spent: Personal vs community females Fig. 62 Daily calories spent: Personal vs Lightly Active community members Fig. 63 Daily calories spent: Personal vs Lightly Active community females 26440 Multimed Tools Appl (2018) 77:26397–26448 Fig. 64 Daily calories spent: Personal vs community Fig. 65 Daily average calories spent: Personal vs Active community Fig. 66 Daily sleep: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26441 Fig. 67 Daily sleep: Personal vs community females Fig. 68 Daily sleep: Personal vs Lightly Active community members Fig. 69 Daily sleep: Personal vs Lightly Active community females 26442 Multimed Tools Appl (2018) 77:26397–26448 Fig. 70 Daily sleep: Personal vs community Fig. 71 Daily average sleep: Personal vs Active community Fig. 72 Daily steps: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26443 Fig. 73 Daily steps: Personal vs Lightly Active community members Fig. 74 Daily steps: Personal vs community females Fig. 75 Daily steps: Personal vs Lightly Active community females 26444 Multimed Tools Appl (2018) 77:26397–26448 Fig. 76 Daily steps: Personal vs community community, positively skewed because of low number of calories spent and normal. This splits the community into sedentary or lightly active. Figure 59 shows parameter correlation. Red highlights indicate no positive correlation has been established whereas blue highlights indicate positive correlation. The correlation becomes stronger as the shade becomes darker. Unlike with FitNalyser, the correlation map does not indicate a correlation between daily activity and sleep. Figures 60, 61, 62, 63, 64 and 65 compare the number of calories spent between the wearer and the community datasets. The results are attributed to females normally having a higher MBR than males because of having, in general, a higher percentage of fat to muscle in comparison to males who have, in general, a higher percentage of muscle to fat. Figures 66, 67, 68, 69, 70 and 71 show the outcome of tracking sleep activity. The figures reveal a higher sleep activity in males than in females. Figures 72, 73, 74, 75, 76 and 77 show the outcome of tracking daily steps through walking. These figures reveal a higher walking activity, and thus steps, by males than females in sharp contrast to past studies which report the reverse. A second significant difference between Bru-Fit and FitNalyser takes place during Step 6, at which a Fitbit open script executes the transfer back to the smartphone via Fitbit’s servers of Fig. 77 Daily average steps: Personal vs community Multimed Tools Appl (2018) 77:26397–26448 26445 the models produced by JMP PRO and the recommendations generated by the GA and PO. At Step 7, the analysis of the personal dataset collated on the wearable device has picked up that that the wearer has exceeded their daily goals with midweek being the most active days and Sundays the least active with walking being the most popular activity. Daily tracking of sleep efficiency has picked up that this rises during weekdays but declines at weekends despite a higher sleep activity during weekends. There were also significant variations in sleep efficien- cy across community members belonging to the same age group. This may be attributed to circadian rhythm sleep disorder CRSD which is common among younger generations. The results also reveal that males are more likely to be overweight and fall into the sedentary range than females are. Unlike FitNalyser the figures do not correlate sleep and daily activity. There is clearly room for improvement in relation to physical activity and sleeping at weekends and the generated recommendation visualised on Fig. 29 reflects that. The wearer may also wish to consider as much evidence behind this recommendation as possible and may also wish to consider their performance in relation to that of their peers. Figures 17 through to 28 provide all the daily and weekly evidence the wearer may seek. During Step 8, the wearer, having considered the evidence and recommendation, may revisit their current daily goals and activities, and set up new activity goals, e.g. reduce the number of hours spent sleeping and increase the number of hours spent walking. During Step 9, the smart phone synchronises with the wearable to auto-update any revised goals. Figure 17 shows how goals set on this wearable are visualised with Bru-Fit. 5 Concluding discussion The paper demonstrates how wearable technology may be used beyond the hype to improve personal lifestyles by suggesting areas where generating a change in daily or routine habits may lead to a healthier lifestyle. Currently, such cycles appear to be driven by irrelevant or incomplete information which may not be easy to interpret but under peer pressure and the fear of exclusion, wearers adopt ad hoc change regardless. Our research and development work demonstrates that a planned approach to data collection, analysis and visualisation coupled with a machine learning recommender approach may transpose wearer decisions to be informed and that the use of peer group data is for generating correlated recommendations not for applying social pressure. A mobile app might not be able to support the entire cycle and consequently backbone computing support is required. The analysis has also thrown some accepted truths into dispute, i.e. the correlation between exercising and sleeping is very low suggesting the two are more independent than once thought. This might be the result of exogenous factors such as workstyle or underlining health conditions such as cardiovascular diseases. Areas for further research and development naturally include, in addition to the rising need for standardisation, the addition of more sensors to track further personal fitness and alarms to warn of the possibility of ambulatory incidents, an approach recently put on trial by the National Health Service (NHS) in the UK. This would raise the significance of the personal recommendations, especially in the health sector, but this would require a more extensive clinical trial as it would have serious repercussions to health and fitness. Finally, suggestions of automatic readjustment of personal goals following a recommendation, i.e. without wearer intervention, has divided the community after clinicians recommend that this should only be encouraged under clinical supervision. 26446 Multimed Tools Appl (2018) 77:26397–26448 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and repro- duction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. References 1. Anaya LHS, Alsadoon A, Costadopoulos N, Prasad PWC (2017) Ethical Implications of User Perceptions of Wearable Devices, Sci Eng Ethics, online first 2. Argus Insights Inc. (2016) Wearables demand report. Available at: http://www.argusinsights.com/wearables- demand-14-15 3. ARM Ltd (2014) Advanced Architectures and Technologies for the Development of Wearable Devices. White Paper, pp. 11 4. Bandodkar AJ, Jeerapan I, Wang J (2016) Wearable chemical sensors: present challenges and future prospects. ACS Sensors 1(5):464–482 5. Chen M, Ma Y, Song J, Lai C, Hu B (2016) Smart clothing: connecting human with clouds and big data for sustainable health monitoring. Mobile Networks and Applications 21:1–21 6. Delgado-Gonzalo R, Parak J, Tarniceriu A, Renevey P, Bertschi, M, Korhonen I (2015) Evaluation of accuracy and reliability of PulseOn optical heart rate monitoring device. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Milan, pp 430-433 7. Demir E, Demir K, Odabaşı S, Odabaşı F (2016) A challenge for higher education: Wearable technology for fashion design departments. World Journal on Educational Technology 8(1):69–77 8. Guesmi C, Mabrouk IB, Had L, Talb L, Gharsallah A (2016) G-Shaped Dual Band Wearable Button Antenna for ISM Band Applications. International Journal on Communications Antenna and Propagations 6(1):39–43 9. Hill C (2015) Wearables–the future of biometric technology? Biometric Technology Today 2015(8):5–9 10. IDC Research Inc. (2016) IDC Forecasts Wearables Shipments to Reach 213.6 Million Units Worldwide in 2020 with Watches and Wristbands Driving Volume While Clothing and Eyewear Gain Traction. Available at: http://www.idc.com/getdoc.jsp?containerId=prUS41530816 11. Kalinauckas A (2015a) Wearable technology. Eng Technol 10(4):36–43 12. Kalinauckas A (2015b) Wearable technology: smarten up your outfit - E & T Magazine. Available at: http://eandt.theiet.org/magazine/2015/04/wearable-technology.cfm 13. Kolodzey L, Grantcharov PD, Rivas H, Schijven MP, Grantcharov TP (2017) Wearable technology in the operating room: a systematic review. BMJ Innovations Journal 3:55–63 14. Ledger D (2016) Inside Wearables – Part 1,2,3. Endeavour Partners. Available at: http://endeavourpartners. net/inside-wearables-part-3 15. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115 16. Matzeu G, Florea L, Diamond D (2015) Advances in wearable chemical sensor design for monitoring biological fluids. Sensors Actuators B Chem 211:403–418 17. News.samsung.com (2016) Samsung Expands Industry-Leading Wearable Line with Samsung Gear Fit. Available at: https://news.samsung.com/global/samsung-expands-industry-leading-wearable-line-with-samsung-gear-fit 18. Parak J, Tarniceriu A, Renevey P, Bertschi M, Delgado-Gonzalo R, Korhonen I (2015) Evaluation of the beat-to-beat detection accuracy of PulseOn wearable optical heart rate monitor. In: Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, Milan, pp 8099-8102 19. Park S, Chung K, Jayaraman S (2014) Wearables: Fundamentals, advancements, and a roadmap for the future. Wearable Sensors: Fundamentals, Implementation and Applications. Encyclopedia of Information Science and Technology, pp. 1-23 20. Patel MS, Asch DA, Volpp KG (2015) Wearable devices as facilitators, not drivers, of health behavior change. JAMA 313(5):459–460 21. Rettberg JW (2016) Seeing ourselves through technology: How we use selfies, blogs and wearable devices to see and shape ourselves. Palgrave Macmillan 22. Richter F (2016) Infographic: The Global Wearables Market in 2015. Statista Infographics. Available at: https://www.statista.com/chart/4423/wearable-device-shipments-2015 23. Robson K, Pitt L, Kietzmann J, Halvorson W, Wallstrom A (2016) Wearable technology: trends and opportunities for organizations. In: Celebrating America’s Pastimes: Baseball, Hot Dogs, Apple Pie and Marketing? Springer, pp. 801-801 Multimed Tools Appl (2018) 77:26397–26448 26447 24. Salah H, Maclntosh E, Rajakulendran N (2015) Wearable Tech: Leveraging Canadian Innovation to Improve Health. MaRS Market Insights 25. Schüll ND (2016) Data for life: Wearable technology and the design of self-care, BioSocieties 26. Shelgikar AV, Anderson P, Stephens MR (2016) Sleep tracking, wearable technology, and opportunities for research and clinical care. CHEST Journal 50(3):732–743 27. Shih PC, Han K, Poole ES, Rosson MB, Carroll JM (2015) Use and adoption challenges of wearable activity trackers, iConference 2015 Proceedings, p. 12 28. Shoaib M, Scholten H, Havinga PJM (2013) Towards physical activity recognition using smartphone sensors. 2013 IEEE 10th International Conference on Ubiquitous Intelligence & Computing and 2013 IEEE 10th International Conference on Autonomic & Trusted Computing, Sorrento, pp 80–87 29. Sofokleous AA, Angelides MC (2008) DCAF: an MPEG-21 dynamic content adaptation framework, Multimedia Tools and Applications 40:151–182 30. Sofokleous AA, Angelides MC (2009) Dynamic selection of a video content adaptation strategy from a Pareto front, The Computer Journal 52:413–428 31. Stoppa M, Chiolerio A (2014) Wearable electronics and smart textiles: a critical review. Sensors 14(7): 11957–11992 32. Thierer AD (2015) The internet of things and wearable technology: Addressing privacy and security concerns without derailing innovation. Adam Thierer, The Internet of Things and Wearable Technology: Addressing Privacy and Security Concerns without Derailing Innovation, pp. 21 33. Viscusi S (2016) The Current and Future State of Wearables: Trends, Attitudes, and Total Domination. Available at: http://www.wearabletechworld.com/topics/wearable-tech/articles/417581-current-future-state- wearables-trends-attitudes-total-domination 34. Woolley M (2015) Woolley’s Wearables – Android & Bluetooth - Bluetooth Blog. Bluetooth Blog. Available at: http://blog.bluetooth.com/woolleys-wearables-android-bluetooth 35. Yang J, Wei D, Tang L, Song X, Luo W, Chu J, Gao T, Shi H, Du C (2015) Wearable temperature sensor based on graphene nanowalls. RSC Adv 5(32):25609–25615 36. Zachariah T, Klugman N, Campbell B, Adkins J, Jackson N, Dutta P (2015) The internet of things has a gateway problem. In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, Santa Fe, pp 27–32. https://doi.org/10.1145/2699343.2699344 Marios C. Angelides holds a Chair in Computing at Brunel University London. He is a Chartered Engineer (CEng) and a Chartered Fellow of the British Computer Society (FBCS CITP). He holds a BSc and a PhD both from the London School of Economics (LSE) where he also began his academic career as a lecturer more than 25 years ago. For over two decades, he has been researching multimedia communications with a focus on the application of MPEG standards and game theory through evolutionary computation. In 1995, Kluwer published his first book, BMultimedia Information Systems^. In 2011, Wiley published his edited book on BMPEG Applications^ and in 2014 IEEE/Wiley published his edited book on BDigital Games^. 26448 Multimed Tools Appl (2018) 77:26397–26448 Lissette Andrea Cabello Wilson is a lecturer in the Department of Computer Engineering at Escuela Superior Politecnica del Litoral (ESPOL) in Ecuador. She holds a BSc in Telematics Engineering from ESPOL and an MSc in Computer Communication Networks from Brunel University London where she is currently a research associate in the Electronics and Computer Engineering Department. She is also an associate lecturer in the Universidad de Guayaquil where she has participated in the accreditation process to obtain the CEAACES certification. Her research interests include network security and virtualization. Paola Burneo Echeverría works in industry specializing in VOIP networks and applications. She holds a BSc in Electronics Engineering (Telecommunications) from the Universidad Politécnica Salesiana in Cuenca in Ecuador and an MSc in Computer Communication Networks from Brunel University London where she is currently a research associate in the Electronics and Computer Engineering Department. Her research interests include data mining and software defined networks.

Journal

Multimedia Tools and ApplicationsSpringer Journals

Published: Jun 4, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off