TY - JOUR
AU - Lee,, Hyeonkyeong
AB - Abstract Background Living labs are user-focused experimental environments in which users and producers co-create innovative solutions in real-life settings. The aim of this study was to review and synthesize health-related studies that used the living labs approach. Methods An integrative literature review of 15 studies was conducted on the application of living lab principles and their usefulness for investing health problems. Three reviewers independently used methodological assessment tools to evaluate the data quality. Results Twelve of 15 studies were published during the past 5 years, while 14 of the 15 studies were conducted in Europe. Older adults were the target population in 9 of the 15 studies. The research topics varied, including detecting and monitoring daily life, fall prevention and social support. All the studies applied multi-method approaches and a real-life setting. Use of the living lab approach appeared to improve the quality of life, physical and social health and cognitive function of the target populations. Conclusions The results showed that the living lab approach was more commonly used to investigate health problems in older adult populations. Living lab appears to be an appropriate method for developing innovative solutions to improve the health of vulnerable groups. Introduction Living labs are a real-life experimental environment where both users and producers co-create innovative solutions through Public-Private-People Partnerships, fostering user-driven open innovation (https://enoll.org).1 Since Mitchell2 first introduced living labs into urban planning and city design, the methodology has been adopted by many researchers and organizations. The approach emphasizes participation of all stakeholders in the developmental process of a product, service or system.3 Living labs operate as intermediaries among citizens, organizations, companies, cities and regions for joint value co-creation, prototyping or validation to expand innovation and businesses.4 In addition to user-centric innovation, stakeholder engagement and real-life contexts, other scholars also mentioned creation of sustainable value as a key component of living labs.3 According to the European Network of Living Labs (EnoLL), in 2018, more than 440 living labs had been recognized over the past 12 years and there are currently more than 150 active living lab worldwide. About 52% of work by living labs involves health and wellbeing (https://www.iemed.org/recursos-compartits/pdfs/ENOLL%20introduction.pdf). In responding to substantial social health challenges, living labs have gained attention as an innovative approach to developing specific health problem solutions for aging populations and chronic diseases.5 In recent years, the living lab approach has frequently been applied to the development of health devices such as personal hygiene and health using wearable devices6 and smart home assistant for older adults.7 In addition, studies have reported applying living labs to varied stakeholders such as patients,8,9 clinical settings8,10 and communities.10,11 Consequently, living labs are increasingly being applied to address issues associated with aging,7 resulting in more health care researchers becoming aware of living labs as a participatory research approach. Compared with traditional research methodology, living labs allow researchers to integrate technical innovations into health interventions through inter-professional cooperation. When compared with community-based participatory research (CBPR), which has similar characteristics as living labs, such as user involvement, stakeholder participation and co-learning,12 living labs are applicable to a broader spectrum of problems and are not limited to community-based applications. However, evidence regarding the impact of health-related living lab research is scarce. Given the lack of theoretical knowledge on living labs in the health field, we performed an integrative review. Integrative reviews are used to consolidate previous empirical or theoretical studies to provide a comprehensive understanding of specific phenomena.13–15 The aim of this study was to perform an integrative review of the individual- and population-based health related studies that used a living lab approach. The specific aims were (i) to identify which population-specific health problems were addressed using a living lab approach; (ii) to summarize whether the basic principles of the living lab approach were applied to the studies; and (iii) to identify the usefulness of the living lab approach to investigate health problems. Methods The integrative review was conducted in five stages: (i) problem formulation, (ii) literature search, (iii) data evaluation, (iv) data analysis and (v) presentation.16–18 Problem formulation is the same as the specific aims mentioned above. Literature search A literature review was conducted to extract studies for the integrative review. PubMed, CINAHL, Embase and PsycINFO were searched using the search term ‘living lab.’ ‘Living lab’ has been used here as a comprehensive term for diverse innovation. Currently, there are few theoretical knowledge about living lab, we searched for a ‘living lab’ term to identify how it was used theoretically in the health field. Living lab research has been actively reported since the 2000s, yet there are very few studies; therefore, the entire body of peer-reviewed English language research literature on living labs published prior to September 2018 was examined without limiting the year of publication. Grey literature like dissertations and reports were excluded; only peer-reviewed journal articles were considered. Inclusion and exclusion criteria This review included studies using living lab methods with individual- and population-based groups in the physical, mental and social health research areas. Quantitative, qualitative or mixed methods studies were considered. Studies were excluded if they were not relevant to living labs or to physical, mental or social health; were not individual- or population-based (e.g. a study on home energy management system); or were concept papers. Studies that did not meet the inclusion criteria were not selected. Search outcomes After the original 136 articles were retrieved, 94 were reviewed after removal of duplicates and 77 did not meet the inclusion criteria (figure 1). Initial article screening included independent review of titles and abstracts by three researchers (J.K., Y.K. and H.J.) who then reached agreement on study inclusion. Where consensus could not be reached, a fourth reviewer (H.L.) resolved the disagreement. Full texts (n = 17) were obtained for studies that met the inclusion criteria and examined; two of these studies were excluded because they came from the same project, so 15 papers were included in the analyses. Figure 1 Open in new tabDownload slide Study selection process Figure 1 Open in new tabDownload slide Study selection process Quality assessment All the analyzed papers were subjected to a quality assessment to determine if the design and implementation of these studies are appropriately reported, and to what extent these papers can make a meaningful contribution to our understanding of the field.17,18 The 15 papers selected for the study were assessed using a quality appraisal criteria checklist based on Bowling16 and Gazarian,17 and modified by Kangasniemi et al.18 The quality appraisal was based on six structural methodology domains: aim, study design, method, theoretical framework, limitations and implications, using a three-point scale coded with ‘yes’, ‘poor’ or ‘not reported’ (table 1). Table 1 Quality appraisal criteriaa Methodological details . Aguirre (2016) . Ahmed (2017) . Alaoui (2015) . Blain (2014) . Boulay (2011) . Bygholm (2014) . Konstantinidis (2017) . Krieg-Brückner (2010) . Nøhr (2009) . Ogonowski (2016) . Phanareth (2017) . Sevrin (2015) . Swinkels (2018) . Ward (2015) . Wu (2014) . Aims and objectives clearly described Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Study design adequately described Y Y NR NR Y Y Y NR Y Y Y NR Y Y Y Research methods appropriate Y Y Y Y Y Y Y Y Y Y Y P Y Y Y Explicit theoretical framework P P NR Y NR Y P NR NR NR NR NR P NR NR Limitations presented Y Y Y NR Y Y Y Y NR Y Y Y Y Y Y Implications discussed Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Methodological details . Aguirre (2016) . Ahmed (2017) . Alaoui (2015) . Blain (2014) . Boulay (2011) . Bygholm (2014) . Konstantinidis (2017) . Krieg-Brückner (2010) . Nøhr (2009) . Ogonowski (2016) . Phanareth (2017) . Sevrin (2015) . Swinkels (2018) . Ward (2015) . Wu (2014) . Aims and objectives clearly described Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Study design adequately described Y Y NR NR Y Y Y NR Y Y Y NR Y Y Y Research methods appropriate Y Y Y Y Y Y Y Y Y Y Y P Y Y Y Explicit theoretical framework P P NR Y NR Y P NR NR NR NR NR P NR NR Limitations presented Y Y Y NR Y Y Y Y NR Y Y Y Y Y Y Implications discussed Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a Scale: Y, yes; P, poor; NR, not reported. Open in new tab Table 1 Quality appraisal criteriaa Methodological details . Aguirre (2016) . Ahmed (2017) . Alaoui (2015) . Blain (2014) . Boulay (2011) . Bygholm (2014) . Konstantinidis (2017) . Krieg-Brückner (2010) . Nøhr (2009) . Ogonowski (2016) . Phanareth (2017) . Sevrin (2015) . Swinkels (2018) . Ward (2015) . Wu (2014) . Aims and objectives clearly described Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Study design adequately described Y Y NR NR Y Y Y NR Y Y Y NR Y Y Y Research methods appropriate Y Y Y Y Y Y Y Y Y Y Y P Y Y Y Explicit theoretical framework P P NR Y NR Y P NR NR NR NR NR P NR NR Limitations presented Y Y Y NR Y Y Y Y NR Y Y Y Y Y Y Implications discussed Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Methodological details . Aguirre (2016) . Ahmed (2017) . Alaoui (2015) . Blain (2014) . Boulay (2011) . Bygholm (2014) . Konstantinidis (2017) . Krieg-Brückner (2010) . Nøhr (2009) . Ogonowski (2016) . Phanareth (2017) . Sevrin (2015) . Swinkels (2018) . Ward (2015) . Wu (2014) . Aims and objectives clearly described Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Study design adequately described Y Y NR NR Y Y Y NR Y Y Y NR Y Y Y Research methods appropriate Y Y Y Y Y Y Y Y Y Y Y P Y Y Y Explicit theoretical framework P P NR Y NR Y P NR NR NR NR NR P NR NR Limitations presented Y Y Y NR Y Y Y Y NR Y Y Y Y Y Y Implications discussed Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a Scale: Y, yes; P, poor; NR, not reported. Open in new tab Data extraction and analysis The following information was extracted from each article: first author, year, country, target population, stakeholder type, user role, topic, design purpose, innovation type and outcome. All 15 articles were classified according to living lab principles4: multi-method approaches, user engagement, multi-stakeholder participation, real-life setting and co-creation. Three researchers (J.K., Y.K. and H.J.) extracted and analyzed the data and disagreement was resolved by the fourth researcher (H.L.). Results Study characteristics The study characteristics are reported in table 2. Note that the outcomes in table 2 include the expected outcomes from the research on living labs as well as the quantitative figures of these studies. The studies were published between 2009 and March 2018: Three studies (20%) were published between 2009 and 2011 and 12 studies (80%) were published after 2014. Fourteen of the 15 studies were from Europe: France (n = 5), Denmark (n = 3), Greece (n = 1), Germany (n = 2), Spain (n = 1), Netherlands (n = 1) and the United Kingdom (n = 1). The remaining study was from Canada. The preponderance of European studies is likely because of the large European living lab network.19 Table 2 Characteristics of study (n = 15) No. . First author (year)/country . Target population (n)/mean age . Stakeholders type . User role . Topic . Design . Purpose . Innovation type . Outcomes . 1 Aguirre (2016)/ Spain Households (5): people at risk of social exclusion, vulnerable older adults/(–) Red cross Social staff Medical staff Technical workers Engineers Architects Tester Health and social sensors Mixed method To develop social-healthcare system for vulnerable citizens Social sensor Good acceptance of system Increased interaction Reduced response time and social inclusion level between participants via integral approach Provided a remote health and behavior monitoring system for older adults, disability or mental illness 2 Ahmed (2017)/ Canada People with disabilities/(–) Researchers Project partners Clinicians Private industry Tester and feedback Physical environment (mall environment, rehabilitation) Methodology research (program development) To apply the Precede-Proceed model to the development of an inclusive mall environment RehabMall (signs and floor identifier, ramps, new elevator, information kiosk and light) Performance of the overall RehabMaLL (ongoing evaluation, no outcomes presented) 3 Alaoui (2015)/ France Older adults (5)/67–92 years Technical partners (experts) Tester and feedback, sharing experience Social interaction and home environment Methodology research To evaluate usability of social TV at home Social TV applications (FoSIBLE) Satisfaction about social TV services, usability and usefulness of system in the daily life 4 Blain (2014)/ France Older adults/(–) Hospital Nursing home Social worker University Regional aging institute Research laboratories Enterprise, municipalities Idea suggestion and tester Fall prevention Qualitative study To prevent fall injury Home automation Reduced fall-related hospitalizations in Languedoc-Roussillon 5 Boulay (2011)/ France Patients with Alzheimer’s disease (7)/88.5 years Psychologist Engineer Hospital Test and feedback Music therapy game Quantitative study To evaluate usability of MINWii music game for patients with dementia MINWii: Music game by pointing at virtual keyboard with a Wiimote Pistol Improved or at least slow down the decline of patient’s cognitive abilities Provided positive interaction with the caregivers (3 months)a 6 Bygholm (2014)/Denmark Older adults in nursing home (17)/(–) Nurses Nurse assistants Residents Nursing home management Purchasers from the Municipality Researchers Tester and feedback Use of intelligent beds Mixed method To evaluate usability of intelligent beds in a nursing home Intelligent bed Activated innovative thinking on use of Ambient Assisted Living technology (3 months)a 7 Konstantinidis (2017)/ Greece Older adults (14)/73.4 ± 3.8 years Lab of Medical Physics of Medical School Tester and feedback Physical activity, exergaming (active and healthy aging) Quantitative study To evaluate usability front of wFFA To evaluate of the perceived usefulness of wFFA To sort the applications and devices in terms of satisfaction, difficulty and perceived monitoring ability and usefulness Exergaming engine: wFFA (webFitForAll) Screen size of wFFA changed from small to large wFFA had higher usability score (90.4) than did previous version (76.6) wFFA had highest satisfaction level wFFA had higher familiarization score (92.9) than did previous version (87.1) 8 Krieg-Brückner (2010)/Germany Older adults/(–) – – Mobility assistance Methodology research To compensate for diminishing physical and cognitive faculties Rolland wheelchair iWalker walker Bremen Ambient Assisted Living Lab Expectation of mobility assistant improvement 9 Nøhr (2009)/ Denmark Families with one or more diabetic member (8)/ age range 8–67 years Service provider Researchers at Aalborg University IT-consultants from The Danish Technological Institute Advantage group Representatives from the Foundation of Skagen Health Idea suggestion, tester, sharing experience Self-management of chronic diseases Qualitative study To increase the chronic patients’ ability to engage in self-management Utilization of PDA, RFID Understood how diabetes is dealt with in everyday living and what is important to participants Provided perspectives on diabetes (1 year, every Friday, Saturday, Sunday)a 10 Ogonowski (2016)/Germany Older adults (12)/72.8 ± 6.57 years Physician Physiotherapist University Tester and feedback, sharing experience Fall prevention Mixed methods study To develop ICT-based fall prevention exergame Exergame-iStoppFalls Reduced fall risk (inactive participants from 0.29 to −0.27/active participants from 0.68 to −0.03) Improvement of strength and balance Transferred exercise into daily routines Improvement of quality of life and self-efficacy Increased activity and social engagement (6 months)a 11 Phanareth (2017)/Denmark Patients with COPD (93)/ 73.9 years Ministry of Health Hospital, Local Health Clinic Enterprise Patient associations Municipalities Tester Integrated people-centered health services Methodology research (device testing) To improve the health care system for people with long-term conditions Epital care model Developed an Epital care model service, which is a proactive, preventive monitoring system for health care (5 years)a 12 Sevrin (2015)/ France Older adults/(–) – – Daily health monitoring Methodology paper To detect and track in daily activities of the older adults Kinect depth cameras (indoor location tracking system) Expectation of improvement of homecare health management for a better end of life 13 Swinkels (2018)/Netherlands Chronic patients (30)/68 years Health care professionals Enterprises Research institutes Health care insurers Policy makers Tester and feedback E-health application in primary health care Qualitative study To show the process of adoption of e-health in primary care E-health Integrated E-health technologies for implementation in primary health care (3.3 years)a 14 Ward (2015)/ UK Older adults or vulnerable person (14)/ age range 50–80 years Neighborhood Carers Responder Representatives from the assistive technology industry Tester and feedback Assistive technology Mixed methods study To detect usual daily routines using home sensor technologies AroundMe™ service home sensor technologies Promoted independence of the user and provided support for carers (3 months)a 15 Wu (2014)/ France Older adults with mild cognitive impairment (MCI) (6), cognitively intact healthy (CIH) older adults (5)/79.3 years – Tester and feedback Assistive technology physical and social support Mixed methods study To observe robot acceptance in older adults, particularly subsequent to a 1-month direct experience with a robot Human robot No significant differences between the MCI and CIH groups in scores of robot-acceptance (except scores for attitudes toward robots decreased in the CIH group (P = 0.07) No significant differences between MCI and CIH group in task-completion time, errors, or help [only completion time is increased in the MCI group (P = 0.028) (4 weeks)a No. . First author (year)/country . Target population (n)/mean age . Stakeholders type . User role . Topic . Design . Purpose . Innovation type . Outcomes . 1 Aguirre (2016)/ Spain Households (5): people at risk of social exclusion, vulnerable older adults/(–) Red cross Social staff Medical staff Technical workers Engineers Architects Tester Health and social sensors Mixed method To develop social-healthcare system for vulnerable citizens Social sensor Good acceptance of system Increased interaction Reduced response time and social inclusion level between participants via integral approach Provided a remote health and behavior monitoring system for older adults, disability or mental illness 2 Ahmed (2017)/ Canada People with disabilities/(–) Researchers Project partners Clinicians Private industry Tester and feedback Physical environment (mall environment, rehabilitation) Methodology research (program development) To apply the Precede-Proceed model to the development of an inclusive mall environment RehabMall (signs and floor identifier, ramps, new elevator, information kiosk and light) Performance of the overall RehabMaLL (ongoing evaluation, no outcomes presented) 3 Alaoui (2015)/ France Older adults (5)/67–92 years Technical partners (experts) Tester and feedback, sharing experience Social interaction and home environment Methodology research To evaluate usability of social TV at home Social TV applications (FoSIBLE) Satisfaction about social TV services, usability and usefulness of system in the daily life 4 Blain (2014)/ France Older adults/(–) Hospital Nursing home Social worker University Regional aging institute Research laboratories Enterprise, municipalities Idea suggestion and tester Fall prevention Qualitative study To prevent fall injury Home automation Reduced fall-related hospitalizations in Languedoc-Roussillon 5 Boulay (2011)/ France Patients with Alzheimer’s disease (7)/88.5 years Psychologist Engineer Hospital Test and feedback Music therapy game Quantitative study To evaluate usability of MINWii music game for patients with dementia MINWii: Music game by pointing at virtual keyboard with a Wiimote Pistol Improved or at least slow down the decline of patient’s cognitive abilities Provided positive interaction with the caregivers (3 months)a 6 Bygholm (2014)/Denmark Older adults in nursing home (17)/(–) Nurses Nurse assistants Residents Nursing home management Purchasers from the Municipality Researchers Tester and feedback Use of intelligent beds Mixed method To evaluate usability of intelligent beds in a nursing home Intelligent bed Activated innovative thinking on use of Ambient Assisted Living technology (3 months)a 7 Konstantinidis (2017)/ Greece Older adults (14)/73.4 ± 3.8 years Lab of Medical Physics of Medical School Tester and feedback Physical activity, exergaming (active and healthy aging) Quantitative study To evaluate usability front of wFFA To evaluate of the perceived usefulness of wFFA To sort the applications and devices in terms of satisfaction, difficulty and perceived monitoring ability and usefulness Exergaming engine: wFFA (webFitForAll) Screen size of wFFA changed from small to large wFFA had higher usability score (90.4) than did previous version (76.6) wFFA had highest satisfaction level wFFA had higher familiarization score (92.9) than did previous version (87.1) 8 Krieg-Brückner (2010)/Germany Older adults/(–) – – Mobility assistance Methodology research To compensate for diminishing physical and cognitive faculties Rolland wheelchair iWalker walker Bremen Ambient Assisted Living Lab Expectation of mobility assistant improvement 9 Nøhr (2009)/ Denmark Families with one or more diabetic member (8)/ age range 8–67 years Service provider Researchers at Aalborg University IT-consultants from The Danish Technological Institute Advantage group Representatives from the Foundation of Skagen Health Idea suggestion, tester, sharing experience Self-management of chronic diseases Qualitative study To increase the chronic patients’ ability to engage in self-management Utilization of PDA, RFID Understood how diabetes is dealt with in everyday living and what is important to participants Provided perspectives on diabetes (1 year, every Friday, Saturday, Sunday)a 10 Ogonowski (2016)/Germany Older adults (12)/72.8 ± 6.57 years Physician Physiotherapist University Tester and feedback, sharing experience Fall prevention Mixed methods study To develop ICT-based fall prevention exergame Exergame-iStoppFalls Reduced fall risk (inactive participants from 0.29 to −0.27/active participants from 0.68 to −0.03) Improvement of strength and balance Transferred exercise into daily routines Improvement of quality of life and self-efficacy Increased activity and social engagement (6 months)a 11 Phanareth (2017)/Denmark Patients with COPD (93)/ 73.9 years Ministry of Health Hospital, Local Health Clinic Enterprise Patient associations Municipalities Tester Integrated people-centered health services Methodology research (device testing) To improve the health care system for people with long-term conditions Epital care model Developed an Epital care model service, which is a proactive, preventive monitoring system for health care (5 years)a 12 Sevrin (2015)/ France Older adults/(–) – – Daily health monitoring Methodology paper To detect and track in daily activities of the older adults Kinect depth cameras (indoor location tracking system) Expectation of improvement of homecare health management for a better end of life 13 Swinkels (2018)/Netherlands Chronic patients (30)/68 years Health care professionals Enterprises Research institutes Health care insurers Policy makers Tester and feedback E-health application in primary health care Qualitative study To show the process of adoption of e-health in primary care E-health Integrated E-health technologies for implementation in primary health care (3.3 years)a 14 Ward (2015)/ UK Older adults or vulnerable person (14)/ age range 50–80 years Neighborhood Carers Responder Representatives from the assistive technology industry Tester and feedback Assistive technology Mixed methods study To detect usual daily routines using home sensor technologies AroundMe™ service home sensor technologies Promoted independence of the user and provided support for carers (3 months)a 15 Wu (2014)/ France Older adults with mild cognitive impairment (MCI) (6), cognitively intact healthy (CIH) older adults (5)/79.3 years – Tester and feedback Assistive technology physical and social support Mixed methods study To observe robot acceptance in older adults, particularly subsequent to a 1-month direct experience with a robot Human robot No significant differences between the MCI and CIH groups in scores of robot-acceptance (except scores for attitudes toward robots decreased in the CIH group (P = 0.07) No significant differences between MCI and CIH group in task-completion time, errors, or help [only completion time is increased in the MCI group (P = 0.028) (4 weeks)a a Period of engagement for examining outcomes. COPD, chronic obstructive pulmonary disease; PDA, personal digital assistant. Open in new tab Table 2 Characteristics of study (n = 15) No. . First author (year)/country . Target population (n)/mean age . Stakeholders type . User role . Topic . Design . Purpose . Innovation type . Outcomes . 1 Aguirre (2016)/ Spain Households (5): people at risk of social exclusion, vulnerable older adults/(–) Red cross Social staff Medical staff Technical workers Engineers Architects Tester Health and social sensors Mixed method To develop social-healthcare system for vulnerable citizens Social sensor Good acceptance of system Increased interaction Reduced response time and social inclusion level between participants via integral approach Provided a remote health and behavior monitoring system for older adults, disability or mental illness 2 Ahmed (2017)/ Canada People with disabilities/(–) Researchers Project partners Clinicians Private industry Tester and feedback Physical environment (mall environment, rehabilitation) Methodology research (program development) To apply the Precede-Proceed model to the development of an inclusive mall environment RehabMall (signs and floor identifier, ramps, new elevator, information kiosk and light) Performance of the overall RehabMaLL (ongoing evaluation, no outcomes presented) 3 Alaoui (2015)/ France Older adults (5)/67–92 years Technical partners (experts) Tester and feedback, sharing experience Social interaction and home environment Methodology research To evaluate usability of social TV at home Social TV applications (FoSIBLE) Satisfaction about social TV services, usability and usefulness of system in the daily life 4 Blain (2014)/ France Older adults/(–) Hospital Nursing home Social worker University Regional aging institute Research laboratories Enterprise, municipalities Idea suggestion and tester Fall prevention Qualitative study To prevent fall injury Home automation Reduced fall-related hospitalizations in Languedoc-Roussillon 5 Boulay (2011)/ France Patients with Alzheimer’s disease (7)/88.5 years Psychologist Engineer Hospital Test and feedback Music therapy game Quantitative study To evaluate usability of MINWii music game for patients with dementia MINWii: Music game by pointing at virtual keyboard with a Wiimote Pistol Improved or at least slow down the decline of patient’s cognitive abilities Provided positive interaction with the caregivers (3 months)a 6 Bygholm (2014)/Denmark Older adults in nursing home (17)/(–) Nurses Nurse assistants Residents Nursing home management Purchasers from the Municipality Researchers Tester and feedback Use of intelligent beds Mixed method To evaluate usability of intelligent beds in a nursing home Intelligent bed Activated innovative thinking on use of Ambient Assisted Living technology (3 months)a 7 Konstantinidis (2017)/ Greece Older adults (14)/73.4 ± 3.8 years Lab of Medical Physics of Medical School Tester and feedback Physical activity, exergaming (active and healthy aging) Quantitative study To evaluate usability front of wFFA To evaluate of the perceived usefulness of wFFA To sort the applications and devices in terms of satisfaction, difficulty and perceived monitoring ability and usefulness Exergaming engine: wFFA (webFitForAll) Screen size of wFFA changed from small to large wFFA had higher usability score (90.4) than did previous version (76.6) wFFA had highest satisfaction level wFFA had higher familiarization score (92.9) than did previous version (87.1) 8 Krieg-Brückner (2010)/Germany Older adults/(–) – – Mobility assistance Methodology research To compensate for diminishing physical and cognitive faculties Rolland wheelchair iWalker walker Bremen Ambient Assisted Living Lab Expectation of mobility assistant improvement 9 Nøhr (2009)/ Denmark Families with one or more diabetic member (8)/ age range 8–67 years Service provider Researchers at Aalborg University IT-consultants from The Danish Technological Institute Advantage group Representatives from the Foundation of Skagen Health Idea suggestion, tester, sharing experience Self-management of chronic diseases Qualitative study To increase the chronic patients’ ability to engage in self-management Utilization of PDA, RFID Understood how diabetes is dealt with in everyday living and what is important to participants Provided perspectives on diabetes (1 year, every Friday, Saturday, Sunday)a 10 Ogonowski (2016)/Germany Older adults (12)/72.8 ± 6.57 years Physician Physiotherapist University Tester and feedback, sharing experience Fall prevention Mixed methods study To develop ICT-based fall prevention exergame Exergame-iStoppFalls Reduced fall risk (inactive participants from 0.29 to −0.27/active participants from 0.68 to −0.03) Improvement of strength and balance Transferred exercise into daily routines Improvement of quality of life and self-efficacy Increased activity and social engagement (6 months)a 11 Phanareth (2017)/Denmark Patients with COPD (93)/ 73.9 years Ministry of Health Hospital, Local Health Clinic Enterprise Patient associations Municipalities Tester Integrated people-centered health services Methodology research (device testing) To improve the health care system for people with long-term conditions Epital care model Developed an Epital care model service, which is a proactive, preventive monitoring system for health care (5 years)a 12 Sevrin (2015)/ France Older adults/(–) – – Daily health monitoring Methodology paper To detect and track in daily activities of the older adults Kinect depth cameras (indoor location tracking system) Expectation of improvement of homecare health management for a better end of life 13 Swinkels (2018)/Netherlands Chronic patients (30)/68 years Health care professionals Enterprises Research institutes Health care insurers Policy makers Tester and feedback E-health application in primary health care Qualitative study To show the process of adoption of e-health in primary care E-health Integrated E-health technologies for implementation in primary health care (3.3 years)a 14 Ward (2015)/ UK Older adults or vulnerable person (14)/ age range 50–80 years Neighborhood Carers Responder Representatives from the assistive technology industry Tester and feedback Assistive technology Mixed methods study To detect usual daily routines using home sensor technologies AroundMe™ service home sensor technologies Promoted independence of the user and provided support for carers (3 months)a 15 Wu (2014)/ France Older adults with mild cognitive impairment (MCI) (6), cognitively intact healthy (CIH) older adults (5)/79.3 years – Tester and feedback Assistive technology physical and social support Mixed methods study To observe robot acceptance in older adults, particularly subsequent to a 1-month direct experience with a robot Human robot No significant differences between the MCI and CIH groups in scores of robot-acceptance (except scores for attitudes toward robots decreased in the CIH group (P = 0.07) No significant differences between MCI and CIH group in task-completion time, errors, or help [only completion time is increased in the MCI group (P = 0.028) (4 weeks)a No. . First author (year)/country . Target population (n)/mean age . Stakeholders type . User role . Topic . Design . Purpose . Innovation type . Outcomes . 1 Aguirre (2016)/ Spain Households (5): people at risk of social exclusion, vulnerable older adults/(–) Red cross Social staff Medical staff Technical workers Engineers Architects Tester Health and social sensors Mixed method To develop social-healthcare system for vulnerable citizens Social sensor Good acceptance of system Increased interaction Reduced response time and social inclusion level between participants via integral approach Provided a remote health and behavior monitoring system for older adults, disability or mental illness 2 Ahmed (2017)/ Canada People with disabilities/(–) Researchers Project partners Clinicians Private industry Tester and feedback Physical environment (mall environment, rehabilitation) Methodology research (program development) To apply the Precede-Proceed model to the development of an inclusive mall environment RehabMall (signs and floor identifier, ramps, new elevator, information kiosk and light) Performance of the overall RehabMaLL (ongoing evaluation, no outcomes presented) 3 Alaoui (2015)/ France Older adults (5)/67–92 years Technical partners (experts) Tester and feedback, sharing experience Social interaction and home environment Methodology research To evaluate usability of social TV at home Social TV applications (FoSIBLE) Satisfaction about social TV services, usability and usefulness of system in the daily life 4 Blain (2014)/ France Older adults/(–) Hospital Nursing home Social worker University Regional aging institute Research laboratories Enterprise, municipalities Idea suggestion and tester Fall prevention Qualitative study To prevent fall injury Home automation Reduced fall-related hospitalizations in Languedoc-Roussillon 5 Boulay (2011)/ France Patients with Alzheimer’s disease (7)/88.5 years Psychologist Engineer Hospital Test and feedback Music therapy game Quantitative study To evaluate usability of MINWii music game for patients with dementia MINWii: Music game by pointing at virtual keyboard with a Wiimote Pistol Improved or at least slow down the decline of patient’s cognitive abilities Provided positive interaction with the caregivers (3 months)a 6 Bygholm (2014)/Denmark Older adults in nursing home (17)/(–) Nurses Nurse assistants Residents Nursing home management Purchasers from the Municipality Researchers Tester and feedback Use of intelligent beds Mixed method To evaluate usability of intelligent beds in a nursing home Intelligent bed Activated innovative thinking on use of Ambient Assisted Living technology (3 months)a 7 Konstantinidis (2017)/ Greece Older adults (14)/73.4 ± 3.8 years Lab of Medical Physics of Medical School Tester and feedback Physical activity, exergaming (active and healthy aging) Quantitative study To evaluate usability front of wFFA To evaluate of the perceived usefulness of wFFA To sort the applications and devices in terms of satisfaction, difficulty and perceived monitoring ability and usefulness Exergaming engine: wFFA (webFitForAll) Screen size of wFFA changed from small to large wFFA had higher usability score (90.4) than did previous version (76.6) wFFA had highest satisfaction level wFFA had higher familiarization score (92.9) than did previous version (87.1) 8 Krieg-Brückner (2010)/Germany Older adults/(–) – – Mobility assistance Methodology research To compensate for diminishing physical and cognitive faculties Rolland wheelchair iWalker walker Bremen Ambient Assisted Living Lab Expectation of mobility assistant improvement 9 Nøhr (2009)/ Denmark Families with one or more diabetic member (8)/ age range 8–67 years Service provider Researchers at Aalborg University IT-consultants from The Danish Technological Institute Advantage group Representatives from the Foundation of Skagen Health Idea suggestion, tester, sharing experience Self-management of chronic diseases Qualitative study To increase the chronic patients’ ability to engage in self-management Utilization of PDA, RFID Understood how diabetes is dealt with in everyday living and what is important to participants Provided perspectives on diabetes (1 year, every Friday, Saturday, Sunday)a 10 Ogonowski (2016)/Germany Older adults (12)/72.8 ± 6.57 years Physician Physiotherapist University Tester and feedback, sharing experience Fall prevention Mixed methods study To develop ICT-based fall prevention exergame Exergame-iStoppFalls Reduced fall risk (inactive participants from 0.29 to −0.27/active participants from 0.68 to −0.03) Improvement of strength and balance Transferred exercise into daily routines Improvement of quality of life and self-efficacy Increased activity and social engagement (6 months)a 11 Phanareth (2017)/Denmark Patients with COPD (93)/ 73.9 years Ministry of Health Hospital, Local Health Clinic Enterprise Patient associations Municipalities Tester Integrated people-centered health services Methodology research (device testing) To improve the health care system for people with long-term conditions Epital care model Developed an Epital care model service, which is a proactive, preventive monitoring system for health care (5 years)a 12 Sevrin (2015)/ France Older adults/(–) – – Daily health monitoring Methodology paper To detect and track in daily activities of the older adults Kinect depth cameras (indoor location tracking system) Expectation of improvement of homecare health management for a better end of life 13 Swinkels (2018)/Netherlands Chronic patients (30)/68 years Health care professionals Enterprises Research institutes Health care insurers Policy makers Tester and feedback E-health application in primary health care Qualitative study To show the process of adoption of e-health in primary care E-health Integrated E-health technologies for implementation in primary health care (3.3 years)a 14 Ward (2015)/ UK Older adults or vulnerable person (14)/ age range 50–80 years Neighborhood Carers Responder Representatives from the assistive technology industry Tester and feedback Assistive technology Mixed methods study To detect usual daily routines using home sensor technologies AroundMe™ service home sensor technologies Promoted independence of the user and provided support for carers (3 months)a 15 Wu (2014)/ France Older adults with mild cognitive impairment (MCI) (6), cognitively intact healthy (CIH) older adults (5)/79.3 years – Tester and feedback Assistive technology physical and social support Mixed methods study To observe robot acceptance in older adults, particularly subsequent to a 1-month direct experience with a robot Human robot No significant differences between the MCI and CIH groups in scores of robot-acceptance (except scores for attitudes toward robots decreased in the CIH group (P = 0.07) No significant differences between MCI and CIH group in task-completion time, errors, or help [only completion time is increased in the MCI group (P = 0.028) (4 weeks)a a Period of engagement for examining outcomes. COPD, chronic obstructive pulmonary disease; PDA, personal digital assistant. Open in new tab Older adults were the target population in nine studies; three studies focused on patients with specific diseases such as Alzheimer’s disease, chronic obstructive pulmonary disease and other chronic diseases; and one study focused on people with disabilities. The other study populations were urban and rural households and families with a diabetic family member. Twelve studies had stakeholders such as academia, the public sector, the private sector and individuals. The user role in 13 studies included ‘tester’ and ‘feedback provider’; two studies had ‘idea generator’ user roles and three studies had ‘sharing experience’ user roles. Regarding study design, five were mixed methods studies; five were methodology papers such as device testing and protocol development; three were qualitative studies; and two were quantitative studies. Quality appraisal Five studies were deemed to report six criteria according to all three evaluators (Y.K., H.J. and J.K.). The evaluators agreed that these studies described the aim, method, design and theoretical framework. One of the five studies met all six criteria appropriately. They also agreed to evaluate the ‘theoretical framework’ as described in each paper including technological framework. Only two studies described the theoretical framework, such as explaining the theories behind technological development.10,20 All studies included at least three or more criteria and clearly described the aim, objectives and implications. Thirteen studies reported the study limitations, such as problems related to the developed product and related technology. Research topics Nine of the 15 studies were related to older adult health, including detection and monitoring improvements in daily life (n = 3), fall prevention (n = 2), physical activity (n = 1), social support (n = 1), mobility assistance (n = 1) and the safety of older adults in nursing homes (n = 1). These studies all suggested that the living lab approach is effective for investigating older adults’ health problems and physical environment. There were three studies on health management systems such as the e-health environment (n = 2) and integrated health care systems for patients with long-term conditions. One study focused on physical environment changes to increase the activity of people with disabilities. The remaining study focused on chronic disease self-management for families with a diabetic family member. Basic principles of the living lab approach According to the Living Labs Methodology Handbook,4 there are five key common elements of the living labs method: multi-method approach, user engagement, multiple stakeholders, real-life settings and co-creation of an environment for innovation. In this review, the ‘multi-method approach’ element was evaluated in terms of the use of quantitative and qualitative data collection methods, as well as the various methods used in the studies. The ‘user engagement’ element was related to users’ involvement in influencing the innovation process. The involvement of various stakeholders (e.g. related agency stakeholders, experts, or technology providers) in the research process was assessed as ‘multi-stakeholder participation’. The ‘real-life setting’ element was evaluated in terms of whether the study environment was a ‘test bed’ (e.g. city, hospital, nursing home, participants’ home), or an actual site where the services and products would be used. ‘Co-creation’ focused on the use of user-centered innovation activities, such as the development of products or services in collaboration with multiple stakeholders in real-life settings. Using these criteria, we examined whether the studies applied the living lab principles (figure 2). All studies were scored based on the presence or absence of each principle (1 = present, 0.5 = somewhat present, 0 = absent). All the studies applied the principles of multi-method approaches and real-life settings. The user-engagement principle was applied to all except two studies.1,20 Regarding the multi-stakeholder element, 12 studies included multiple stakeholders encompassing four sectors: health care profession, private sector, public sector and others. Health care profession and private sector stakeholders were involved in 80% of all studies. Public sector stakeholders were involved in 47% of all studies and ‘other’ stakeholders were involved in 33%. The principle of co-creation was applied in ten studies and partially applied in four studies. Figure 2 Open in new tabDownload slide Application of the living lab principles in the selected studies Figure 2 Open in new tabDownload slide Application of the living lab principles in the selected studies Innovative development for health All the studies used innovative methods to develop products and implement approaches. The living lab products were varied, and included objects (e.g. an autonomous wheelchair), services, technology and systems (e.g. health management systems). Specific products included an exergame for older adults’ physical health21; a home sensor 20,22 and robot23 related to daily activity and home automation10; an exergame for fall prevention24; a music game for patients with dementia25; an intelligent bed for nursing home residents26; and a Rolland wheelchair and an iWalker for people with disabilities.20 In addition, social TV27 was applied to improve social support for older adults; a personal digital assistant was applied to interview the family members of people with diabetes28; and a mall environment was renovated to accommodate people with disabilities, which involved adding signs and floor identifiers at key locations, a large elevator for wheelchairs and new family washrooms.29 Finally, health management systems applied e-health or integrated health systems.8,9,11 Improvement of physical, social and cognitive health Outcomes of the living lab approach were investigated using qualitative and quantitative methods (n = 11). The studies reported that the use of living labs helped improve quality of life, physical and social health and cognitive function. Eight of the 11 studies reported the period of engagement for examining outcomes (4 weeks - 5 years). Three of these studies have been conducted for 3 months. In mixed-methods studies, people at risk of social exclusion showed good health care system acceptance and increased social interaction11; older adults, nurses and assistants in a nursing home activated an innovative use of ambient assisted living technology26; perceived fall risk score was reduced in an older population; older adults addressed quality of life and self-efficacy improvements24; and independence in daily living was promoted among older adults through the use of home sensor technologies (e.g. ambient heat detector, wireless door contacts).23 In three qualitative studies, older adults reduced fall-related hospitalizations10; families with a diabetic member increased their understanding of diabetes management and support within everyday living28; and patients with chronic disease benefited from using e-Health for self-management.9 In two quantitative studies, patients with Alzheimer’s disease showed improvements in cognitive abilities,25 and older adults reported improved exergame usability scores over a previous version of the exergame.21 Discussion Living labs have been recognized as innovative participatory research methods because they allow users to participate in the design, development and creation of innovative solutions to health issues in a real-life setting.4 Our results showed that the living lab approach was commonly used to investigate health problems, particularly in older adult populations and patients. Participants contributed to solutions by suggesting ideas, sharing their experience and testing devices or giving feedback. This involvement was essential to developing solutions for older peoples’ health issues. More importantly, the involvement itself would be a part of effective intervention because previous studies have shown that older adults’ social participation decreases depression and increases life satisfaction30 and quality of life.31 Vulnerable groups such as people with disabilities and people at risk of social exclusion might benefit more from living lab approaches. They might also require different types of interventions, from community social and educational support to medical technology. Ahmed et al.29 used a living lab approach to provide opportunities for a people with disabilities to experience an inclusive mall environment. Another study aimed to improve quality of life by applying the Information and Communication Technology (ICT) Social Health Care service to residents living in rural mountain regions.11 Davis and Farmer32 reported that it was necessary to include ‘invisible’ populations such as the homeless, marginalized persons and people with disabilities to study social living-lab-based rural health service technology. The living lab approach was also applied to an environmental health policy study to reduce health inequalities among vulnerable regions and residents.33 These studies suggest that living labs are appropriate for developing health care services and investigating health problems for remote, rural areas and vulnerable groups. Most investigations reviewed in our study applied the basic principles of the living lab approach that collectively lead to innovation. However, the number and the roles of multi-stakeholders were relatively limited in most of the studies we reviewed. Swinkels et al.9 stated that collaboration of multi-stakeholders is essential and its success depends on the stakeholders’ collective efforts. Seong et al.34 mentioned that users’ active participation and public–private citizen collaborations are important to creating innovative results. Our study found diverse types of stakeholders in the analyzed studies. Health care professions were the most common, whereas non-governmental organizations and patient associations participated less. Enterprises participated more in the private sector. Swinkels et al.9 reported that entrepreneur stakeholders were expected to collaborate with other stakeholders to enter new markets, and to realize sustainable and interoperable ICT infrastructures. Therefore, private sector stakeholders, including enterprise and technology development companies, are important to the sustainability of solutions in terms of their real-life application, product development and commercial marketability. Further, the co-learning process provides an opportunity to approach solutions from users’ perspectives and develop solutions in collaboration with the user-stakeholders (i.e. patient association participation).8 The real-life setting and multi-method approaches of living labs also foster user participation.35 In an ICT-based fall prevention project, the users participated in the research design, interviewed other users to derive user requirements and tested the developed system in participants’ homes.24 Users were observed through video and field notes to evaluate the usefulness of intelligent beds in nursing homes and the results were discussed and reflected in innovation activities.26 The living lab principles are similar to the principles of CBPR, which is also a popular approach for promoting community health. The common elements between living labs and CBPR are that target populations participate directly in defining and investigating the problem; multiple stakeholders also directly participate in idea generation and problem-solving; and the problem-solving process uses a bottom-up rather than a top-down approach.4,18,36 However, there are also several differences in the two approaches.12,36–40 The living lab approach is comprehensive and encompasses all areas of health, culture, energy, education and technology.4,16 In contrast, CBPR focuses more on fostering social change in target communities to address health problems and eliminate health inequity.36–38 In other words, living labs seek to address social problems through innovative technology,1,22,23 whereas CBPR emphasizes strengthening community capacity to achieve health equity thorough social interaction and community resources.12 In addition, CBPR rigorously applies the research process based on a theoretical framework in the research design, implementation and evaluation,36 resulting in data-based objective outcomes. In contrast, most living lab studies use subjective evaluations based on participant feedback and responses.9,10 The reason for using more subjective evaluations is that living labs are more focused on participants’ responses for investigating problem than theoretical procedures. There were some limitations in this study. First, the study was limited to peer reviewed journals—grey literature without peer review process was excluded (e.g. dissertation and reports). Reports that are in progress may be excluded, so further investigation is needed. Second, one-third of the included studies were methodology papers, which limited identification of the outcomes of living labs. Our findings showed that information about the research design and evaluation was not always reported in the articles. There are many living lab projects, but there are few where the research findings are systematically published in academic journals. More active participation of academia is expected in the future. Conclusions This study showed that the living lab approach was used for improving quality of life, physical and social health and cognitive function mostly in older adult populations. Two principles of living lab, multi-method approach and real-life setting, were applied to all studies, but more attention is required to ensure multi-stakeholder participation, including stakeholders from the private sector and academia. Living labs are useful for developing innovative solutions to improve health, particularly in vulnerable groups. Funding This research was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP; Ministry of Science, ICT & Future Planning) (NRF-2017R1A2B4008671). Conflicts of interest: None declared. Key points The living lab approach—a research methodology for sensing, prototyping, validating and refining complex solutions in multiple and evolving real-life contexts—is particularly useful for investigating the health problems of older adults. Living labs can be used to develop innovative solutions for improving the health of high-risk populations, such as those in remote rural areas and other vulnerable groups. The living lab approach is especially appropriate for exploring and developing health policies designed to reduce health inequities in vulnerable populations. References 1 Sevrin L , Noury N , Abouchi N , et al. Preliminary results on algorithms for multi-kinect trajectory fusion in a living lab . IRBM 2015 ; 36 : 361 – 6 . Google Scholar Crossref Search ADS WorldCat 2 Mitchell WJ . Me++: The Cyborg Self and the Networked City . Cambridge, MA : MIT Press , 2004 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 3 Feurstein K , Hesmer A , Hribernik K , Thoben K-D , Schumacher J . Living labs—a new development strategy. In: Schumacher J , Niitamo VP , editors. European Living Labs—A New Approach for Human Centric Regional Innovation . Berlin : Wissenschaftlicher Verlag , 2008 ; 1 – 14 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 4 Malmberg K , Vaittinen I , Ståhlbröst A , et al. D2.2 Living Labs Methodology Handbook . ENoLL, 2017 ; 1 – 56 . 5 van Geenhuizen M , Guldemond N . Living labs in health innovation: critical factors in their application. In: IEEE, editor. 2013 International Conference on Engineering, Technology and Innovation (ICE) & IEEE International Technology Management Conference: 24–26 June 2013; Hague, Netherlands. Picscataway NJ: IEEE, 2013 ; 1 – 9 6 Su SW , Hsieh YT . Integrated metal-frame antenna for smartwatch wearable device . IEEE Trans Antennas Propag 2015 ; 63 : 3301 – 5 . Google Scholar Crossref Search ADS WorldCat 7 Hawk N , Romine M , Bartle G . The living labs: innovation in real-life settings . Q Rev Distance Educ 2012 ; 13 : 225 . OpenURL Placeholder Text WorldCat 8 Phanareth K , Vingtoft S , Christensen AS , et al. The Epital Care model: a new person-centered model of technology-enabled integrated care for people with long term conditions . JMIR Res Protoc 2017 ; 6 : e6 . Google Scholar Crossref Search ADS PubMed WorldCat 9 Swinkels ICS , Huygens MWJ , Schoenmakers TM , et al. Lessons learned from a living lab on the broad adoption of ehealth in primary health care . J Med Internet Res 2018 ; 20 : e83 . Google Scholar Crossref Search ADS PubMed WorldCat 10 Blain H , Abecassis F , Adnet PA , et al. Living lab falls-MACVIA-LR: the falls prevention initiative of the European innovation partnership on active and healthy ageing (EIP on AHA) in Languedoc-Roussillon . Eur Geriatr Med 2014 ; 5 : 416 – 25 . Google Scholar Crossref Search ADS WorldCat 11 Aguirre E , Led S , Lopez-Iturri P , et al. Implementation of context aware e-health environments based on social sensor networks . Sensors (Basel) 2016 ; 16 : 310 . Google Scholar Crossref Search ADS PubMed WorldCat 12 Wallerstein N , Duran B , Oetzel JG , Minkler M . Community-Based Participatory Research for Health: Advancing Social and Health Equity . Hoboken, NJ : John Wiley & Sons , 2017 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 13 Whittemore R , Knafl K . The integrative review: updated methodology . J Adv Nurs 2005 ; 52 : 546 – 53 . Google Scholar Crossref Search ADS PubMed WorldCat 14 Cooper HM . Scientific guidelines for conducting integrative research reviews . Rev Educ Res 1982 ; 52 : 291 – 302 . Google Scholar Crossref Search ADS WorldCat 15 Russell CL . An overview of the integrative research review . Prog Transplant 2005 ; 15 : 8 – 13 . Google Scholar Crossref Search ADS PubMed WorldCat 16 Bowling A . Research Methods in Health. Investigating Health and Health Services , 2nd edn Buckingham : Open University Press , 2002 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 17 Gazarian PK . Use of the critical decision method in nursing research: an integrative review . Adv Nurs Sci 2013 ; 36 : 106 – 17 . Google Scholar Crossref Search ADS WorldCat 18 Kangasniemi M , Pakkanen P , Korhonen AJ . Professional ethics in nursing: an integrative review . J Adv Nurs 2015 ; 71 : 1744 – 57 . Google Scholar Crossref Search ADS PubMed WorldCat 19 ENoLL. Living Labs project award 2017. Available at: https://issuu.com/enoll/docs/livinglabprojects17_print3 (28 October 2018, date last accessed). 20 Krieg-Brückner B , Röfer T , Shi H , Gersdorf B . Mobility assistance in the Bremen Ambient Assisted Living Lab . GeroPsych (Bern) 2010 ; 23 : 121 – 30 . Google Scholar Crossref Search ADS WorldCat 21 Konstantinidis EI , Bamparopoulos G , Bamidis PD . Moving real exergaming engines on the web: the webFitForAll Case study in an active and healthy ageing living lab environment . IEEE J Biomed Health Inform 2017 ; 21 : 859 – 66 . Google Scholar Crossref Search ADS PubMed WorldCat 22 Ward G , Holliday N , Awang D , Harson D . Creative approaches to service design: using co-creation to develop a consumer focused assistive technology service . Technol Disabil 2015 ; 27 : 5 – 15 . Google Scholar Crossref Search ADS WorldCat 23 Wu YH , Wrobel J , Cornuet M , et al. Acceptance of an assistive robot in older adults: a mixed-method study of human–robot interaction over a 1-month period in the Living Lab setting . Clin Interv Aging 2014 ; 9 : 801 . Google Scholar Crossref Search ADS PubMed WorldCat 24 Ogonowski C , Aal K , Vaziri D , et al. ICT-based fall prevention system for older adults: qualitative results from a long-term field study . ACM Trans Comput Hum Interact 2016 ; 23 : 29 . Google Scholar Crossref Search ADS WorldCat 25 Boulay M , Benveniste S , Boespflug S , et al. A pilot usability study of MINWii, a music therapy game for demented patients . Technol Health Care 2011 ; 19 : 233 – 46 . Google Scholar Crossref Search ADS PubMed WorldCat 26 Bygholm A , Kanstrup AM . Learning from an Ambient Assisted Living Lab: the case of the intelligent bed . Stud Health Technol Inform 2014 ; 205 : 318 – 22 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 27 Alaoui M , Lewkowicz M . Practical issues related to the implication of elderlies in the design process—the case of a Living Lab approach for designing and evaluating social TV services . IRBM 2015 ; 36 : 259 – 65 . Google Scholar Crossref Search ADS WorldCat 28 Nohr C , Bertelsen P , Kanstrup AM . Breaking barriers to self-management of chronic diseases—the MaXi project . Stud Health Technol Inform 2009 ; 143 : 264 – 9 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 29 Ahmed S , Swaine B , Milot M , et al. Creating an inclusive mall environment with the PRECEDE-PROCEED model: a living lab case study . Disabil Rehabil 2017 ; 39 : 2198 – 206 . Google Scholar Crossref Search ADS PubMed WorldCat 30 Amiri E , Khiavi FF , Dargahi H , Dastjerd E . Retirement homes: social participation and quality of life . Electron Physician 2017 ; 9 : 4108 . Google Scholar Crossref Search ADS PubMed WorldCat 31 He Q , Cui Y , Liang L , et al. Social participation, willingness and quality of life: a population‐based study among older adults in rural areas of China . Geriatr Gerontol Int 2017 ; 17 : 1593 – 602 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 32 Davis H , Farmer J . Including the rural excluded: digital technology and diverse community participation. In: Dezuanni M , Foth M , Mallan K , Hughes H , editors. Digital Participation Through Social Living Labs . Oxford : Chandos Publishing , 2018 ; 223 – 44 . Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC 33 Schuurman D , De Marez L , Ballon P . The impact of living lab methodology on open innovation contributions and outcomes . Technol Innov Manag Rev 2016 ; 1 : 7 – 16 . Google Scholar Crossref Search ADS WorldCat 34 Seong JE , Song WC , Park IY . Operating system and examples of living labs . STEPI Insight 2013 ; 127 : 1 – 46 . OpenURL Placeholder Text WorldCat 35 Myung HN , Lee HY . Role of local government in responding to environmental health challenges: a case study of Chungnam . Environ Health Toxicol 2017 ; 32 : e2017023 . Google Scholar Crossref Search ADS PubMed WorldCat 36 Wallerstein N , Duran B The theoretical, historical and practice roots of CBPR. In: Wallerstein N , Duran B , Oetze JG , Minkler M , editors. Community Based Participatory Research for Health: Advancing Social and Health Equity . San Francisco, CA : Jossey-Baasass , 2008 ; 25 – 46 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 37 LaVeaux D , Christopher SJP . Contextualizing CBPR: key principles of CBPR meet the Indigenous research context . Pimatisiwin 2009 ; 7 : 1 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 38 Jacquez F , Vaughn LM , Wagner E . Youth as partners, participants or passive recipients: a review of children and adolescents in community-based participatory research (CBPR) . Am J Commun Psychol 2013 ; 51 : 176 – 89 . Google Scholar Crossref Search ADS WorldCat 39 Seong J , Song W , Jung B , et al. Current status of Korean living labs and its development plan . Sci Technol Policy Institute 2017 ; 9 : 1 – 194 . OpenURL Placeholder Text WorldCat 40 Eriksson M , Niitamo V-P , Kulkki S . State-of-the-Art in Utilizing Living Labs Approach to User-Centric ICT Innovation—A European Approach. Lulea : Center for Distance-spanning Technology, Lulea University of Technology , 2005 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC © The Author(s) 2019. Published by Oxford University Press on behalf of the European Public Health Association. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Living labs for health: an integrative literature review
JO - The European Journal of Public Health
DO - 10.1093/eurpub/ckz105
DA - 2020-02-01
UR - https://www.deepdyve.com/lp/oxford-university-press/living-labs-for-health-an-integrative-literature-review-86cdt5Ry3S
SP - 55
VL - 30
IS - 1
DP - DeepDyve
ER -