Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A knowledge translation intervention to enhance clinical application of a virtual reality system in stroke rehabilitation

A knowledge translation intervention to enhance clinical application of a virtual reality system... Background: Despite increasing evidence for the effectiveness of virtual reality (VR)-based therapy in stroke rehabilitation, few knowledge translation (KT) resources exist to support clinical integration. KT interventions addressing known barriers and facilitators to VR use are required. When environmental barriers to VR integration are less amenable to change, KT interventions can target modifiable barriers related to therapist knowledge and skills. Methods: A multi-faceted KT intervention was designed and implemented to support physical and occupational therapists in two stroke rehabilitation units in acquiring proficiency with use of the Interactive Exercise Rehabilitation System (IREX; GestureTek). The KT intervention consisted of interactive e-learning modules, hands-on workshops and experiential practice. Evaluation included the Assessing Determinants of Prospective Take Up of Virtual Reality (ADOPT-VR) Instrument and self-report confidence ratings of knowledge and skills pre- and post-study. Usability of the IREX was measured with the System Usability Scale (SUS). A focus group gathered therapist experiences. Frequency of IREX use was recorded for 6 months post-study. Results: Eleven therapists delivered a total of 107 sessions of VR-based therapy to 34 clients with stroke. On the ADOPT-VR, significant pre-post improvements in therapist perceived behavioral control (p = 0.003), self-efficacy (p = 0.005) and facilitating conditions (p =0.019) related to VR use were observed. Therapist intention to use VR did not change. Knowledge and skills improved significantly following e-learning completion (p = 0.001) and was th sustained 6 months post-study. Below average perceived usability of the IREX (19 percentile) was reported. Lack of time was the most frequently reported barrier to VR use. A decrease in frequency of perceived barriers to VR use was not significant (p = 0.159). Two therapists used the IREX sparingly in the 6 months following the study. Therapists reported that client motivation to engage with VR facilitated IREX use in practice but that environmental and IREX-specific barriers limited use. Conclusions: Despite increased knowledge and skills in VR use, the KT intervention did not alter the number of perceived barriers to VR use, intention to use or actual use of VR. Poor perceived system usability had an impact on integration of this particular VR system into clinical practice. Keywords: Knowledge translation, Virtual reality, Rehabilitation, Stroke * Correspondence: [email protected] Department of Physical Therapy, Movement Sciences and Rehabilitation, Bouve College of Health Sciences, Northeastern University, 407c Robinson Hall, 360 Huntington Ave, Boston, MA 02115, USA Full list of author information is available at the end of the article © 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Levac et al. BMC Health Services Research (2016) 16:557 Page 2 of 11 Background strategies for integrating VR-based therapy in stroke re- Although recent reviews have synthesized the mounting habilitation using a more multi-faceted approach. evidence for the effectiveness of virtual reality (VR) inter- Systematic reviews of KT strategies in rehabilitation indi- ventions in stroke rehabilitation [1–4], little is known about cate that educational strategies are the most commonly the extent and nature of VR use by physical and occupa- used KT intervention [21–23] but that diversity of study tional therapists outside of a research context [5, 6]. VR is outcomes and interventions limits conclusions about defined as any computer hardware and software system which KT interventions are ideally suited to specific con- that generates simulations of real or imagined environ- texts [23]. However, active multi-component interventions ments with which participants interact using their own improve knowledge and practice behaviors of PTs to a movements [7, 8]. Key VR characteristics of immersion, greater extent than passive strategies [24] and interventions feedback, and interactivity within the virtual world [9] are addressing specific determinants of practice produce small expressed to varying degrees in both rehabilitation-specific to moderate effects [25]. Online KT resources, defined as systems and off-the-shelf video gaming consoles. A recent “e-learning products that translate evidence-based know- survey of Canadian physical therapists (PTs) and occupa- ledge to disseminate information that increases awareness, tional therapists (OTs) found that 46 % of respondents had informs clinical practice and/or stimulates practice change” clinical experience with VR, with the Nintendo Wii being [26] may positively influence self-reported knowledge and the most familiar and accessible system (Glegg SMN, Levac skills [27, 28], and are becoming a popular educational DE, Miller P, Colquhoun H, Wright V: A survey of physical method for health professionals [29]. Indeed, e-learning and occupational therapists’ virtual reality use and learning modules and videos were the preferred educational formats needs, unpublished). Seventy-six percent of respondents of respondents to the Canadian VR use and learning needs were interested in learning more about VR, with top learn- survey (Glegg SMN, Levac DE, Miller P, Colquhoun H, ing needs relating to equipment set up, game familiarity, Wright V: A survey of physical and occupational therapists’ and matching games to client goals (Glegg SMN, Levac virtual reality use and learning needs, unpublished). DE, Miller P, Colquhoun H, Wright V: A survey of physical The purpose of this study was to develop and to evalu- and occupational therapists’ virtual reality use and learning ate a KT intervention that incorporated an online mod- needs, unpublished). Knowledge translation (KT) resources ule and experiential practice to train PTs and OTs in VR may support clinicians motivated to use VR in developing implementation for stroke rehabilitation. The KT inter- the competencies required for evidence-based VR applica- vention was built on previously identified support needs tion [10]. As such, developing accessible, effective KT re- and modifiable barriers in therapist knowledge and skills sources is one way to promote the sustainable integration known to influence VR adoption [5, 20, 30–32]. The KT of VR systems within stroke rehabilitation settings. intervention included a component focused on incorpor- KT resources for PTs and OTs should support clinicians ating motor learning strategies into VR-based rehabilita- in designing, monitoring, adapting and evaluating VR- tion; these findings are reported elsewhere (Levac DE, et based treatment programs [10]. In addition to technical al: Promoting therapists’ use of motor learning strategies competence in VR system operation, clinicians must make during virtual reality-based stroke rehabilitation, in prep- decisions about which systems and games are most appro- aration). The objectives of this study that pertain to this priate for their clients’ stages of recovery, goals, and article were to: 1) evaluate the impact of the intervention physical and cognitive capabilities [10, 11]. Existing frame- on therapists’ confidence related to VR knowledge and works to support decision-making about VR system skills and perceptions of facilitators and barriers related selection [12, 13], task analyses of a growing list of games to VR use; 2) assess usability of the VR system; 3) obtain [14, 15] and qualitative reports of therapist and client per- therapists’ perspectives about the KT intervention and ceptions of VR use in rehabilitation [11, 16–18] are help- VR use in practice; and finally 4) measure the frequency ful, but best practice recommendations to guide VR of continued VR use following the KT intervention. implementation are needed. Known barriers to VR use include lack of time, knowledge, skills and resources; tech- Methods nical issues; and client factors, while practical evidence- The KT intervention took place at 2 sites and was evalu- informed KT resources are proven facilitators [19, 20]. A ated using a pre-post design. Study procedures are also previous KT study supporting VR integration in two in- reported in (Levac DE, et al: Promoting therapists’ use patient acquired brain injury rehabilitation settings tar- of motor learning strategies during virtual reality-based geted modifiable barriers and facilitators using a clinical stroke rehabilitation, in preparation). resource manual, mentoring, technical support and train- ing sessions. Changes in self-reported knowledge and VR system skills was reported by participating therapists [20]. The Motion-capture technology enables players to view present KT study builds on these results, focusing on theirmirrorimage in the virtual environment of the Levac et al. BMC Health Services Research (2016) 16:557 Page 3 of 11 GestureTek Interactive Rehabilitation Exercise (IREX) aspects of IREX usage (i.e., set-up, etc.). Therapists com- software platform (www.gesturetekhealth.com, Ges- pleted the final outcome measures approximately 1 month tureTek, Toronto, ON, Canada). Interaction with the after their final study VR-based therapy session. virtual environment is through body movements to The same process was implemented at site 2. However participate with games that address multiple upper here therapists had the additional opportunity to seek extremity or full body movement goals, while motiv- guidance from a site 1 participant who would act as a ating clients to participate [33]. clinician-mentor, via email, Skype or phone. KT inter- ventions at each site differed only in this opportunity for Procedures mentorship, which was included to explore whether it PTs and OTs were recruited from two rehabilitation centers would impact VR use at either site. At the end of the with inpatient and outpatient stroke rehabilitation units in study, a focus group was conducted at each site to cap- two distinct urban centers in Ontario, Canada. Each site had ture therapists’ experiences with VR use. Following the acquired the IREX for clinical and/or research purposes, completion of the KT intervention, participants were however therapists had not yet been trained on its use. The invited to take part in the sustainability phase, which in- KT intervention was delivered at each site over 5 months, volved monitoring their use of the VR system over six with a staggered start time 8 months later for site 2. months; the research team avoided follow-up communi- During the first month, site 1 therapists completed the cation with the therapists during this time (Fig. 1). self-paced e-learning modules outside of work hours, and also participated in face to face hands-on learning, audit Intervention and feedback sessions. Over the next 4 months, each ther- The Template for Intervention Description and Replica- apist identified up to 4 patients with the following criteria: tion (TIDieR) Checklist for reporting interventions was 1) experienced a stroke within the last 12 months, 2) re- used to guide our intervention description below. A simi- ceiving inpatient or outpatient PT and/or OT services lar description can be found in (Levac DE, et al: Promot- targeting motor skills, and 3) sufficient cognitive and ing therapists’ use of motor learning strategies during physical skills to engage in VR activities (as determined by virtual reality-based stroke rehabilitation, in preparation). treating therapist). Therapists completed up to four ses- sions of VR-based therapy per patient. Therapists were 1. E-learning modules: Three online modules provided free to choose the length of time they would use the IREX foundational knowledge about clinical VR use. The with each client, and were responsible for all technical first module contained information about IREX Fig. 1 Study procedures, including timing of interventions and outcome measurement Levac et al. BMC Health Services Research (2016) 16:557 Page 4 of 11 operation and game characteristics, while the last two learning sessions. Feedback was individualized to each modules focused on the application of motor learning clinicians’ performance and learning needs. strategies to VR-based therapy. Outcomes related to 3. Experiential learning: Clinicians used the IREX the integration of motor learning strategies are system with up to 4 clients. Feedback from the study reported in (Levac DE, et al: Promoting therapists’ use investigators about IREX use was available on of motor learning strategies during virtual reality- request via phone, email or in-person support during based stroke rehabilitation, in preparation). Two co- this period. investigators, with feedback from other authors, de- 4. Didactic reminders: Therapists received weekly veloped the modules by integrating their clinical and e-mails from the PI identifying ‘tips’ for VR use from research experience using VR. Embedded video clips the e-learning modules during the experiential phase. were filmed with consenting client and staff volun- 5. Mentorship: Interested clinicians at site 2 were teers. Interactive learning activities and knowledge matched with 1 of 3 clinician mentors from site 1 checks required learners to integrate and to demon- and received biweekly encouragement to contact strate their knowledge and skills. Based on pilot test- mentors with questions or support needs. Mentors ing of module usability with therapists at [removed also reached out periodically to mentees via email. for review], changes were made to the wording of the knowledge checks and material was removed to make Outcomes the content more manageable. The first module in- Knowledge and skills cluded video clips illustrating IREX game play as illus- Confidence ratings: Using a seven-point Likert scale, trated in Fig. 2.Additional file 1 lists the module therapists rated their confidence in the knowledge and learning objectives. Therapists completed each e- skills associated with each learning objective before and learning module in approximately two hours. Thera- after completing the e-learning modules as well as at the pists were also provided with a print manual contain- end of the study. The time between pre- and post- ing the information presented in the online modules. module confidence ratings was not recorded and 2. Hands-on learning: Hands-on experience using the depended on how long therapists spent completing the IREX was provided during 1 h-long individual module; this ranged from 2 h to 3 days. session and two group sessions led by the PI and a clinician (MB). These sessions provided additional Confidence, barriers and facilitators education and discussion opportunities about VR ADOPT-VR: Therapists completed the Assessing Determi- system operation and trouble-shooting through the nants of Prospective Uptake of Virtual Reality (ADOPT- use of case scenarios. Case scenarios illustrated VR) Instrument pre- and post-study. Based on an extended ‘typical’ stroke rehabilitation clients and included Theory of Planned Behavior [34]), the ADOPT-VR exam- discussion questions related to game selection, ines 11 constructs thought to influence VR adoption, in- progression of difficulty, and tailoring the task to cluding attitudes, self-efficacy, and intention to use VR. The individual client needs. Feedback about VR skills was ADOPT-VR has 24 items scored on a nine-point Likert provided to participants by the PI during the hands-on scale (1: Strongly disagree – 9: Strongly agree) as well as Fig. 2 Two screenshots from the Interactive e-learning module Levac et al. BMC Health Services Research (2016) 16:557 Page 5 of 11 nine multiple-response and short-answer questions. Sum- changes in confidence ratings at a 0.05 alpha level. De- mary scores in 11 categories are reported. Established face scriptive data were summarized using frequency counts, and content validity [30], good internal consistency (alpha and means or medians as appropriate. Focus group re- = 0.876) [35] and responsiveness to change [20] have been sponses were categorized using content analysis [39] by demonstrated. two investigators. System usability Results System Usability Scale: The System Usability Scale (SUS) Participant demographics [36], which has demonstrated reliability, sensitivity and Six PTs and five OTs participated in the study, enrolling concurrent validity [37], was used to evaluate perspectives a total of 34 clients. Therapists at site 1 had a mean of IREX usability at the end of the study. The SUS is a 10 19.3 years (SD 8.1) work experience, and therapists at item questionnaire scored on a five-point Likert scale; site 2 had a mean of 11.4 (SD 9.4) years work experi- scores are converted to percentiles. A score of 68 and ence. Client mean age was 62.8 (SD 16.4) years at site 1 above is considered above average [38]. Using a traditional and 60.1 (SD 15.0) years at site 2. All therapists were school grading scale (from A – F) is recommended to novice IREX users. Pre-study, none reported familiarity complement the score and convey usability [34]. with any of the IREX games. Following the study, famil- iarity range was between 4–10 (mean 6.25) games. Sustainability Therapists who agreed to continue in the sustainability Intervention fidelity phase of the study recorded the frequency of their IREX The target client enrolment for each therapist was 4 clients. use in the six months post-study completion and pro- At site 1, three therapists enrolled four clients, one therap- vided written responses to two questions about why they ist enrolled three clients, and two therapists enrolled two had or had not used the IREX. clients each. At site 2, two therapists enrolled four clients, one therapist enrolled three clients, and two therapists en- Focus groups rolled two clients each. No uptake of the mentoring offered The PI conducted a 1.5-h focus group at each site at the by site 1 clinicians to site 2 clinicians occurred, as measured end of the study to explore therapist perspectives on by participant self-report of phone, email or Skype contact. participation in the KT intervention, use of the IREX in As such, the KT interventions were identical and results for practice, and experience of integrating motor learning both sites are presented together. strategies (reported in [32]]. The focus group was audio- recorded and transcribed for analysis. Knowledge and skills (confidence ratings) Figure 3 illustrates changes in confidence ratings for Analyses each learning objective for the e-learning module. A sig- Non-parametric Wilcoxon signed rank tests evaluated nificant change from pre-module to post-module com- changes between pre- and post-study category scores pletion was observed (Z = −3.180, p =0.001, median and frequency counts for the ADOPT-VR as well as improvement of 3.18 (range 1.46) and this gain was Fig. 3 E-learning module objective confidence ratings. Median pre-module, post-module and post-study confidence ratings on each of the e-learning module objectives. Therapists were asked to rate their confidence, on a scale of 1 to 7, in the knowledge and skills associated with each learning objective. Additional file 1 provides the full list of learning objectives Levac et al. BMC Health Services Research (2016) 16:557 Page 6 of 11 Table 1 ADOPT-VR pre-post study changes and descriptive data fun and motivating method of therapy for clients; at times it seems that clients don't even realize it is ther- ADOPT-VR Category p-value Pre median Post median (range) (range) apy”), its potential to add variety to treatment sessions, Attitude 0.1 6.67 (4.33–7.67) 6.67 (4–9) and its ability to target frequent task repetitions. The two therapists who did not plan on continuing to use Perceived Usefulness 0.167 6.17 (4.00–8.00) 6.67 (3.67–8.33) the IREX cited lack of time and caseload factors as con- Ease of Use 0.677 5.17 (2.67–7.67) 5.17 (1.67–8) tributing reasons. Compatibility 0.363 4.75 (2–7) 5 (1.5–8) Figure 4 illustrates changes in pre- and post-study fre- Social Norms 0.308 5 (1–9) 4.5 (1–6) quency counts of reported barriers to IREX use. A de- Peer Influence 0.194 4.75 (1–7.5) 5 (3–7.5) crease in the frequency of reported barriers to IREX use Superior Influence 0.403 3 (1–6.5) 3 1.5–7.5) from pre to post study was observed (decrease from 50 to 37 barriers; Z = −1.409, p = 0.159). A decrease in the Perceived Behavioural 0.003* 3 (1–6.5) 6.24 (4.5–8) Control frequency of barriers specifically targeted through this Self-Efficacy 0.005* 2.5 (1–8) 7 (6–9) intervention (time to learn about the IREX, access to evidence and educational opportunities; decrease from Facilitating Conditions 0.019* 2.75 (1–6) 4.25 (1–7) 22 to 9; Z = 1.1.633, p = 0.102) were also observed. In Behavioural Intentions 0.21 5.5 (1.67–7.67) 5.33 (1.33–7) short answer responses that accompany the ratings, *Significant at p < =0.05 therapists described their most significant barriers to maintained at the end of the study (median decrease of IREX use as time, short client length of stay, caseload is- 0.12 points (range 1.24) from post-module completion). sues, technical problems with the IREX, and inability to use the system to address OT-related fine-motor goals. Knowledge and skills: ADOPT-VR variable outcomes For example, one therapist wrote: “I would use it much Significant pre-post improvement in therapists’ per- more if games included grasp-release and object manipu- ceived behavioral control (Z = −2.945, p = 0.003), self- lation.” Lack of time to use the IREX in a treatment ses- efficacy (Z = −2.802, p = 0.005) and facilitating conditions sion was the most frequently reported barrier to its use and barriers (Z = −2.352, p =0.019) were observed. No both pre- and post-study. For example, a therapist wrote: other constructs demonstrated significant changes (see “The timeslot necessary to arrange a decent session is Table 1). large and the opportunity to set aside that amount of time, plus travel and not have overlap with other pa- Barriers and facilitators (ADOPT-VR) tients arriving for treatment is a huge barrier.” Following Nine therapists indicated in their post-study ADOPT-VR the study, therapists described two new barriers on the that they intended to continue to use the IREX because ADOPT-VR: inappropriate clients and inability to use of its motivating appeal for clients (e.g. “I think it is a the IREX to target fine motor goals. Nine therapists Fig. 4 Pre-post study changes in the number of barriers to VR adoption. Frequency count of barriers reported by therapists pre- and post-study Levac et al. BMC Health Services Research (2016) 16:557 Page 7 of 11 reported that adjusting game parameters and resolving Focus group technical issues were areas in which they lacked confi- The following three themes were identified regarding dence following the study; two therapists did not report the KT intervention and use of the IREX in practice. any areas of low self-efficacy. On the post-study ADOPT-VR, seven of the 11 thera- Benefits and challenges of the KT intervention pists reported greater familiarity with the IREX because Therapists appreciated the combination of self-paced of the KT intervention and one therapist reported as a online learning with group and individual practice ses- facilitator to IREX use at post-study, having access to pa- sions. The different formats presented in the e-learning tients who were motivated to use VR; three therapists modules (e.g. written information, videos and online ac- did not describe any facilitators. One therapist commen- tivities) were also appreciated, although technical issues ted: “It would be a nice addition to enhance and add (e.g. video speed and quality) were an issue for some. variety to practice for subsets of the patient population. I Given that the e-learning component took place outside could identify a few key individuals who would find the of working hours, therapists reinforced that they would program motivational and uplifting.” No other facilita- not have undertaken this commitment without the reim- tors were mentioned pre- or post-study. bursement provided by the study. Therapists agreed that the opportunity to trial the system/games in person had been helpful; for example, one therapist stated: “I think System usability it’s a necessity, really, to play around with it yourself, Low perceived usability of the IREX was evidenced by a and put yourself in there, and feel the demands and the th SUS mean score of 54.25 (19 percentile, below aver- challenges yourself, will help you decide who to use it age). Figure 5 shows the proportion of respondents who with and how to use it more efficiently for sure.” rated the system by letter grade (from A- F). However, the fact that their first independent use of the IREX was within the context of a regular treatment session was a challenge for some therapists. This influ- Sustainability enced their choice of clients because of the need to bal- Nine therapists indicated their intention to continue to ance client safety and expectation of standard treatment use the IREX on the ADOPT-VR, and five therapists with attending to their own (i.e. therapists’) learning. For (three from site 1 and two from site 2) agreed to partici- some therapists, choice of the first client was premised pate in the sustainability phase of the study. Of the par- on physical factors, such as independent standing bal- ticipating therapists, one used the system at site 1 on ance, while others sought patients whom they identified two occasions and one used the system at site 2 on a as being ‘tech-savvy’. For example, one therapist stated: single occasion. In written responses following the sus- “I personally chose a client that I felt was going to be tainability phase, these therapists indicated that lack of comfortable with the idea of that type of technology. time, location of the VR room, technical difficulties with Someone that was really keen on computers…That they the IREX, and lack of appropriate patients were factors would understand that, there might be some glitches and limiting their use during the post-study period. that ….they were giving me [emphasis] an opportunity. They were helping me. That’s what I told them. They were helping me to learn the system, and at the same time, some things that might be helpful for them, too.” In- deed, therapists expressed that a refresher session during the experiential learning phase would be a useful com- ponent of the KT intervention, particularly when more than a few weeks had passed between the initial learning and the therapist’s first use of the IREX with a client. Therapists reported using only the games that they had learned during the KT intervention, implementing fre- quent repetitions of these games. Client enjoyment was a facilitator of IREX use in practice Client enjoyment was perceived as a facilitator of IREX use by therapists. For example, one therapist stated: Fig. 5 System Usability Scale ratings. An A grade corresponds to a “[VR] gives a different kind of feedback than what you percentile score above 80. A score of 68 is a C grade and scores can do just in the therapy room. It’s a different context. below 51 correspond to a grade of F And the fact that it’s a game… one of my patients told Levac et al. BMC Health Services Research (2016) 16:557 Page 8 of 11 me that he didn’t feel like a patient when he was doing therapy gym, so independent sessions did not fit the stand- it, and he loved that…it was fun, and you know, it made ard treatment model. With travel time taken into consider- him feel very different from being in a hospital patient, ation, less time was available in a session to use the IREX. so… there was great value in that.” Another noted: “They For example, a therapist stated: “….it would be the exception see it more as a fun and more exciting… I may do the that I would do VR. I see some benefit for some patients, but same movement, but it’s more interactive than, let’s say if it’satime–cost ratio. It’s the amount of extra time and work, I’m doing the same repetitive movement without that andwhatittakes me away from to do VR, versusactually video. You know, without that interaction with the doing something else in the amount of time….” Finally, the game….. if you choose the right patients, it gets them ex- short client lengths of stay at these institutions limited the cited about possibly the choices or the games that you’re frequency with which the IREX could be integrated into doing.” Therapists were persistent in encouraging clients treatment, since other treatment goals (i.e. discharge readi- who were initially reluctant to try VR-based therapy as ness) were a priority. they found interest increased with exposure. One ther- apist noted “… those people that didn’t necessarily buy in Discussion right away, by the end were just like, great, engaged, and This study targeted therapist knowledge and skills with motivated, and, you know, really partaking and… right the goal of increasing clinical uptake of an evidence- in there.” based VR system that was already in situ at two clinical sites. Significant increases in self-reports of knowledge Barriers to IREX use in practice and skill were observed and, importantly, sustained over Participants described system and environmental bar- time; however, this change did not translate to increased riers to IREX use in practice. While they were being ini- VR use, although intentions of therapists to continue to tially excited to learn a new technology, hardware and use VR were high. The low sample size during the sus- software malfunctions that interrupted game play, and tainability phase limits the evaluation of long-term be- poor lighting which interfered with IREX operation were havior change in the presence of high intention to use prevalent. Therapists were frustrated when the games VR. The study findings strengthen existing knowledge did not work. Each site had a contact person who could about the facilitators and barriers to VR use with other trouble shoot minor issues related to lighting, but un- inpatient populations (21) and add to the evidence base foreseen software issues with the system itself required by exploring in greater depth the experience of thera- maintenance by company representatives, necessitating pists learning to use VR in practice. replacement of equipment and delaying the study at The KT intervention format and content were tailored each site by several weeks. towards learning needs identified by inpatient clinicians Therapists reported that the IREX games were not al- in a previous study [19]. For example, Glegg et al. [20] ways an ideal fit with therapy goals: “Like, if more games found that a print educational manual was not well [were available], or just modify it so there’s more for the accessed by clinicians working in ABI rehabilitation and middle [functioning] person… because I had patients, suggested e-learning modules, reimbursed training time, where I wasn’t comfortable standing them. They were a lit- and mentoring strategies, all of which were part of this tle bit past sitting and just trying to move, or if they were study. The e-learning module and the workshops em- sitting, like I could work a little bit on their trunk, but there phasized information specific to previously identified was nothing in between.” Another said: “…it wasn’tmatch- therapist learning needs [20]. Confidence ratings imme- ing with his goals, and there weren’t really games that were diately post-module completion indicated the online appropriate for his level of activity in his upper extremity.” learning was effective at improving self-reported know- Therapists reported that they would be inclined to use a ledge and skills in VR use, with improvements sustained different VR system that could be set up in the standard months later. Objective assessments of knowledge and therapy gym and allow for greater independent patient skill change would be valuable adjuncts to future studies use. One therapist said: “I think there’s a value to some of to validate this self-report data. the virtual reality type games, like in the Wii or like the The therapists in this study were novice VR users and Kinect or whatever. If there was a way to translate that they entered the study with positive attitudes towards into clients, perhaps in their home setting with family, I VR, high perceived usefulness ratings of VR, and strong think that could be quite interesting for clients…” intention to use the IREX; these ratings remained high From an environmental perspective, the time required for at post-test. Because intention to use VR was strong at system set up and to transport a client to the room housing the study outset, no changes in this construct were ex- the IREX, which was at some distance from the standard pected at post-test. Self-efficacy and perceived behavioral treatment area, was an issue. Therapists in these two set- control improved significantly post-study, although 82 % tings typically treat multiple clients at once in the large of participants still felt that adjusting game parameters Levac et al. BMC Health Services Research (2016) 16:557 Page 9 of 11 and resolving technical issues were areas in which they research that explores the optimal timing and delivery of lacked confidence. These findings are consistent with mentoring for VR adoption, as well as therapists’ per- previous research [17]. Compatibility, social norms, peer spectives about the desirability, benefits and challenges and superior influence constructs of the ADOPT-VR of various mentoring models would build on the know- demonstrated no change; these findings are not surpris- ledge base in this area. ing, given that these variables were not targeted by the Although the majority of participants indicated they KT intervention, and were not anticipated to change. Of would continue to use the IREX and found VR useful, note is the absence of change on the perceived ease of the five therapists who participated in the sustainability use construct, which is in contrast with previous work phase did not frequently use the system. While the small [20]. A reasonable explanation is the degree of technical sample size and timeframe for follow-up limits difficulty experienced during the current study, as af- generalizability, the reported reasons behind the limited firmed by the SUS findings; these difficulties were expe- VR use are consistent with the barriers identified in this rienced to a greater extent than in previous research and previous research [16, 17]. However, despite this ob- [17]. Technical trouble-shooting will likely always be re- served behavior, a statistically significant increase in fa- quired for VR implementation; having a support person cilitating conditions to VR use was found among the available to manage this aspect is important because larger group of therapists at post-study. Factors such as these technical skills are outside of most therapists’ skill perceived benefits of VR for clients, the novelty of the sets, and the need to trouble-shoot draws time away intervention, or the support provided through the KT from both practice opportunities, and client treatment. intervention may be responsible for this change, al- Indeed, ongoing accessible technical support has been though we did not see an impact on actual use of the raised as an important element of VR adoption [17], and VR system. More research is required to determine the the low SUS scores suggest that additional support may relative influence of specific barriers and facilitators on have been highly valued by these therapists. therapists’ behavioral intention and actual behavior re- Therapists spoke positively of the motivational appeal lated to VR use. To provide greater insight into the sus- of the IREX for clients, which is in accordance with pre- tainability of VR use in the clinical context, and the vious work describing positive client and therapist per- changing nature or importance of the factors influencing ceptions of VR use in practice [5, 11, 17, 40]. However, VR use over time, large-scale follow-up studies are after gaining experience with the IREX, therapists identi- needed. fied new organizational and client-specific barriers to its The poor recruitment of clients by therapists during use, including a lack of match to treatment goals. Bar- the study was likely related to short client length of stay, riers identified through the ADOPT-VR and in the focus some clients declining to participate in the study, and groups are consistent with previous work, although the fact that the IREX was not functioning for several treatment delivery method as a barrier (i.e. treating mul- weeks during the study period at each site. Although this tiple patients at a time) is new to these settings [17]. problem was accommodated by lengthening the trial, This may be related to the client population, with the the decreased ‘dosage’ for therapists, in terms of reduced therapists from the previous study working in pediatric exposure to IREX, likely had an impact on the extent to and adult ABI rehabilitation likely being more accus- which therapists were able to integrate clinical know- tomed to 1:1 interventions. ledge gained during the KT intervention. Such factors The use of an onsite knowledge broker or clinician ‘ex- may make it more challenging to sustain momentum pert user’ is a KT strategy with emerging evidence of ef- and behavior change. fectiveness [41, 42]. At site 2, the on-site RA was an Therapists at these sites now have a base level of tech- expert user with whom clinicians could problem-solve nical and theoretical knowledge about VR use in practice about technical issues. This on-site clinical and technical that they can potentially transfer to other VR systems, support, found to be a facilitator of VR use by Glegg et though specific learning related to games, equipment al. [20], was not available at site 1. Glegg et al. [20] ob- set-up and direct clinical application will be required. served a spontaneous increase in participating therapists Barriers unique to this VR system and settings should mentoring fellow on-site clinicians in VR use. While in- not be generalized to reflect poorly on the potential for formal on-site mentoring was not monitored during this VR implementation as a whole. With respect to the on- study, the use of cross-site mentoring did not occur des- going use of the IREX at these clinical sites, investigators pite explicit attempts at facilitation. It may be that our are working with company representatives to resolve the study did not sufficiently facilitate mentor-mentee trust, technical issues that arose. Study findings are being dis- accessibility, and perceived expertise. Additional efforts cussed with clinical management to strategize around to support the establishment of credibility of mentors by how to address the identified barriers to the use of the mentees may have led to greater success. Future technology. Next steps will involve engaging the Levac et al. BMC Health Services Research (2016) 16:557 Page 10 of 11 therapists in a needs assessment to assist them in Availability of data and materials The datasets during and/or analysed during the current study available from matching the characteristics of other available VR sys- the corresponding author on reasonable request. tems with their specific client and practice setting needs to ensure greater success in future VR implementation Authors’ contributions DL, HS and SG conceived of the study. DL, HS, HC, HF, SG, VDP, and JH efforts. Participated in the design and monitoring of the study. DL and SG created the study materials. DL and SG performed the statistical analysis. DL and PM performed the qualitative analysis. DL Drafted the manuscript. DL, SG, HC, Conclusions HS, PM, JH, VDP, HF, DV critically revised the manuscript for important This study confirmed previous research demonstrating intellectual content. All authors read and approved the final manuscript. therapists’ positive attitudes toward VR and its perceived Competing interests usefulness as a clinical tool for neurorehabilitation. The The authors declare they have no competing interests. KT intervention designed to translate knowledge about use of the VR system to therapists in two stroke rehabili- Consent for publication Verbal informed consent was obtained from the person depicted in the tation units was well-received, and successfully incorpo- Fig. 2 to publish individual, identifiable data. rated e-learning, experiential learning and reminders to significantly increase self-reported confidence, know- Ethical approval and consent to participate University and hospital review boards provided ethical approval for the study ledge and skills in VR use. While these gains were sus- and informed consent was obtained from all participants. tained at follow-up, further research is required to examine sustainability of VR use over time in the clinical Author details Department of Physical Therapy, Movement Sciences and Rehabilitation, setting, as well as the feasibility and potential benefits of Bouve College of Health Sciences, Northeastern University, 407c Robinson more structured mentoring models to support VR use. 2 Hall, 360 Huntington Ave, Boston, MA 02115, USA. Therapy Department, Qualitative findings suggest that system and context- Sunny Hill Health Centre for Children, 3644 Slocan Street, Vancouver, BC V5M 3E8, Canada. School of Rehabilitation Sciences, Faculty of Health Sciences, specific barriers merit ongoing attention in order for University of Ottawa, 200 Lees (A121), Ottawa, ON K1S 5S9, Canada. KT-based interventions that support adoption to be ef- 4 Department of Occupational Science and Occupational Therapy, University fective. Ongoing training of increasing complexity, of Toronto, 160-500 University Ave Toronto, Ontario M5G 1V7, Canada. School of Rehabilitation Sciences, Faculty of Health Sciences, McMaster paired with interspersed practice in clinical application University, Institute of Applied Health Sciences, Room 403, 1400 Main St. of the technology, as well as ongoing assessment of ther- 6 West, Hamilton, ON L8S 1C7, Canada. Bruyere Research Institute, Élisabeth apist support needs are recommended to meet the chan- Bruyère Hospital, Bruyère Continuing Care, 43 Bruyère Street, Ottawa, ON K1N 5C8, Canada. School of Rehabilitation Therapy, Queen’s University, ging needs of therapists over time. Louise D. Acton Building, 31 George Street, Kingston, ON K7L 3 N6, Canada. School of Rehabilitation Science, McMaster University, IAHS Building Room 403, 1400 Main Street West, Hamilton, ON L8S 1C7, Canada. Hamilton Additional file Health Sciences, Regional Rehabilitation Centre, 300 Wellington St. North, Hamilton, ON, Canada. Department of Psychiatry and Behavioural Additional file 1: Description of data: E-learning module learning objec- Neurosciences, DeGroote School of Medicine, McMaster Univerity, 12 Main tives Module 1: “Introduction to the GestureTek VR system. (DOCX 14 kb) Street West, Hamilton, ON l8S 1C7, Canada. Received: 3 December 2015 Accepted: 30 September 2016 Acknowledgements The authors would like to thank the therapists and clients who participated References in this study as well as the research assistants Jennifer Chau, Liliane 1. Laver K, George S, Thomas S, Deutsch JE, Crotty M. Cochrane review: virtual Letourneau and Denise Sartor. Thank you to Marie Brien for participating in reality for stroke rehabilitation. Eur J Phys Rehabil Med. 2012;48:523–30. the clinician workshops at Site 1 and to Laurie Wishart for grantsmanship 2. Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke advice. We appreciate the input and support of clinical managers on the rehabilitation. Cochrane Database Syst Rev. 2015; doi:10.1002/14651858 stroke rehabilitation units at Elizabeth Bruyere Continuing Care and the 3. Darekar A, McFadyen BJ, Lamontagne A, Fung J. Efficacy of virtual reality- Regional Rehabilitation Centre, Hamilton Health Sciences. Andrew Dunn and based intervention on balance and mobility disorders post-stroke: a scoping Craig Ross at BC Children’s Hospital contributed to e-learning module devel- review. J Neuroeng Rehabil. 2015;12:46. opment. Abstracts of preliminary data have been presented at the Canadian 4. Weiss PL, Rand D, Katz N, Kizony R. Video capture virtual reality as a flexible Stroke Congress in 2013 and 2014, the International Conference for Virtual and effective rehabilitation tool. J NeuroEng Rehabil. 2004;1:12. Rehabilitation in 2013, and the American Congress of Rehabilitation Medicine 5. Levac D, Miller P. Integrating virtual reality video games into practice: in 2014. A short paper entitled Integrating motor learning and virtual reality th Clinicians’ experiences. Physiother Theor Pract. 2013;29:504–12. into practice: A knowledge translation challenge was presented at the 10 6. Glegg SM, Holsti L, Stanton S, Hanna S, Velikonja D, Ansley B, et al. Using International Conference on Disability, Virtual Reality and Associated Tech- virtual reality in clinical practice: A multi-site exploratory study. nologies in 2014 and published in the conference proceedings. NeuroRehabil. 2014;35:563–77. 7. Wilson PN, Foreman N, Stanton D. Virtual reality, disability and rehabilitation. Funding Disabil Rehabil. 1997;19:213–20. This project has been generously funded, by a grant from the Ontario 8. Sveistrup H. Motor rehabilitation using virtual reality. J NeuroEng Rehabil. 2004;1:10. Ministry of Health and Long-Term Care, administered and supported by the 9. Brey P. Virtual Reality and Computer Simulation. In: Einer K, Himma K, Tavani Ontario Stroke Network. The views expressed do not necessarily reflect those HT, editors. The handbook of information and computer ethics. Hoboken: of the Ministry or the Ontario Stroke Network. Dr. Levac’s postdoctoral fel- John Wiley & Sons, Inc; 2008. p. 361–84. lowship was supported by the Canadian Institutes of Health Research, the 10. Levac DE, Galvin J. When is virtual reality 'therapy'? Arch Phys Med Rehabil. Canadian Child Health Clinician Scientist Program and NeuroDevNet. 2013;94:795–8. Levac et al. BMC Health Services Research (2016) 16:557 Page 11 of 11 11. Tatla SK, Shirzad N, Lohse KR, Virji-Babul N, Hoens AM, Holsti L, et al. 34. Taylor S, Todd PA. Understanding information technology usage: a test of Therapists' perceptions of social media and video game technologies in competing models. Inform Syst Res. 1995;6(2):144–76. upper limb rehabilitation. JMIR Serious Games. 2015;3(1):e2. 35. Glegg SM, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors 12. Levac DE, Galvin J. Facilitating clinical decision-making about the use of influencing therapists' adoption of virtual reality for brain injury virtual reality within paediatric motor rehabilitation: application of a rehabilitation. Cyberpsychol Behav Soc Netw. 2013;16(5):385–401. classification framework. Dev Neurorehabil. 2011;14:177–84. 36. Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW, editor. Usability evaluation in industry. Bristol: Taylor & Francis; 1996. p. 189–94. 13. Galvin J, Levac D. Facilitating clinical decision-making about the use of virtual reality within paediatric motor rehabilitation: describing and 37. Bangor A, Kortum PT, Miller JT. An empirical investigation of the system usability scale. Int J Human-Comp Interact. 2008;24(6):574–94. classifying virtual reality systems. Dev Neurorehabil. 2011;14:112–22. 38. US Department of Health and Human Services. System Usability Scale. 14. Brien M, Sveistrup H. An intensive virtual reality program improves balance Available at: http://www.usability.gov/how-to-and-tools/methods/system- and functional mobility of adolescents with cerebral palsy. Pediatr Phys usability-scale.html. Accessed 17 Sept 2015. Ther. 2011;23:258–66. 39. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. 15. Levac D, Fox E, Espy D, Pradhan S, Deutsch J. ‘Kinect-ing’ with clinicians: A Qual Health Res. 2005;15(9):1277–88. knowledge translation resource to support decision-making about virtual 40. Annema J, Verstraete M, Abeele V, Desmet S, Geerts D. Videogames in reality video game use in rehabilitation. Phys Ther. 2015;95:1–15. therapy: A therapist's persepective. Fun Games. 2010;94:8. 16. Levac D, Miller P, Missiuna C. Usual and virtual reality video game-based 41. Schleifer Taylor J, Verrier MC, Landry MD. What Do We Know about physiotherapy interventions for children and youth with acquired brain Knowledge Brokers in Paediatric Rehabilitation? A Systematic Search and injuries. Phys Occup Ther Pediatr. 2012;32:180–95. Narrative Summary. Physiother Can. 2014;66:143–52. 17. Thornton M, Marshall S, McComas J, Finestone H, McCormick A, Sveistrup H. 42. Russell DJ, Rivard LM, Walter SD, Rosenbaum PL, Roxborough L, Cameron D, Benefits of activity and virtual reality based balance exercise programmes et al. Using knowledge brokers to facilitate the uptake of pediatric for adults with traumatic brain injury: perceptions of participants and their measurement tools into clinical practice: a before-after intervention study. caregivers. Brain Inj. 2005;19:989–1000. Implement Sci. 2010;5:92. doi:10.1186/1748-5908-5-92. 18. Lewis GN, Rosie JA. Virtual reality games for movement rehabilitation in neurological conditions: how do we meet the needs of users? Disabil Rehabil. 2012;34:1880–6. 19. Glegg SMN, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors influencing therapists' adoption of virtual reality for brain injury rehabilitation. J Cyberther Rehabil. 2012;5:4. 20. Glegg SMN, et al. Evaluating change in virtual reality adoption for brain injury rehabilitation following knowledge translation. Disabil Rehabil: Assist Technol. In Press. 21. Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:70. doi:10.1186/1748-5908-7-70. 22. Jones CA, Roop SC, Pohar SL, Albrecht L, Scott SD. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. 23. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50. doi:10.1186/1748-5908-7-50. 24. Menon A, Korner-Bitensky N, Kastner M, McKibbon KA, Straus S. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. 25. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015;4:CD005470. 26. Levac D, Glegg S, Camden C, Rivard L. Best practice guidelines for the development, implementation, and evaluation of online knowledge translation resources in rehabilitation. Phys Ther. 2015;95:648–62. 27. Anita M. Benefits of an E-learning Intervention for Implementing Stroke Rehabilitation Best Practices. PhD Thesis. Canada: University of Toronto; 28. Menon A, Korner-Bitensky N, Chignell M, Straus S. Usability testing of two e- learning resources: methods to maximiz potential for clinican use. J Rehabil Med. 2012;44(4):338–45. 29. Imperial College London, World Health Organization. eLearning for undergraduate health professional education - a systematic review informing a radical transformation of health workforce development. Al- Shorbaji N, Atun R, Car J, Majeed A, Wheeler E, editors. 2015. p. 156. http:// Submit your next manuscript to BioMed Central whoeducationguidelines.org/content/elearning-report. 30. Glegg SMN. Virtual reality for brain injury rehabilitation: An evaluation of and we will help you at every step: clinical practice, therapists’ adoption and knowledge translation. MSc Thesis. • We accept pre-submission inquiries University of British Columbia. 2012. https://open.library.ubc.ca/cIRcle/ collections/ubctheses/24/items/1.0073087. � Our selector tool helps you to find the most relevant journal 31. Kramer TL, Pyne JM, Kimbrell TA, Savary PE, Smith JL, Jegley SM. Clinician � We provide round the clock customer support perceptions of virtual reality to assess and treat returning veterans. Psychiatr � Convenient online submission Serv. 2010;61:1153–6. 32. Markus LA, Willems KE, Maruna CC, Schmitz CL, Pellino TA, Wish JR, et al. � Thorough peer review Virtual reality: feasibility of implementation in a regional burn center. Burns. � Inclusion in PubMed and all major indexing services 2009;35:967–9. � Maximum visibility for your research 33. Glegg S, Tatla SK, Holsti L. The GestureTek virtual reality system for physical and cognitive rehabilitation: A scoping review. Disabil Rehabil Assist Submit your manuscript at Technol. 2014;9:89–111. www.biomedcentral.com/submit http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Health Services Research Springer Journals

A knowledge translation intervention to enhance clinical application of a virtual reality system in stroke rehabilitation

Download PDF
11 pages

Loading next page...
 
/lp/springer-journals/a-knowledge-translation-intervention-to-enhance-clinical-application-wuhnP10BvO

References (62)

Publisher
Springer Journals
Copyright
2016 The Author(s).
eISSN
1472-6963
DOI
10.1186/s12913-016-1807-6
Publisher site
See Article on Publisher Site

Abstract

Background: Despite increasing evidence for the effectiveness of virtual reality (VR)-based therapy in stroke rehabilitation, few knowledge translation (KT) resources exist to support clinical integration. KT interventions addressing known barriers and facilitators to VR use are required. When environmental barriers to VR integration are less amenable to change, KT interventions can target modifiable barriers related to therapist knowledge and skills. Methods: A multi-faceted KT intervention was designed and implemented to support physical and occupational therapists in two stroke rehabilitation units in acquiring proficiency with use of the Interactive Exercise Rehabilitation System (IREX; GestureTek). The KT intervention consisted of interactive e-learning modules, hands-on workshops and experiential practice. Evaluation included the Assessing Determinants of Prospective Take Up of Virtual Reality (ADOPT-VR) Instrument and self-report confidence ratings of knowledge and skills pre- and post-study. Usability of the IREX was measured with the System Usability Scale (SUS). A focus group gathered therapist experiences. Frequency of IREX use was recorded for 6 months post-study. Results: Eleven therapists delivered a total of 107 sessions of VR-based therapy to 34 clients with stroke. On the ADOPT-VR, significant pre-post improvements in therapist perceived behavioral control (p = 0.003), self-efficacy (p = 0.005) and facilitating conditions (p =0.019) related to VR use were observed. Therapist intention to use VR did not change. Knowledge and skills improved significantly following e-learning completion (p = 0.001) and was th sustained 6 months post-study. Below average perceived usability of the IREX (19 percentile) was reported. Lack of time was the most frequently reported barrier to VR use. A decrease in frequency of perceived barriers to VR use was not significant (p = 0.159). Two therapists used the IREX sparingly in the 6 months following the study. Therapists reported that client motivation to engage with VR facilitated IREX use in practice but that environmental and IREX-specific barriers limited use. Conclusions: Despite increased knowledge and skills in VR use, the KT intervention did not alter the number of perceived barriers to VR use, intention to use or actual use of VR. Poor perceived system usability had an impact on integration of this particular VR system into clinical practice. Keywords: Knowledge translation, Virtual reality, Rehabilitation, Stroke * Correspondence: [email protected] Department of Physical Therapy, Movement Sciences and Rehabilitation, Bouve College of Health Sciences, Northeastern University, 407c Robinson Hall, 360 Huntington Ave, Boston, MA 02115, USA Full list of author information is available at the end of the article © 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Levac et al. BMC Health Services Research (2016) 16:557 Page 2 of 11 Background strategies for integrating VR-based therapy in stroke re- Although recent reviews have synthesized the mounting habilitation using a more multi-faceted approach. evidence for the effectiveness of virtual reality (VR) inter- Systematic reviews of KT strategies in rehabilitation indi- ventions in stroke rehabilitation [1–4], little is known about cate that educational strategies are the most commonly the extent and nature of VR use by physical and occupa- used KT intervention [21–23] but that diversity of study tional therapists outside of a research context [5, 6]. VR is outcomes and interventions limits conclusions about defined as any computer hardware and software system which KT interventions are ideally suited to specific con- that generates simulations of real or imagined environ- texts [23]. However, active multi-component interventions ments with which participants interact using their own improve knowledge and practice behaviors of PTs to a movements [7, 8]. Key VR characteristics of immersion, greater extent than passive strategies [24] and interventions feedback, and interactivity within the virtual world [9] are addressing specific determinants of practice produce small expressed to varying degrees in both rehabilitation-specific to moderate effects [25]. Online KT resources, defined as systems and off-the-shelf video gaming consoles. A recent “e-learning products that translate evidence-based know- survey of Canadian physical therapists (PTs) and occupa- ledge to disseminate information that increases awareness, tional therapists (OTs) found that 46 % of respondents had informs clinical practice and/or stimulates practice change” clinical experience with VR, with the Nintendo Wii being [26] may positively influence self-reported knowledge and the most familiar and accessible system (Glegg SMN, Levac skills [27, 28], and are becoming a popular educational DE, Miller P, Colquhoun H, Wright V: A survey of physical method for health professionals [29]. Indeed, e-learning and occupational therapists’ virtual reality use and learning modules and videos were the preferred educational formats needs, unpublished). Seventy-six percent of respondents of respondents to the Canadian VR use and learning needs were interested in learning more about VR, with top learn- survey (Glegg SMN, Levac DE, Miller P, Colquhoun H, ing needs relating to equipment set up, game familiarity, Wright V: A survey of physical and occupational therapists’ and matching games to client goals (Glegg SMN, Levac virtual reality use and learning needs, unpublished). DE, Miller P, Colquhoun H, Wright V: A survey of physical The purpose of this study was to develop and to evalu- and occupational therapists’ virtual reality use and learning ate a KT intervention that incorporated an online mod- needs, unpublished). Knowledge translation (KT) resources ule and experiential practice to train PTs and OTs in VR may support clinicians motivated to use VR in developing implementation for stroke rehabilitation. The KT inter- the competencies required for evidence-based VR applica- vention was built on previously identified support needs tion [10]. As such, developing accessible, effective KT re- and modifiable barriers in therapist knowledge and skills sources is one way to promote the sustainable integration known to influence VR adoption [5, 20, 30–32]. The KT of VR systems within stroke rehabilitation settings. intervention included a component focused on incorpor- KT resources for PTs and OTs should support clinicians ating motor learning strategies into VR-based rehabilita- in designing, monitoring, adapting and evaluating VR- tion; these findings are reported elsewhere (Levac DE, et based treatment programs [10]. In addition to technical al: Promoting therapists’ use of motor learning strategies competence in VR system operation, clinicians must make during virtual reality-based stroke rehabilitation, in prep- decisions about which systems and games are most appro- aration). The objectives of this study that pertain to this priate for their clients’ stages of recovery, goals, and article were to: 1) evaluate the impact of the intervention physical and cognitive capabilities [10, 11]. Existing frame- on therapists’ confidence related to VR knowledge and works to support decision-making about VR system skills and perceptions of facilitators and barriers related selection [12, 13], task analyses of a growing list of games to VR use; 2) assess usability of the VR system; 3) obtain [14, 15] and qualitative reports of therapist and client per- therapists’ perspectives about the KT intervention and ceptions of VR use in rehabilitation [11, 16–18] are help- VR use in practice; and finally 4) measure the frequency ful, but best practice recommendations to guide VR of continued VR use following the KT intervention. implementation are needed. Known barriers to VR use include lack of time, knowledge, skills and resources; tech- Methods nical issues; and client factors, while practical evidence- The KT intervention took place at 2 sites and was evalu- informed KT resources are proven facilitators [19, 20]. A ated using a pre-post design. Study procedures are also previous KT study supporting VR integration in two in- reported in (Levac DE, et al: Promoting therapists’ use patient acquired brain injury rehabilitation settings tar- of motor learning strategies during virtual reality-based geted modifiable barriers and facilitators using a clinical stroke rehabilitation, in preparation). resource manual, mentoring, technical support and train- ing sessions. Changes in self-reported knowledge and VR system skills was reported by participating therapists [20]. The Motion-capture technology enables players to view present KT study builds on these results, focusing on theirmirrorimage in the virtual environment of the Levac et al. BMC Health Services Research (2016) 16:557 Page 3 of 11 GestureTek Interactive Rehabilitation Exercise (IREX) aspects of IREX usage (i.e., set-up, etc.). Therapists com- software platform (www.gesturetekhealth.com, Ges- pleted the final outcome measures approximately 1 month tureTek, Toronto, ON, Canada). Interaction with the after their final study VR-based therapy session. virtual environment is through body movements to The same process was implemented at site 2. However participate with games that address multiple upper here therapists had the additional opportunity to seek extremity or full body movement goals, while motiv- guidance from a site 1 participant who would act as a ating clients to participate [33]. clinician-mentor, via email, Skype or phone. KT inter- ventions at each site differed only in this opportunity for Procedures mentorship, which was included to explore whether it PTs and OTs were recruited from two rehabilitation centers would impact VR use at either site. At the end of the with inpatient and outpatient stroke rehabilitation units in study, a focus group was conducted at each site to cap- two distinct urban centers in Ontario, Canada. Each site had ture therapists’ experiences with VR use. Following the acquired the IREX for clinical and/or research purposes, completion of the KT intervention, participants were however therapists had not yet been trained on its use. The invited to take part in the sustainability phase, which in- KT intervention was delivered at each site over 5 months, volved monitoring their use of the VR system over six with a staggered start time 8 months later for site 2. months; the research team avoided follow-up communi- During the first month, site 1 therapists completed the cation with the therapists during this time (Fig. 1). self-paced e-learning modules outside of work hours, and also participated in face to face hands-on learning, audit Intervention and feedback sessions. Over the next 4 months, each ther- The Template for Intervention Description and Replica- apist identified up to 4 patients with the following criteria: tion (TIDieR) Checklist for reporting interventions was 1) experienced a stroke within the last 12 months, 2) re- used to guide our intervention description below. A simi- ceiving inpatient or outpatient PT and/or OT services lar description can be found in (Levac DE, et al: Promot- targeting motor skills, and 3) sufficient cognitive and ing therapists’ use of motor learning strategies during physical skills to engage in VR activities (as determined by virtual reality-based stroke rehabilitation, in preparation). treating therapist). Therapists completed up to four ses- sions of VR-based therapy per patient. Therapists were 1. E-learning modules: Three online modules provided free to choose the length of time they would use the IREX foundational knowledge about clinical VR use. The with each client, and were responsible for all technical first module contained information about IREX Fig. 1 Study procedures, including timing of interventions and outcome measurement Levac et al. BMC Health Services Research (2016) 16:557 Page 4 of 11 operation and game characteristics, while the last two learning sessions. Feedback was individualized to each modules focused on the application of motor learning clinicians’ performance and learning needs. strategies to VR-based therapy. Outcomes related to 3. Experiential learning: Clinicians used the IREX the integration of motor learning strategies are system with up to 4 clients. Feedback from the study reported in (Levac DE, et al: Promoting therapists’ use investigators about IREX use was available on of motor learning strategies during virtual reality- request via phone, email or in-person support during based stroke rehabilitation, in preparation). Two co- this period. investigators, with feedback from other authors, de- 4. Didactic reminders: Therapists received weekly veloped the modules by integrating their clinical and e-mails from the PI identifying ‘tips’ for VR use from research experience using VR. Embedded video clips the e-learning modules during the experiential phase. were filmed with consenting client and staff volun- 5. Mentorship: Interested clinicians at site 2 were teers. Interactive learning activities and knowledge matched with 1 of 3 clinician mentors from site 1 checks required learners to integrate and to demon- and received biweekly encouragement to contact strate their knowledge and skills. Based on pilot test- mentors with questions or support needs. Mentors ing of module usability with therapists at [removed also reached out periodically to mentees via email. for review], changes were made to the wording of the knowledge checks and material was removed to make Outcomes the content more manageable. The first module in- Knowledge and skills cluded video clips illustrating IREX game play as illus- Confidence ratings: Using a seven-point Likert scale, trated in Fig. 2.Additional file 1 lists the module therapists rated their confidence in the knowledge and learning objectives. Therapists completed each e- skills associated with each learning objective before and learning module in approximately two hours. Thera- after completing the e-learning modules as well as at the pists were also provided with a print manual contain- end of the study. The time between pre- and post- ing the information presented in the online modules. module confidence ratings was not recorded and 2. Hands-on learning: Hands-on experience using the depended on how long therapists spent completing the IREX was provided during 1 h-long individual module; this ranged from 2 h to 3 days. session and two group sessions led by the PI and a clinician (MB). These sessions provided additional Confidence, barriers and facilitators education and discussion opportunities about VR ADOPT-VR: Therapists completed the Assessing Determi- system operation and trouble-shooting through the nants of Prospective Uptake of Virtual Reality (ADOPT- use of case scenarios. Case scenarios illustrated VR) Instrument pre- and post-study. Based on an extended ‘typical’ stroke rehabilitation clients and included Theory of Planned Behavior [34]), the ADOPT-VR exam- discussion questions related to game selection, ines 11 constructs thought to influence VR adoption, in- progression of difficulty, and tailoring the task to cluding attitudes, self-efficacy, and intention to use VR. The individual client needs. Feedback about VR skills was ADOPT-VR has 24 items scored on a nine-point Likert provided to participants by the PI during the hands-on scale (1: Strongly disagree – 9: Strongly agree) as well as Fig. 2 Two screenshots from the Interactive e-learning module Levac et al. BMC Health Services Research (2016) 16:557 Page 5 of 11 nine multiple-response and short-answer questions. Sum- changes in confidence ratings at a 0.05 alpha level. De- mary scores in 11 categories are reported. Established face scriptive data were summarized using frequency counts, and content validity [30], good internal consistency (alpha and means or medians as appropriate. Focus group re- = 0.876) [35] and responsiveness to change [20] have been sponses were categorized using content analysis [39] by demonstrated. two investigators. System usability Results System Usability Scale: The System Usability Scale (SUS) Participant demographics [36], which has demonstrated reliability, sensitivity and Six PTs and five OTs participated in the study, enrolling concurrent validity [37], was used to evaluate perspectives a total of 34 clients. Therapists at site 1 had a mean of IREX usability at the end of the study. The SUS is a 10 19.3 years (SD 8.1) work experience, and therapists at item questionnaire scored on a five-point Likert scale; site 2 had a mean of 11.4 (SD 9.4) years work experi- scores are converted to percentiles. A score of 68 and ence. Client mean age was 62.8 (SD 16.4) years at site 1 above is considered above average [38]. Using a traditional and 60.1 (SD 15.0) years at site 2. All therapists were school grading scale (from A – F) is recommended to novice IREX users. Pre-study, none reported familiarity complement the score and convey usability [34]. with any of the IREX games. Following the study, famil- iarity range was between 4–10 (mean 6.25) games. Sustainability Therapists who agreed to continue in the sustainability Intervention fidelity phase of the study recorded the frequency of their IREX The target client enrolment for each therapist was 4 clients. use in the six months post-study completion and pro- At site 1, three therapists enrolled four clients, one therap- vided written responses to two questions about why they ist enrolled three clients, and two therapists enrolled two had or had not used the IREX. clients each. At site 2, two therapists enrolled four clients, one therapist enrolled three clients, and two therapists en- Focus groups rolled two clients each. No uptake of the mentoring offered The PI conducted a 1.5-h focus group at each site at the by site 1 clinicians to site 2 clinicians occurred, as measured end of the study to explore therapist perspectives on by participant self-report of phone, email or Skype contact. participation in the KT intervention, use of the IREX in As such, the KT interventions were identical and results for practice, and experience of integrating motor learning both sites are presented together. strategies (reported in [32]]. The focus group was audio- recorded and transcribed for analysis. Knowledge and skills (confidence ratings) Figure 3 illustrates changes in confidence ratings for Analyses each learning objective for the e-learning module. A sig- Non-parametric Wilcoxon signed rank tests evaluated nificant change from pre-module to post-module com- changes between pre- and post-study category scores pletion was observed (Z = −3.180, p =0.001, median and frequency counts for the ADOPT-VR as well as improvement of 3.18 (range 1.46) and this gain was Fig. 3 E-learning module objective confidence ratings. Median pre-module, post-module and post-study confidence ratings on each of the e-learning module objectives. Therapists were asked to rate their confidence, on a scale of 1 to 7, in the knowledge and skills associated with each learning objective. Additional file 1 provides the full list of learning objectives Levac et al. BMC Health Services Research (2016) 16:557 Page 6 of 11 Table 1 ADOPT-VR pre-post study changes and descriptive data fun and motivating method of therapy for clients; at times it seems that clients don't even realize it is ther- ADOPT-VR Category p-value Pre median Post median (range) (range) apy”), its potential to add variety to treatment sessions, Attitude 0.1 6.67 (4.33–7.67) 6.67 (4–9) and its ability to target frequent task repetitions. The two therapists who did not plan on continuing to use Perceived Usefulness 0.167 6.17 (4.00–8.00) 6.67 (3.67–8.33) the IREX cited lack of time and caseload factors as con- Ease of Use 0.677 5.17 (2.67–7.67) 5.17 (1.67–8) tributing reasons. Compatibility 0.363 4.75 (2–7) 5 (1.5–8) Figure 4 illustrates changes in pre- and post-study fre- Social Norms 0.308 5 (1–9) 4.5 (1–6) quency counts of reported barriers to IREX use. A de- Peer Influence 0.194 4.75 (1–7.5) 5 (3–7.5) crease in the frequency of reported barriers to IREX use Superior Influence 0.403 3 (1–6.5) 3 1.5–7.5) from pre to post study was observed (decrease from 50 to 37 barriers; Z = −1.409, p = 0.159). A decrease in the Perceived Behavioural 0.003* 3 (1–6.5) 6.24 (4.5–8) Control frequency of barriers specifically targeted through this Self-Efficacy 0.005* 2.5 (1–8) 7 (6–9) intervention (time to learn about the IREX, access to evidence and educational opportunities; decrease from Facilitating Conditions 0.019* 2.75 (1–6) 4.25 (1–7) 22 to 9; Z = 1.1.633, p = 0.102) were also observed. In Behavioural Intentions 0.21 5.5 (1.67–7.67) 5.33 (1.33–7) short answer responses that accompany the ratings, *Significant at p < =0.05 therapists described their most significant barriers to maintained at the end of the study (median decrease of IREX use as time, short client length of stay, caseload is- 0.12 points (range 1.24) from post-module completion). sues, technical problems with the IREX, and inability to use the system to address OT-related fine-motor goals. Knowledge and skills: ADOPT-VR variable outcomes For example, one therapist wrote: “I would use it much Significant pre-post improvement in therapists’ per- more if games included grasp-release and object manipu- ceived behavioral control (Z = −2.945, p = 0.003), self- lation.” Lack of time to use the IREX in a treatment ses- efficacy (Z = −2.802, p = 0.005) and facilitating conditions sion was the most frequently reported barrier to its use and barriers (Z = −2.352, p =0.019) were observed. No both pre- and post-study. For example, a therapist wrote: other constructs demonstrated significant changes (see “The timeslot necessary to arrange a decent session is Table 1). large and the opportunity to set aside that amount of time, plus travel and not have overlap with other pa- Barriers and facilitators (ADOPT-VR) tients arriving for treatment is a huge barrier.” Following Nine therapists indicated in their post-study ADOPT-VR the study, therapists described two new barriers on the that they intended to continue to use the IREX because ADOPT-VR: inappropriate clients and inability to use of its motivating appeal for clients (e.g. “I think it is a the IREX to target fine motor goals. Nine therapists Fig. 4 Pre-post study changes in the number of barriers to VR adoption. Frequency count of barriers reported by therapists pre- and post-study Levac et al. BMC Health Services Research (2016) 16:557 Page 7 of 11 reported that adjusting game parameters and resolving Focus group technical issues were areas in which they lacked confi- The following three themes were identified regarding dence following the study; two therapists did not report the KT intervention and use of the IREX in practice. any areas of low self-efficacy. On the post-study ADOPT-VR, seven of the 11 thera- Benefits and challenges of the KT intervention pists reported greater familiarity with the IREX because Therapists appreciated the combination of self-paced of the KT intervention and one therapist reported as a online learning with group and individual practice ses- facilitator to IREX use at post-study, having access to pa- sions. The different formats presented in the e-learning tients who were motivated to use VR; three therapists modules (e.g. written information, videos and online ac- did not describe any facilitators. One therapist commen- tivities) were also appreciated, although technical issues ted: “It would be a nice addition to enhance and add (e.g. video speed and quality) were an issue for some. variety to practice for subsets of the patient population. I Given that the e-learning component took place outside could identify a few key individuals who would find the of working hours, therapists reinforced that they would program motivational and uplifting.” No other facilita- not have undertaken this commitment without the reim- tors were mentioned pre- or post-study. bursement provided by the study. Therapists agreed that the opportunity to trial the system/games in person had been helpful; for example, one therapist stated: “I think System usability it’s a necessity, really, to play around with it yourself, Low perceived usability of the IREX was evidenced by a and put yourself in there, and feel the demands and the th SUS mean score of 54.25 (19 percentile, below aver- challenges yourself, will help you decide who to use it age). Figure 5 shows the proportion of respondents who with and how to use it more efficiently for sure.” rated the system by letter grade (from A- F). However, the fact that their first independent use of the IREX was within the context of a regular treatment session was a challenge for some therapists. This influ- Sustainability enced their choice of clients because of the need to bal- Nine therapists indicated their intention to continue to ance client safety and expectation of standard treatment use the IREX on the ADOPT-VR, and five therapists with attending to their own (i.e. therapists’) learning. For (three from site 1 and two from site 2) agreed to partici- some therapists, choice of the first client was premised pate in the sustainability phase of the study. Of the par- on physical factors, such as independent standing bal- ticipating therapists, one used the system at site 1 on ance, while others sought patients whom they identified two occasions and one used the system at site 2 on a as being ‘tech-savvy’. For example, one therapist stated: single occasion. In written responses following the sus- “I personally chose a client that I felt was going to be tainability phase, these therapists indicated that lack of comfortable with the idea of that type of technology. time, location of the VR room, technical difficulties with Someone that was really keen on computers…That they the IREX, and lack of appropriate patients were factors would understand that, there might be some glitches and limiting their use during the post-study period. that ….they were giving me [emphasis] an opportunity. They were helping me. That’s what I told them. They were helping me to learn the system, and at the same time, some things that might be helpful for them, too.” In- deed, therapists expressed that a refresher session during the experiential learning phase would be a useful com- ponent of the KT intervention, particularly when more than a few weeks had passed between the initial learning and the therapist’s first use of the IREX with a client. Therapists reported using only the games that they had learned during the KT intervention, implementing fre- quent repetitions of these games. Client enjoyment was a facilitator of IREX use in practice Client enjoyment was perceived as a facilitator of IREX use by therapists. For example, one therapist stated: Fig. 5 System Usability Scale ratings. An A grade corresponds to a “[VR] gives a different kind of feedback than what you percentile score above 80. A score of 68 is a C grade and scores can do just in the therapy room. It’s a different context. below 51 correspond to a grade of F And the fact that it’s a game… one of my patients told Levac et al. BMC Health Services Research (2016) 16:557 Page 8 of 11 me that he didn’t feel like a patient when he was doing therapy gym, so independent sessions did not fit the stand- it, and he loved that…it was fun, and you know, it made ard treatment model. With travel time taken into consider- him feel very different from being in a hospital patient, ation, less time was available in a session to use the IREX. so… there was great value in that.” Another noted: “They For example, a therapist stated: “….it would be the exception see it more as a fun and more exciting… I may do the that I would do VR. I see some benefit for some patients, but same movement, but it’s more interactive than, let’s say if it’satime–cost ratio. It’s the amount of extra time and work, I’m doing the same repetitive movement without that andwhatittakes me away from to do VR, versusactually video. You know, without that interaction with the doing something else in the amount of time….” Finally, the game….. if you choose the right patients, it gets them ex- short client lengths of stay at these institutions limited the cited about possibly the choices or the games that you’re frequency with which the IREX could be integrated into doing.” Therapists were persistent in encouraging clients treatment, since other treatment goals (i.e. discharge readi- who were initially reluctant to try VR-based therapy as ness) were a priority. they found interest increased with exposure. One ther- apist noted “… those people that didn’t necessarily buy in Discussion right away, by the end were just like, great, engaged, and This study targeted therapist knowledge and skills with motivated, and, you know, really partaking and… right the goal of increasing clinical uptake of an evidence- in there.” based VR system that was already in situ at two clinical sites. Significant increases in self-reports of knowledge Barriers to IREX use in practice and skill were observed and, importantly, sustained over Participants described system and environmental bar- time; however, this change did not translate to increased riers to IREX use in practice. While they were being ini- VR use, although intentions of therapists to continue to tially excited to learn a new technology, hardware and use VR were high. The low sample size during the sus- software malfunctions that interrupted game play, and tainability phase limits the evaluation of long-term be- poor lighting which interfered with IREX operation were havior change in the presence of high intention to use prevalent. Therapists were frustrated when the games VR. The study findings strengthen existing knowledge did not work. Each site had a contact person who could about the facilitators and barriers to VR use with other trouble shoot minor issues related to lighting, but un- inpatient populations (21) and add to the evidence base foreseen software issues with the system itself required by exploring in greater depth the experience of thera- maintenance by company representatives, necessitating pists learning to use VR in practice. replacement of equipment and delaying the study at The KT intervention format and content were tailored each site by several weeks. towards learning needs identified by inpatient clinicians Therapists reported that the IREX games were not al- in a previous study [19]. For example, Glegg et al. [20] ways an ideal fit with therapy goals: “Like, if more games found that a print educational manual was not well [were available], or just modify it so there’s more for the accessed by clinicians working in ABI rehabilitation and middle [functioning] person… because I had patients, suggested e-learning modules, reimbursed training time, where I wasn’t comfortable standing them. They were a lit- and mentoring strategies, all of which were part of this tle bit past sitting and just trying to move, or if they were study. The e-learning module and the workshops em- sitting, like I could work a little bit on their trunk, but there phasized information specific to previously identified was nothing in between.” Another said: “…it wasn’tmatch- therapist learning needs [20]. Confidence ratings imme- ing with his goals, and there weren’t really games that were diately post-module completion indicated the online appropriate for his level of activity in his upper extremity.” learning was effective at improving self-reported know- Therapists reported that they would be inclined to use a ledge and skills in VR use, with improvements sustained different VR system that could be set up in the standard months later. Objective assessments of knowledge and therapy gym and allow for greater independent patient skill change would be valuable adjuncts to future studies use. One therapist said: “I think there’s a value to some of to validate this self-report data. the virtual reality type games, like in the Wii or like the The therapists in this study were novice VR users and Kinect or whatever. If there was a way to translate that they entered the study with positive attitudes towards into clients, perhaps in their home setting with family, I VR, high perceived usefulness ratings of VR, and strong think that could be quite interesting for clients…” intention to use the IREX; these ratings remained high From an environmental perspective, the time required for at post-test. Because intention to use VR was strong at system set up and to transport a client to the room housing the study outset, no changes in this construct were ex- the IREX, which was at some distance from the standard pected at post-test. Self-efficacy and perceived behavioral treatment area, was an issue. Therapists in these two set- control improved significantly post-study, although 82 % tings typically treat multiple clients at once in the large of participants still felt that adjusting game parameters Levac et al. BMC Health Services Research (2016) 16:557 Page 9 of 11 and resolving technical issues were areas in which they research that explores the optimal timing and delivery of lacked confidence. These findings are consistent with mentoring for VR adoption, as well as therapists’ per- previous research [17]. Compatibility, social norms, peer spectives about the desirability, benefits and challenges and superior influence constructs of the ADOPT-VR of various mentoring models would build on the know- demonstrated no change; these findings are not surpris- ledge base in this area. ing, given that these variables were not targeted by the Although the majority of participants indicated they KT intervention, and were not anticipated to change. Of would continue to use the IREX and found VR useful, note is the absence of change on the perceived ease of the five therapists who participated in the sustainability use construct, which is in contrast with previous work phase did not frequently use the system. While the small [20]. A reasonable explanation is the degree of technical sample size and timeframe for follow-up limits difficulty experienced during the current study, as af- generalizability, the reported reasons behind the limited firmed by the SUS findings; these difficulties were expe- VR use are consistent with the barriers identified in this rienced to a greater extent than in previous research and previous research [16, 17]. However, despite this ob- [17]. Technical trouble-shooting will likely always be re- served behavior, a statistically significant increase in fa- quired for VR implementation; having a support person cilitating conditions to VR use was found among the available to manage this aspect is important because larger group of therapists at post-study. Factors such as these technical skills are outside of most therapists’ skill perceived benefits of VR for clients, the novelty of the sets, and the need to trouble-shoot draws time away intervention, or the support provided through the KT from both practice opportunities, and client treatment. intervention may be responsible for this change, al- Indeed, ongoing accessible technical support has been though we did not see an impact on actual use of the raised as an important element of VR adoption [17], and VR system. More research is required to determine the the low SUS scores suggest that additional support may relative influence of specific barriers and facilitators on have been highly valued by these therapists. therapists’ behavioral intention and actual behavior re- Therapists spoke positively of the motivational appeal lated to VR use. To provide greater insight into the sus- of the IREX for clients, which is in accordance with pre- tainability of VR use in the clinical context, and the vious work describing positive client and therapist per- changing nature or importance of the factors influencing ceptions of VR use in practice [5, 11, 17, 40]. However, VR use over time, large-scale follow-up studies are after gaining experience with the IREX, therapists identi- needed. fied new organizational and client-specific barriers to its The poor recruitment of clients by therapists during use, including a lack of match to treatment goals. Bar- the study was likely related to short client length of stay, riers identified through the ADOPT-VR and in the focus some clients declining to participate in the study, and groups are consistent with previous work, although the fact that the IREX was not functioning for several treatment delivery method as a barrier (i.e. treating mul- weeks during the study period at each site. Although this tiple patients at a time) is new to these settings [17]. problem was accommodated by lengthening the trial, This may be related to the client population, with the the decreased ‘dosage’ for therapists, in terms of reduced therapists from the previous study working in pediatric exposure to IREX, likely had an impact on the extent to and adult ABI rehabilitation likely being more accus- which therapists were able to integrate clinical know- tomed to 1:1 interventions. ledge gained during the KT intervention. Such factors The use of an onsite knowledge broker or clinician ‘ex- may make it more challenging to sustain momentum pert user’ is a KT strategy with emerging evidence of ef- and behavior change. fectiveness [41, 42]. At site 2, the on-site RA was an Therapists at these sites now have a base level of tech- expert user with whom clinicians could problem-solve nical and theoretical knowledge about VR use in practice about technical issues. This on-site clinical and technical that they can potentially transfer to other VR systems, support, found to be a facilitator of VR use by Glegg et though specific learning related to games, equipment al. [20], was not available at site 1. Glegg et al. [20] ob- set-up and direct clinical application will be required. served a spontaneous increase in participating therapists Barriers unique to this VR system and settings should mentoring fellow on-site clinicians in VR use. While in- not be generalized to reflect poorly on the potential for formal on-site mentoring was not monitored during this VR implementation as a whole. With respect to the on- study, the use of cross-site mentoring did not occur des- going use of the IREX at these clinical sites, investigators pite explicit attempts at facilitation. It may be that our are working with company representatives to resolve the study did not sufficiently facilitate mentor-mentee trust, technical issues that arose. Study findings are being dis- accessibility, and perceived expertise. Additional efforts cussed with clinical management to strategize around to support the establishment of credibility of mentors by how to address the identified barriers to the use of the mentees may have led to greater success. Future technology. Next steps will involve engaging the Levac et al. BMC Health Services Research (2016) 16:557 Page 10 of 11 therapists in a needs assessment to assist them in Availability of data and materials The datasets during and/or analysed during the current study available from matching the characteristics of other available VR sys- the corresponding author on reasonable request. tems with their specific client and practice setting needs to ensure greater success in future VR implementation Authors’ contributions DL, HS and SG conceived of the study. DL, HS, HC, HF, SG, VDP, and JH efforts. Participated in the design and monitoring of the study. DL and SG created the study materials. DL and SG performed the statistical analysis. DL and PM performed the qualitative analysis. DL Drafted the manuscript. DL, SG, HC, Conclusions HS, PM, JH, VDP, HF, DV critically revised the manuscript for important This study confirmed previous research demonstrating intellectual content. All authors read and approved the final manuscript. therapists’ positive attitudes toward VR and its perceived Competing interests usefulness as a clinical tool for neurorehabilitation. The The authors declare they have no competing interests. KT intervention designed to translate knowledge about use of the VR system to therapists in two stroke rehabili- Consent for publication Verbal informed consent was obtained from the person depicted in the tation units was well-received, and successfully incorpo- Fig. 2 to publish individual, identifiable data. rated e-learning, experiential learning and reminders to significantly increase self-reported confidence, know- Ethical approval and consent to participate University and hospital review boards provided ethical approval for the study ledge and skills in VR use. While these gains were sus- and informed consent was obtained from all participants. tained at follow-up, further research is required to examine sustainability of VR use over time in the clinical Author details Department of Physical Therapy, Movement Sciences and Rehabilitation, setting, as well as the feasibility and potential benefits of Bouve College of Health Sciences, Northeastern University, 407c Robinson more structured mentoring models to support VR use. 2 Hall, 360 Huntington Ave, Boston, MA 02115, USA. Therapy Department, Qualitative findings suggest that system and context- Sunny Hill Health Centre for Children, 3644 Slocan Street, Vancouver, BC V5M 3E8, Canada. School of Rehabilitation Sciences, Faculty of Health Sciences, specific barriers merit ongoing attention in order for University of Ottawa, 200 Lees (A121), Ottawa, ON K1S 5S9, Canada. KT-based interventions that support adoption to be ef- 4 Department of Occupational Science and Occupational Therapy, University fective. Ongoing training of increasing complexity, of Toronto, 160-500 University Ave Toronto, Ontario M5G 1V7, Canada. School of Rehabilitation Sciences, Faculty of Health Sciences, McMaster paired with interspersed practice in clinical application University, Institute of Applied Health Sciences, Room 403, 1400 Main St. of the technology, as well as ongoing assessment of ther- 6 West, Hamilton, ON L8S 1C7, Canada. Bruyere Research Institute, Élisabeth apist support needs are recommended to meet the chan- Bruyère Hospital, Bruyère Continuing Care, 43 Bruyère Street, Ottawa, ON K1N 5C8, Canada. School of Rehabilitation Therapy, Queen’s University, ging needs of therapists over time. Louise D. Acton Building, 31 George Street, Kingston, ON K7L 3 N6, Canada. School of Rehabilitation Science, McMaster University, IAHS Building Room 403, 1400 Main Street West, Hamilton, ON L8S 1C7, Canada. Hamilton Additional file Health Sciences, Regional Rehabilitation Centre, 300 Wellington St. North, Hamilton, ON, Canada. Department of Psychiatry and Behavioural Additional file 1: Description of data: E-learning module learning objec- Neurosciences, DeGroote School of Medicine, McMaster Univerity, 12 Main tives Module 1: “Introduction to the GestureTek VR system. (DOCX 14 kb) Street West, Hamilton, ON l8S 1C7, Canada. Received: 3 December 2015 Accepted: 30 September 2016 Acknowledgements The authors would like to thank the therapists and clients who participated References in this study as well as the research assistants Jennifer Chau, Liliane 1. Laver K, George S, Thomas S, Deutsch JE, Crotty M. Cochrane review: virtual Letourneau and Denise Sartor. Thank you to Marie Brien for participating in reality for stroke rehabilitation. Eur J Phys Rehabil Med. 2012;48:523–30. the clinician workshops at Site 1 and to Laurie Wishart for grantsmanship 2. Laver KE, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke advice. We appreciate the input and support of clinical managers on the rehabilitation. Cochrane Database Syst Rev. 2015; doi:10.1002/14651858 stroke rehabilitation units at Elizabeth Bruyere Continuing Care and the 3. Darekar A, McFadyen BJ, Lamontagne A, Fung J. Efficacy of virtual reality- Regional Rehabilitation Centre, Hamilton Health Sciences. Andrew Dunn and based intervention on balance and mobility disorders post-stroke: a scoping Craig Ross at BC Children’s Hospital contributed to e-learning module devel- review. J Neuroeng Rehabil. 2015;12:46. opment. Abstracts of preliminary data have been presented at the Canadian 4. Weiss PL, Rand D, Katz N, Kizony R. Video capture virtual reality as a flexible Stroke Congress in 2013 and 2014, the International Conference for Virtual and effective rehabilitation tool. J NeuroEng Rehabil. 2004;1:12. Rehabilitation in 2013, and the American Congress of Rehabilitation Medicine 5. Levac D, Miller P. Integrating virtual reality video games into practice: in 2014. A short paper entitled Integrating motor learning and virtual reality th Clinicians’ experiences. Physiother Theor Pract. 2013;29:504–12. into practice: A knowledge translation challenge was presented at the 10 6. Glegg SM, Holsti L, Stanton S, Hanna S, Velikonja D, Ansley B, et al. Using International Conference on Disability, Virtual Reality and Associated Tech- virtual reality in clinical practice: A multi-site exploratory study. nologies in 2014 and published in the conference proceedings. NeuroRehabil. 2014;35:563–77. 7. Wilson PN, Foreman N, Stanton D. Virtual reality, disability and rehabilitation. Funding Disabil Rehabil. 1997;19:213–20. This project has been generously funded, by a grant from the Ontario 8. Sveistrup H. Motor rehabilitation using virtual reality. J NeuroEng Rehabil. 2004;1:10. Ministry of Health and Long-Term Care, administered and supported by the 9. Brey P. Virtual Reality and Computer Simulation. In: Einer K, Himma K, Tavani Ontario Stroke Network. The views expressed do not necessarily reflect those HT, editors. The handbook of information and computer ethics. Hoboken: of the Ministry or the Ontario Stroke Network. Dr. Levac’s postdoctoral fel- John Wiley & Sons, Inc; 2008. p. 361–84. lowship was supported by the Canadian Institutes of Health Research, the 10. Levac DE, Galvin J. When is virtual reality 'therapy'? Arch Phys Med Rehabil. Canadian Child Health Clinician Scientist Program and NeuroDevNet. 2013;94:795–8. Levac et al. BMC Health Services Research (2016) 16:557 Page 11 of 11 11. Tatla SK, Shirzad N, Lohse KR, Virji-Babul N, Hoens AM, Holsti L, et al. 34. Taylor S, Todd PA. Understanding information technology usage: a test of Therapists' perceptions of social media and video game technologies in competing models. Inform Syst Res. 1995;6(2):144–76. upper limb rehabilitation. JMIR Serious Games. 2015;3(1):e2. 35. Glegg SM, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors 12. Levac DE, Galvin J. Facilitating clinical decision-making about the use of influencing therapists' adoption of virtual reality for brain injury virtual reality within paediatric motor rehabilitation: application of a rehabilitation. Cyberpsychol Behav Soc Netw. 2013;16(5):385–401. classification framework. Dev Neurorehabil. 2011;14:177–84. 36. Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW, editor. Usability evaluation in industry. Bristol: Taylor & Francis; 1996. p. 189–94. 13. Galvin J, Levac D. Facilitating clinical decision-making about the use of virtual reality within paediatric motor rehabilitation: describing and 37. Bangor A, Kortum PT, Miller JT. An empirical investigation of the system usability scale. Int J Human-Comp Interact. 2008;24(6):574–94. classifying virtual reality systems. Dev Neurorehabil. 2011;14:112–22. 38. US Department of Health and Human Services. System Usability Scale. 14. Brien M, Sveistrup H. An intensive virtual reality program improves balance Available at: http://www.usability.gov/how-to-and-tools/methods/system- and functional mobility of adolescents with cerebral palsy. Pediatr Phys usability-scale.html. Accessed 17 Sept 2015. Ther. 2011;23:258–66. 39. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. 15. Levac D, Fox E, Espy D, Pradhan S, Deutsch J. ‘Kinect-ing’ with clinicians: A Qual Health Res. 2005;15(9):1277–88. knowledge translation resource to support decision-making about virtual 40. Annema J, Verstraete M, Abeele V, Desmet S, Geerts D. Videogames in reality video game use in rehabilitation. Phys Ther. 2015;95:1–15. therapy: A therapist's persepective. Fun Games. 2010;94:8. 16. Levac D, Miller P, Missiuna C. Usual and virtual reality video game-based 41. Schleifer Taylor J, Verrier MC, Landry MD. What Do We Know about physiotherapy interventions for children and youth with acquired brain Knowledge Brokers in Paediatric Rehabilitation? A Systematic Search and injuries. Phys Occup Ther Pediatr. 2012;32:180–95. Narrative Summary. Physiother Can. 2014;66:143–52. 17. Thornton M, Marshall S, McComas J, Finestone H, McCormick A, Sveistrup H. 42. Russell DJ, Rivard LM, Walter SD, Rosenbaum PL, Roxborough L, Cameron D, Benefits of activity and virtual reality based balance exercise programmes et al. Using knowledge brokers to facilitate the uptake of pediatric for adults with traumatic brain injury: perceptions of participants and their measurement tools into clinical practice: a before-after intervention study. caregivers. Brain Inj. 2005;19:989–1000. Implement Sci. 2010;5:92. doi:10.1186/1748-5908-5-92. 18. Lewis GN, Rosie JA. Virtual reality games for movement rehabilitation in neurological conditions: how do we meet the needs of users? Disabil Rehabil. 2012;34:1880–6. 19. Glegg SMN, Holsti L, Velikonja D, Ansley B, Brum C, Sartor D. Factors influencing therapists' adoption of virtual reality for brain injury rehabilitation. J Cyberther Rehabil. 2012;5:4. 20. Glegg SMN, et al. Evaluating change in virtual reality adoption for brain injury rehabilitation following knowledge translation. Disabil Rehabil: Assist Technol. In Press. 21. Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:70. doi:10.1186/1748-5908-7-70. 22. Jones CA, Roop SC, Pohar SL, Albrecht L, Scott SD. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. 23. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50. doi:10.1186/1748-5908-7-50. 24. Menon A, Korner-Bitensky N, Kastner M, McKibbon KA, Straus S. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. 25. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev. 2015;4:CD005470. 26. Levac D, Glegg S, Camden C, Rivard L. Best practice guidelines for the development, implementation, and evaluation of online knowledge translation resources in rehabilitation. Phys Ther. 2015;95:648–62. 27. Anita M. Benefits of an E-learning Intervention for Implementing Stroke Rehabilitation Best Practices. PhD Thesis. Canada: University of Toronto; 28. Menon A, Korner-Bitensky N, Chignell M, Straus S. Usability testing of two e- learning resources: methods to maximiz potential for clinican use. J Rehabil Med. 2012;44(4):338–45. 29. Imperial College London, World Health Organization. eLearning for undergraduate health professional education - a systematic review informing a radical transformation of health workforce development. Al- Shorbaji N, Atun R, Car J, Majeed A, Wheeler E, editors. 2015. p. 156. http:// Submit your next manuscript to BioMed Central whoeducationguidelines.org/content/elearning-report. 30. Glegg SMN. Virtual reality for brain injury rehabilitation: An evaluation of and we will help you at every step: clinical practice, therapists’ adoption and knowledge translation. MSc Thesis. • We accept pre-submission inquiries University of British Columbia. 2012. https://open.library.ubc.ca/cIRcle/ collections/ubctheses/24/items/1.0073087. � Our selector tool helps you to find the most relevant journal 31. Kramer TL, Pyne JM, Kimbrell TA, Savary PE, Smith JL, Jegley SM. Clinician � We provide round the clock customer support perceptions of virtual reality to assess and treat returning veterans. Psychiatr � Convenient online submission Serv. 2010;61:1153–6. 32. Markus LA, Willems KE, Maruna CC, Schmitz CL, Pellino TA, Wish JR, et al. � Thorough peer review Virtual reality: feasibility of implementation in a regional burn center. Burns. � Inclusion in PubMed and all major indexing services 2009;35:967–9. � Maximum visibility for your research 33. Glegg S, Tatla SK, Holsti L. The GestureTek virtual reality system for physical and cognitive rehabilitation: A scoping review. Disabil Rehabil Assist Submit your manuscript at Technol. 2014;9:89–111. www.biomedcentral.com/submit

Journal

BMC Health Services Research – Springer Journals

Published: Dec 1, 2016

Keywords: public health; health administration; health informatics; nursing research

There are no references for this article.