TY - JOUR AU - Cully, Jeffrey A AB - Implications Practice: Flexible, implementation-oriented training programs for evidence-based psychotherapies can be used to the enhance adoption of these therapies in complex healthcare systems. Policy: Implementing evidence-based therapies is a multifaceted endeavor, and training programs benefit from flexibility and inclusion of implementation elements in addition to clinical fidelity training. Research: Future research would benefit from studies that examine how to both foster clinical fidelity and adoption in evidence-based psychotherapy training programs. Despite strong empirical support for the efficacy and effectiveness of evidence-based psychotherapies [EBPs; e.g., [1, 2], EBPs are underused in large healthcare systems [3–5]. Ensuring that EBPs reach the individuals who need them is a pressing challenge facing future EBP innovation. Many factors impede moving EBPs into practice, including system priorities (e.g., incentives), local structures (e.g., workload demands), clinician characteristics (e.g., training in the EBP, preference for other practices), and patient needs (e.g., work schedule). Training and equipping providers to deliver EBPs is critical to this process. Growing evidence suggests that provider training alone is not sufficient to facilitate provider use of EBPs in clinical care settings [3]. Targeted efforts to reduce the research to practice gap are essential to increase delivery of high-quality mental health interventions which are often complex and highly reliant upon relationship and interpersonal factors in concert with technical treatment strategies [6]. Improving and innovating training practices and providing post-training practice support in using the intervention in their clinical setting (e.g., implementation facilitation) [7] hold the potential to assist providers when embedding EBPs into their day-to-day clinical activities. The Veterans Health Administration (VHA), the largest integrated healthcare system in the US, has a long history of programs designed to increase the use of EBPs—particularly for post-traumatic stress disorder (PTSD) and depression [8–10]. Stemming from the Comprehensive VA Mental Health Strategic Plan [11], VA has worked to increase the use of EBPs within mental health settings through the provision of national training and dissemination programs [10]. EBP training programs in VA often involve a multi-day didactic seminar and several weeks of consultation with a certified trainer [10, 12]. These programs have greatly expanded the pool of providers trained to deliver EBPs in VA [9]. However, the adoption of EBPs by providers in their daily practice remains limited. As an example from VA, Rosen et al. [3] found that, among over 500 providers who were trained in an EBP for PTSD, on average providers used the EBP with fewer than two patients in the 18 months after the training, and nearly 30% either did not use the intervention or stopped using it after six months. Interestingly, factors that influenced the use of the EBP by providers included difficulty scheduling sufficient session time and beliefs about the EBP (i.e., perceived strength of evidence). Although EBP implementation programs often attend to systems-related barriers through institutional support for interventions [10], these findings highlight that factors beyond training influence providers’ adoption of EBPs into their frontline practice. Transitioning training programs from clinical trials to implementation and use in frontline clinical care settings is understudied, with little guidance available for the field. Bridging the gap between intervention training and implementation through the creation of enhanced training and support practices hold the potential to enhance EBP adoption in healthcare systems. Barriers to EBP adoption by providers can occur at any or all of the following sequential steps (see Figure 1): (1) strength of evidence, (2) alignment with provider practice setting, (3) resources required to complete training, and (4) delivery factors. EBPs are complex clinical practices and face formidable adoption challenges given the time and resources necessary for training [10]. Providers who are trained to deliver a specific EBP then face additional challenges in adopting and using EBP in frontline practice, such as scheduling weekly sessions in a busy clinic [3, 13]. Strategies to address these challenges can be embedded into the EBP consultation process and may include establishing goals for EBP delivery (e.g., number of patients receiving EBP), reinforcing successes, problem-solving specific delivery challenges, and facilitating discussions with clinical leadership to address broader EBP alignment issues (e.g., scheduling weekly sessions) within the practice setting [14]. Considering these barriers, training in an intervention combined with simultaneous implementation support is one potential strategy to further embed EBPs in clinical practice. A training program is unlikely to reduce all these barriers, but empowering providers to navigate implementation in their clinical context is likely to improve the sustainability of an EBP. Fig 1 Open in new tabDownload slide Factors that influence the uptake of EBPs by providers in front line clinical practice. Fig 1 Open in new tabDownload slide Factors that influence the uptake of EBPs by providers in front line clinical practice. In this article, we describe experiences and lessons learned from two large federally funded projects that sought to test and innovate training and delivery programs for a specific evidence-based psychotherapy–brief cognitive behavioral therapy (i.e., brief CBT) [15]. A comprehensive training and delivery support program was iteratively developed and provided to frontline mental health providers to enhance the adoption and delivery of brief CBT in VA clinical care settings. We present our training and provider support methods as well as case-report data related to the evaluation of these provider support strategies. The overall purpose of this article is to illustrate how innovations in EBP training and support by training programs have the potential to better align with the needs of frontline providers and ultimately translate into improved adoption and use of EBPs in frontline clinical care settings. BRIEF COGNITIVE-BEHAVIORAL PROGRAM EXAMPLES In partnership with clinical operations in VA, our team developed and refined provider training and support programs to improve the delivery of brief cognitive behavioral therapy (bCBT) in primary care [16] and community-based outpatient medical clinics (CBOCs) [17]. Mental health providers in VA primary care settings are tasked with performing rapid assessment and brief treatment for Veterans with mild-to-moderate depression, anxiety, substance use and PTSD that complements care received from their primary care medical providers. Mental health providers in VA CBOC settings may provide care as part of a primary care team but commonly have responsibility for the provision of a wide range of mental health services and, depending on the size of the clinic, may be the only provider of psychotherapy at their facility. Providers in these settings are often challenged with limited resources and time available to provide longer EBP programs that might be available in traditional specialty mental health settings. These programs, developed within the VA’s integrated healthcare system, were informed by implementation science principles while prioritizing the needs of frontline providers to increase delivery of the EBP and to maximize patient outcomes. Both projects used a hybrid effectiveness-implementation design [18] and examined patient clinical outcomes as well as provider training and delivery outcomes. The first study, ACCESS [15], was conducted by mental health providers in VA primary care clinics, and targeted Veterans with anxiety and depression and comorbid cardiopulmonary conditions (e.g., heart failure, chronic obstructive pulmonary disease). The second study recruited Veterans with depression (i.e., no comorbid medical condition required) and providers from VA Community Based Outpatient Clinic (CBOC) settings (referred to as the CBOC study in the current article) [17]. bCBT INTERVENTION The bCBT training approach has been refined through multiple feedback avenues across diverse stakeholder groups (operations, leadership, providers, and patients) designed to improve clinical outcomes and delivery success. In the remainder of this article, we present descriptive data related to training elements and provider feedback as case illustrations to articulate the processes and outcomes of our approach to training and enhancing delivery of the brief CBT program. The bCBT clinical protocol included individual, therapist-led sessions that focused on self-management of mood and physical health symptoms. Providers and patients received corresponding intervention materials to facilitate patient engagement, between-session work, and structure for both in-person and telephone-delivered services. The skills-based intervention was delivered within six to nine sessions and allowed providers and patients to customize treatment to fit the most pressing needs of the patient [19]. Effectiveness of the intervention has been documented through large multisite randomized trials showing improvement in patients’ mental health symptoms (e.g., depression and anxiety) as well as physical health and quality of life [15, 17, 20]. Notably, interventionists in both trials were frontline mental health providers who conducted bCBT in their routine clinical practices. PROVIDER TRAINING PROGRAM For each trial, a comprehensive training and support program was developed to assist providers with acquiring clinical knowledge and skills as well as providing consultation and practical advice in order to increase delivery of the EBP (Table 1). The bCBT training and support program sought to maximize training efficiency and sustainability of the intervention in front-line practice [21]. One major contributor to achieving these goals was flexibility. Over the course of these trials, we learned that a tailored approach was more effective and appreciated by providers. The individualized training curriculum was achieved through a pre-training needs assessment and a discussion between the provider and bCBT trainer to collaboratively select relevant training modules in an online training program. The online, modular training consisted of self-paced presentations and videos that providers could complete at their convenience, asynchronously (total content of all modules was up to 8–10 hr but providers were only assigned modules that fit their professional development needs—see section Training Elements section below). The content focused on treatment elements of each bCBT module and included case examples. Modules covered basics of CBT, goal setting, measurement-based care, cognitive restructuring, and improving health and wellness. Post-module quizzes assessed comprehension and provided feedback on responses [22]. In the ACCESS trial, providers indicated which modules they were interested in completing based on their prior experience and learning goals and were assigned modules in collaboration with CBT expert consultants. We refined this process in the CBOC study, in which bCBT experts were assigned modules based on an assessment of each provider’s prior CBT knowledge, experience, and current need. The number of assigned modules and their completion was tracked throughout the training process. The modular nature of the training allowed for targeting areas where providers believed they were less skilled and focusing providers’ time on those modules. Providers completed the training within 2–3 weeks. The modular approach allows providers to fit training within their schedule rather than blocking entire days of clinic for workshop attendance. Table 1 Strategies used in training and delivery support in the two studies Flexible Provider Training Program . Provider needs assessment to match provider skills with training needs Creation of an individualized training plan Flexile web-based training program with customizable modular-based curriculum Delivery Support Individualized provider delivery goals Expert consultation to address practice challenges and delivery needs Content and delivery fidelity assessment as part of professional development approach (strengths and areas of development) Flexible Provider Training Program . Provider needs assessment to match provider skills with training needs Creation of an individualized training plan Flexile web-based training program with customizable modular-based curriculum Delivery Support Individualized provider delivery goals Expert consultation to address practice challenges and delivery needs Content and delivery fidelity assessment as part of professional development approach (strengths and areas of development) Open in new tab Table 1 Strategies used in training and delivery support in the two studies Flexible Provider Training Program . Provider needs assessment to match provider skills with training needs Creation of an individualized training plan Flexile web-based training program with customizable modular-based curriculum Delivery Support Individualized provider delivery goals Expert consultation to address practice challenges and delivery needs Content and delivery fidelity assessment as part of professional development approach (strengths and areas of development) Flexible Provider Training Program . Provider needs assessment to match provider skills with training needs Creation of an individualized training plan Flexile web-based training program with customizable modular-based curriculum Delivery Support Individualized provider delivery goals Expert consultation to address practice challenges and delivery needs Content and delivery fidelity assessment as part of professional development approach (strengths and areas of development) Open in new tab DELIVERY SUPPORT PROGRAM The second major aim of the program was supporting providers in the use of their newly acquired clinical skills. Like similar programs, the primary mechanism to achieve this aim was consultation with CBT experts. Following completion of the bCBT online clinical training, providers engaged in routine consultation calls with bCBT experts to assist providers in their development of an autonomous EBP practice. Consultants discussed bCBT content and implementation challenges. Assessment of fidelity was an element of the consultation process and was conducted via audio recordings and completing standardized ratings of selected sessions. However, fidelity was more broadly conceptualized as a professional development activity (e.g., building CBT skills beyond the bCBT protocol). In this way, consultants not only guided learners in performing the intervention correctly, but also on how to become fluent in bCBT to make autonomous clinical decisions within the context of their unique clinical setting. Consultants used these reviews and ratings for professional development focused on the individual successes and challenges of each provider delivering bCBT in their practice. At least one consultation session was a required training component, and additional clinical or fidelity discussions often occurred at the request of the provider. Consultants also worked with providers to set delivery goals (i.e., at least 1 new patient per month), and provided data reports on providers’ progress (e.g., note template use, measurement-based care use) to facilitate goal setting and problem-solving. ADOPTION OUTCOMES Determining measurement strategies appropriate to the goals of the training program was a key component of our process. Data were collected on the training and support program including provider satisfaction, training completion rates, as well as delivery data that included number of patients who received bCBT indicated by use of structured electronic medical record templates (a common tool for measuring EBP use in VA) in both projects. The program included implementation-related measurement and evaluation to assess provider reactions to the EBP, the training and support program, as well as practical outcomes including training completion rates and EBP delivery data. We also evaluated providers’ ability to successfully deploy the intervention in the context of their clinical practice in addition to fidelity to the content of the EBP protocol. OUTCOMES OF TRAINING AND SUPPORT PROGRAM In the first trial, 20 primary-care based mental health providers were invited to participate in the program. These providers included staff psychologists, psychology trainees (i.e., interns and fellows), staff social workers, and physicians’ assistants. Of these, one opted not to participate, leaving 19 enrolled providers who completed the training (100%) and 18 of the 19 enrolled providers (95%) who used bCBT with patients in their clinical setting. In the second study, a broader and more diverse provider group was approached to increase programmatic outreach efforts. In total, 30 providers were invited to participate, 17 enrolled in training, and 16 completed training. Of the 16 trained providers, 94% used bCBT with at least one patient in the 3 months following training completion. Disciplines included were the same as the ACCESS study, with the exception of physicians’ assistants. In both studies, there were no incentives for providers’ participation other than offering the training and support program. In addition to clinical outcomes, these studies sought to examine whether the training and support program is feasible for use in primary care and CBOC settings [23]. TRAINING ELEMENTS Regarding the flexible training approach, in the CBOC study (data not available in ACCESS study), of 9 possible modules, the mean number of modules assigned per provider was 8.17 (SD = 1.85), and the mean number of training modules completed was 7.87 (SD = 2.32). On average, providers reported that they spent 3 to 4 hr in the training. These data indicate that overall, busy front-line providers were able to complete nearly all aspects of the online training program whenever it best fit their schedule. It should be noted that, although the content was able to be tailored, many providers were assigned most modules, suggesting that in the CBOC study the majority of the content was either needed or desired by the provider based on the provider need assessments. Although providers reported that the training required less time than expected, evaluation of providers’ fidelity in previously published work from the ACCESS study has shown high ratings of bCBT adherence and skillfulness [14]. The time commitment was relatively minimal relative to multi-day trainings, and more flexibly fit around providers’ existing clinical demands. Providers completed questionnaires and provided feedback related to their training experiences in both studies. In response to a question asking if the training was flexible and had little interference with job duties, (responses on a 1 strongly disagree to 5 strongly agree scale), providers in the ACCESS study reported a mean of 3.44 (SD = 1.01). Providers in the CBOC study, which used a formal needs assessment and tailored curriculum reported a mean of 4.19 (SD = 0.83). This finding suggests that the program’s flexibility and tailored curriculum may have contributed to high completion rates in the second study. Notably, most providers in the CBOC study (12 of 16) indicated that they completed all assigned modules in four hours or less. Provider feedback further illustrates the benefits of the online, asynchronous format: “I really liked the online modality…overall it seems the least intrusive option for presenting the material” and, “this training was different because it was self-directed vs. in-class–it was great.” SUPPORT ELEMENTS Tailoring the support strategy to the adoption context was a key feature of the program. Enabling the providers to use the training as a professional development tool (e.g., improve CBT knowledge and skill, develop implementation skills) was one way to help bCBT fit into the individuals’ context (e.g., clinicians’ career goals) and institutional context (e.g., clinical roles). In the ACCESS study, which focused on mental health care in a medical setting for patients with chronic illnesses such as COPD, one provider noted, “I think it’s been COPD more than anything else and so that was another way for me to kind of learn and grow for the unit as well.” Regarding mentoring support in the CBOC study, one provider noted “…they encouraged a lot of flexibility and how we’ve tried to apply the protocol which I thought was very, very good.” Speaking further to professional development within the clinical context, one provider noted that the training helped give them more confidence in the clinic. Providers rated an average of 4.24 (SD = 0.66) on an item asking if expert consultation was an important part of professional development (responses on a 1 strongly disagree to 5 strongly agree scale). ADOPTION AND DELIVERY DATA Although the providers volunteered to participate, agreed to be a part of a research study, and were referred patients who were recruited for the study, ultimately it was the providers’ choice to use bCBT with the patients referred to them (i.e., they were not required to use bCBT). Table 2 shows the numbers of patients who received bCBT from providers in both studies. Notably, most providers used bCBT with 4 or more patients (range = 1–25), suggesting that the training and support program was suitable for the adoption context. With regards to adoption, one provider noted, “After it ends … think I’m going to continue to use it to help me focus a little bit more- more targeted.” However, providers reported the need to be flexible when using the intervention with patients in their clinical settings (e.g., sometimes needing multiple sessions to cover a skill—rather than rigidly following a session-by-session plan). [13] To facilitate the evaluation of bCBT’s use in practice and simultaneously reduce providers’ documentation burden, we embedded a structured note template within the electronic health record for the CBOC study. The note template allowed providers to select pre-written key components (e.g., skills delivered, homework completion) while providing space for providers to make additional notes. We asked CBOC providers to use the template for any notes associated with bCBT visits. Based on chart reviews conducted as part of the CBOC study, providers used the templates in 99% of their bCBT encounters. In fact, only three bCBT encounters did not have an appropriate template, and two of these were due to a provider responding to urgent patient needs that required a clinical shift away from the bCBT session. Measurement-based care is often an important component of EBP programs in large healthcare systems [24] and may similarly inform the adoption of EBP practices. We trained providers to use measurement-based care and were asked to administer measures throughout the course of bCBT. Consistent with this, in the CBOC study, 94% of providers (n = 15) administered clinical outcome measures in at least 95% of sessions. Measurement and evaluation of EBP implementation is key to sustained use of clinical innovations and incorporating implementation-focused training into these efforts may aid these efforts in routine care. Table 2 Provider adoption of brief cognitive behavioral therapy Number of patients treated per provider . CBOC Study (N = 16 providers) . ACCESS Study (N = 18 providers) . Combined Data (N = 34 providers) . 1–3 5 (31.3%) 1 (5.3%) 6 (17.7%) 4 or more 11 (68.8% 17 (94.4%) 28 (82.4%) 10 or more 4 (25.0%) 7 (38.9%) 11 (32.4%) Number of patients treated per provider . CBOC Study (N = 16 providers) . ACCESS Study (N = 18 providers) . Combined Data (N = 34 providers) . 1–3 5 (31.3%) 1 (5.3%) 6 (17.7%) 4 or more 11 (68.8% 17 (94.4%) 28 (82.4%) 10 or more 4 (25.0%) 7 (38.9%) 11 (32.4%) Note. Providers included in this table used the intervention with at least one patient. CBOC = community-based outpatient clinic; EBP = evidence-based psychotherapy. Percentages are not cumulative. Open in new tab Table 2 Provider adoption of brief cognitive behavioral therapy Number of patients treated per provider . CBOC Study (N = 16 providers) . ACCESS Study (N = 18 providers) . Combined Data (N = 34 providers) . 1–3 5 (31.3%) 1 (5.3%) 6 (17.7%) 4 or more 11 (68.8% 17 (94.4%) 28 (82.4%) 10 or more 4 (25.0%) 7 (38.9%) 11 (32.4%) Number of patients treated per provider . CBOC Study (N = 16 providers) . ACCESS Study (N = 18 providers) . Combined Data (N = 34 providers) . 1–3 5 (31.3%) 1 (5.3%) 6 (17.7%) 4 or more 11 (68.8% 17 (94.4%) 28 (82.4%) 10 or more 4 (25.0%) 7 (38.9%) 11 (32.4%) Note. Providers included in this table used the intervention with at least one patient. CBOC = community-based outpatient clinic; EBP = evidence-based psychotherapy. Percentages are not cumulative. Open in new tab Conclusions Increasing the adoption of EBPs in front-line mental health care settings such as primary care or community-based mental health clinics is a multifaceted endeavor. Although training programs are essential for teaching clinical skills, training alone often does not translate to improved adoption of interventions. This article provides an example of a provider EBP training program that sought to enhance provider EBP delivery through a tailored and flexible training that also provided post-training support to address practice barriers and challenges. Over the course of two pragmatic clinical trials, providers who received our training and support program showed striking adoption rates and very high (greater than 95%) rates of standardized note template and measurement-based care use. This may be in part because our training program emphasized implementation components (e.g., consultation on delivery challenges, training on measurement-based care) in addition to clinical fidelity and provided consultation and tools to enhance delivery (e.g., easy to use clinical documentation, feedback reports). The training and support approaches used in these programs have potential to be applied to future rollout programs to potentially bolster lower adoption rates [3]. Traditional EBP training strategies such as workshops or trainings adapted from randomized controlled trials may prepare providers well to use an intervention with a high degree of fidelity or clinical effectiveness—but often do not address real-world critical practice challenges that limit adoption of the EBP in frontline practice—especially in complex integrated healthcare systems. Initial data and feedback from providers suggest the program has worked to blend training and clinical fidelity with enhancing the likelihood that EBPs could be used sustainably in clinical practice. Specific elements of the training and support program included tailored clinical training curriculum and adoption-focused consultation to support providers after clinical training is completed. Providers reported that the tailored training was a manageable time commitment, which is important to allow providers to complete training in busy clinical settings. The provider data also suggest that the training, including the mentorship and consultation phase, provided value to their professional development, which may also be an important consideration for increasing the uptake of training in EBPs beyond top-down initiatives or institutional mandates. Front-line mental health care settings in which it is a goal to increase EBP use may benefit from incorporating flexible training designs and using ongoing consultation to help providers work through challenges adopting the intervention in their practice. Although training and supporting providers is an important aspect of the dissemination and implementation of EBPs, it is important to note that they are pieces of a larger puzzle. Other equally important factors influence EBP use. Larger policy-level considerations such as priorities of the providers’ institution (i.e., whether the target disorder of the EBP is important to the institution compared to competing conditions) in turn influence factors that would enable training and support to occur, including funding decisions and incentives to deliver EBPs. Further, system leaders and clinic leaders influence the priorities that translate into routine practice. Finally, patients must also be available and willing to participate in EBPs. That is, training and provider uptake are interwoven with policy, system, facility, and patient factors in the effort to make EBPs more available to patients. LIMITATIONS AND FUTURE DIRECTIONS The current paper must be considered in light of its limitations. First, the data presented here are case examples of a specific training approach, and these findings may not be generalizable beyond the contexts of the trials. Future work specifically testing the training strategies as they relate to implementation outcomes in diverse settings is an important next step. Second, although providers worked in front line settings, the training, consultation, and outcome measurement occurred in the context of a funded research study which provided resources that may not be consistently available. Finally, clinical fidelity is important for ensuring quality care and future work would benefit from studies that examine how to both foster clinical fidelity and adoption. Intervention fidelity was high in the ACCESS study [15], but fidelity for CBOC study providers has not yet been determined. Training is an essential part of improving access to EBP and encompasses a wide domain of factors, including training content, the institution’s priorities, and provider factors [23]. Research has found that EBP workshops that focus mainly on clinical fidelity have little impact on providers’ decisions to use EBPs in their practice [24]. It may be that training programs should highlight ways in which a protocol is flexible (within fidelity standards) to meet the demands of front-line practice such as changes in frequency, session length. Training can be used to ensure clinical quality and also encourage successful, sustainable use of EBPs in large healthcare systems. Different providers, as well as different interventions, may require varying approaches to training. For example, EBPs may vary in the degree to which they require intervention-specific technical skills for training (e.g., nuanced exposures) or skills that may already be familiar to learners but applied in a different context (e.g., Veteran-specific content). Ensuring that high-quality EBPs are available to all who could benefit is an important goal of mental health care systems. Using implementation and dissemination science to inform models of training has the potential to help move us towards this goal. Acknowledgments This study was funded by Department of Veterans Affairs, Health Services Research & Development Grant # IIR 13–315 and #IIR 09-088 to Jeffrey A. Cully and use of the facilities and resources of the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN13-413) and the VA South Central Mental Illness Research, Education and Clinical Center. The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the US government or Baylor College of Medicine. Compliance with Ethical Standards Conflict of Interest: None declared. Human Rights: All procedures performed in the studies involving human participants were in accordance with ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed Consent: Informed consewnt was obtained from all individual participants included in the studies. Welfare of Animals: This articles does not contain any studies with animals performed by any of the authors. Transparency Statements Study registration. This study was not formally registered. Analytic plan preregistration. The analysis plan was not formally pre-registered. Data availability. De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author. Analytic code availability. There is no analytic code associated with this study. Materials availability. Some of the materials used to conduct the study are presented in a public archive: __ https://www.mirecc.va.gov/visn16/docs/therapists_guide_to_brief_cbtmanual.pdf REFERENCES 1. Hofmann SG , Asnaani A, Vonk IJ, Sawyer AT, Fang A. The efficacy of cognitive behavioral therapy: a review of meta-analyses . Cognit Ther Res. 2012 ; 36 ( 5 ): 427 – 440 . Google Scholar Crossref Search ADS PubMed WorldCat 2. Stewart RE , Chambless DL. Cognitive-behavioral therapy for adult anxiety disorders in clinical practice: a meta-analysis of effectiveness studies . J Consult Clin Psychol. 2009 ; 77 ( 4 ): 595 – 606 . Google Scholar Crossref Search ADS PubMed WorldCat 3. Rosen CS , Eftekhari A, Crowley JJ, et al. Maintenance and reach of exposure psychotherapy for posttraumatic stress disorder 18 months after training . J Trauma Stress. 2017 ; 30 ( 1 ): 63 – 70 . Google Scholar Crossref Search ADS PubMed WorldCat 4. Finley EP , Garcia HA, Ketchum NS, et al. Utilization of evidence-based psychotherapies in Veterans Affairs posttraumatic stress disorder outpatient clinics . Psychol Serv. 2015 ; 12 ( 1 ): 73 – 82 . Google Scholar Crossref Search ADS PubMed WorldCat 5. Bruns EJ , Kerns SE, Pullmann MD, Hensley SW, Lutterman T, Hoagwood KE. Research, data, and evidence-based treatment use in state behavioral health systems, 2001-2012 . Psychiatr Serv. 2016 ; 67 ( 5 ): 496 – 503 . Google Scholar Crossref Search ADS PubMed WorldCat 6. Kauth MR , Sullivan G, Cully J, Blevins D. Facilitating practice changes in mental health clinics: a guide for implementation development in health care systems . Psychol Serv. 2011 ; 8 ( 1 ): 36 – 47 . Google Scholar Crossref Search ADS WorldCat 7. Smith TL , Landes SJ, Lester-Williams K, et al. Developing alternative training delivery methods to improve psychotherapy implementation in the U.S. Department of Veterans Affairs . Train Educ Prof Psychol. 2017 ; 11 ( 4 ): 266 – 275 . Google Scholar OpenURL Placeholder Text WorldCat 8. Karlin BE , Cross G. From the laboratory to the therapy room: national dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System . Am Psychol. 2014 ; 69 ( 1 ): 19 – 33 . Google Scholar Crossref Search ADS PubMed WorldCat 9. Karlin BE , Ruzek JI, Chard KM, et al. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration . J Trauma Stress. 2010 ; 23 ( 6 ): 663 – 673 . Google Scholar Crossref Search ADS PubMed WorldCat 10. Department of Veterans Affairs . Strategic Plan: 2005–2010 . Office of the Inspector General, Department of Veterans Affairs, Washington, DC ; 2005 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 11. Karlin BE , Brown GK, Trockel M, cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: therapist and patient-level outcomes . J Consul Clin Psychol. 2012 ;5: 707 – 718 . Google Scholar OpenURL Placeholder Text WorldCat 12. Mignogna J , Martin LA, Harik J, et al. “I had to somehow still be flexible”: exploring adaptations during implementation of brief cognitive behavioral therapy in primary care . Implement Sci. 2018 ; 13 ( 1 ): 76 . Google Scholar Crossref Search ADS PubMed WorldCat 13. Mignogna J , Hundt NE, Kauth MR, et al. Implementing brief cognitive behavioral therapy in primary care: a pilot study . Transl Behav Med. 2014 ; 4 ( 2 ): 175 – 183 . Google Scholar Crossref Search ADS PubMed WorldCat 14. Cully JA , Stanley MA, Petersen NJ, et al. Delivery of brief cognitive behavioral therapy for medically ill patients in primary care: a pragmatic randomized clinical trial . J Gen Intern Med. 2017 ; 32 ( 9 ): 1014 – 1024 . Google Scholar Crossref Search ADS PubMed WorldCat 15. Cully JA , Armento ME, Mott J, et al. Brief cognitive behavioral therapy in primary care: a hybrid type 2 patient-randomized effectiveness-implementation design . Implement Sci. 2012 ; 7 : 64 . Google Scholar Crossref Search ADS PubMed WorldCat 16. Johnson AL , Ecker AH, Fletcher TL, et al. Increasing the impact of randomized controlled trials: an example of a hybrid effectiveness–implementation design in psychotherapy research . Transl Behav Med. 10(3):629–636. doi: 10.1093/tbm/iby116 OpenURL Placeholder Text WorldCat 17. Curran GM , Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact . Med Care. 2012 ; 50 ( 3 ): 217 – 226 . Google Scholar Crossref Search ADS PubMed WorldCat 18. Cully JA , Paukert A, Falco J, Stanley M. Cognitive-behavioral therapy: innovations for cardiopulmonary patients with depression and anxiety . Cogn Behav Pract. 2009 ; 16 : 394 – 407 . Google Scholar Crossref Search ADS WorldCat 19. Hundt NE , Renn BN, Sansgiry S, et al. Predictors of response to brief CBT in patients with cardiopulmonary conditions . Health Psychol. 2018 ; 37 ( 9 ): 866 – 873 . Google Scholar Crossref Search ADS PubMed WorldCat 20. Cully JA , Curry AD, Ryan SR, Malik A, Zeno D, Willcockson IU. Development of a computer-aided training program for brief cognitive-behavioral therapy in primary care . Acad Psychiatry. 2013 ; 37 ( 2 ): 120 – 124 . Google Scholar Crossref Search ADS PubMed WorldCat 21. Sorocco K , Mignogna J, Kauth MR, et al. Online CBT training for mental health providers in primary care . J Ment Health Train Educ Pract. 2018 ; 13 ( 4 ): 228 – 237 . Google Scholar Crossref Search ADS WorldCat 22. Fortney JC , Unützer J, Wrenn G, et al. A tipping point for measurement-based care . Focus. 2018 ; 16 ( 3 ): 341 – 350 . Google Scholar Crossref Search ADS WorldCat 23. Beidas RS , Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective . Clin Psychol. 2010 ; 17 ( 1 ): 1 – 30 . Google Scholar OpenURL Placeholder Text WorldCat 24. Herschell AD , Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations . Clin Psychol Rev. 2010 ; 30 ( 4 ): 448 – 466 . Google Scholar Crossref Search ADS PubMed WorldCat Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2021. This work is written by (a) US Government employee(s) and is in the public domain in the US. Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2021. TI - Training and supporting mental health providers to implement evidence-based psychotherapies in frontline practice JF - Translational Behavioral Medicine DO - 10.1093/tbm/ibab084 DA - 2022-01-18 UR - https://www.deepdyve.com/lp/oxford-university-press/training-and-supporting-mental-health-providers-to-implement-evidence-64p5t2wX4m VL - 12 IS - 1 DP - DeepDyve ER -