Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Development of a web-based toolkit to support improvement of care coordination in primary care

Development of a web-based toolkit to support improvement of care coordination in primary care Implications of Healthcare Innovation, Promising practices for the coordination of chronic care exist, Practice: To narrow down the very large num- Implementation, and Policy but how to select and share these practices to support quality ber of available care coordination tools to those (CSHIIP), VA Greater Los Angeles improvement within a healthcare system is uncertain. This study Healthcare System, Sepulveda, relevant to local improvement goals and context, California, USA describes an approach for selecting high-quality tools for an healthcare leaders and managers can adapt and David Geffen School of Medicine, online care coordination toolkit to be used in Veterans Health apply the tool selection process we outline in this University of California at Los Administration (VA) primary care practices. We evaluated tools article. Angeles, Los Angeles, CA, USA in three steps: (1) an initial screening to identify tools relevant to RAND Health, Santa Monica, CA, care coordination in VA primary care, (2) a two-clinician expert USA review process assessing tool characteristics (e.g. frequency Policy: A structured approach to toolkit develop- School of Biomedical Informatics, of problem addressed, linkage to patients’ experience of care, University of Texas Health Science ment can facilitate the spread of care innovations effect on practice workflow, and sustainability with existing Center at Houston, Houston, TX, for quality improvement efforts, but toolkits are resources) and assigning each tool a summary rating, and (3) USA an adjunct to, not a replacement for, a serious 5 semi-structured interviews with VA patients and frontline clini- Department of Psychiatry and commitment on the part of clinic leadership and cians and staff. Of 300 potentially relevant tools identified by Biobehavioral Sciences, University staff to improve care coordination. of California at Los Angeles, Los searching online resources, 65, 38, and 18 remained after steps Angeles, CA, USA one, two and three, respectively. The 18 tools cover five topics: Veterans Evidence-based managing referrals to specialty care, medication management, Research Dissemination and patient after-visit summary, patient activation materials, agenda Research: To ensure that toolkits are effective Implementation Center (VERDICT), setting, patient pre-visit packet, and provider contact information in achieving their desired ends, additional work South Texas VA Health Care for patients. The final toolkit provides access to the 18 tools, as should characterize the development processes System, San Antonio, TX, USA 7 well as detailed information about tools’ expected benefits, and and design features of different toolkits, including University of Texas Health resources required for tool implementation. Future care coordin- the relationship between these processes and fea- Science Center at San Antonio, San ation efforts can benefit from systematically reviewing available tures and subsequent tool adoption, implementa- Antonio, TX, USA tools to identify those that are high quality and relevant. tion, and sustainability. HSR&D Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System, Menlo Park, CA, USA Keywords Division of Primary Care and coordination strategies in the published research lit- Care coordination, Primary care, Quality improve- Population Health, Stanford erature [5], effective methods to identify, document, University, Stanford, CA, USA ment, Toolkit, Distance coaching spread, and sustain evidence-based care coordin- Correspondence to: David ation approaches in routine care are not clear. The A Ganz, David.Ganz@va.gov Coordination Toolkit and Coaching (CTAC) project INTRODUCTION aims to improve patients’ experience of care coord- Cite this as: TBM 2018;8:492–502 Inefficiencies in coordination of chronic care are a doi: 10.1093/tbm/ibx072 ination within VA primary care and between VA significant source of waste in the United States (USD primary care and other outpatient settings, including $25–$45 billion in 2011) [1] and a barrier to achiev- Published by Oxford University Press health care provided in the community, while also on behalf of the Society of Behavioral ing the Triple Aim of improved patient experience Medicine 2018. This work is written by developing better methods for bringing research evi- of care, lower per capita costs, and better population (a) US Government employees(s) and dence on care coordination into routine care. is in the public domain in the US. health [2]. In the Veterans Health Administration To implement research on care coordination in (VA), recent legislation allowing some patients to practice, any coordination strategy must not only be seek care from community medical providers (i.e. evidence-based, but also applicable to the context of outside of VA owned and operated facilities) has fur- the adopting organization. The gap between research ther complicated care coordination in the primary and practice exists in part because research results care practices of a system where patients already for complex problems such as care coordination are have higher rates of comorbidity and mental illness often presented generically, without the supporting than private sector counterparts [3, 4]. Although documentation of the tools required to achieve such a systematic review has shown the benefits of care page 492 to 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH results. Additionally, research results often do not Organizational context apply when conducted under the varied resource The VA is a national integrated healthcare delivery constraints, patient populations, and organizational system serving 5.9 million Veteran patients annu- milieus that are found in practice [6]. These con- ally [11], with a central headquarters in Washington, straints may strongly shape the specific intervention DC and 18 healthcare networks known as Veterans elements that can be applied successfully within any Integrated Service Networks (VISNs); each VISN specific care coordination improvement effort. For includes multiple healthcare “facilities” that are sys- example, though the VA’s shared information technol- tems in themselves, offering inpatient and outpatient ogy infrastructure greatly facilitates internal sharing of care, often across multiple sites. In the VA, most innovations, it also means that innovations in research patients are assigned to a primary care team [12]. studies or available in general repositories such as Since 2010, the VA has focused on transforming its the Agency for Healthcare Research and Quality primary care teams into a patient-centered medi- (AHRQ) Innovations Exchange may not be applic- cal home model with augmented staffing (one reg- able to a local VA initiative because of incompatibility istered nurse care manager, one licensed practical with VA’s electronic health record. In addition, indi- nurse, and one administrative clerk for each primary viduals delivering clinical care may have limited time care provider), called patient-aligned care teams to search for tools, make a comparison between avail- (PACT), and PACTs are tasked with coordinating able options, and decide which tools to adopt. Thus, the patient’s care [13]. VA’s status as an integrated one way to facilitate the uptake of research in prac- healthcare delivery system offers the benefits of a tice is to compile a toolkit—in essence, a “short list” of shared electronic health record and communication action-oriented strategies for improving care coordin- systems, including electronic mail, for VA-owned ation that is compatible with the local context [7]. and operated facilities and selected contractors. In this article, we describe CTAC’s first phase, which involved selecting tools for a toolkit and Toolkit development developing a VA Intranet site to support the tools. The decision of which tools to include in the toolkit We aimed to evaluate candidate tools based on involved three steps: (1) an initial review of content the existing evidence base for effective care coord- to determine if tools met inclusion or exclusion crite- ination strategies, relevance to improving patients’ ria, (2) a two-clinician review process to rate tools on experience of care coordination, tool feasibility and their characteristics, and (3) semi-structured inter- quality, and evaluations by VA patients and frontline views with VA patients and frontline clinicians and VA primary care providers and staff. Within the VA, staff. Figure 1, adapted from the PRISMA statement toolkits are often used in conjunction with other dis- [14], shows the flow of tools through this three-step semination methods, such as learning collaboratives process. [8] and quality improvement coaching, and a later In the first step, the principal investigator and phase of CTAC will compare the additional value project manager identified tools using a snowball of distance-based coaching to implement the tools, approach, starting with two VA toolkits (focused on with access to the toolkit without coaching. PACT and specialty care) [15, 16] and two nonVA toolkits [17, 18]. The focus of this initial review METHODS was to determine the candidate tools’ general rel- evance to the project topic. We excluded tools for Definitions any of the following reasons: tools were restricted Care coordination is defined as “…the deliberate to a particular clinical domain, such as being con- organization of patient care activities between two dition-specific (e.g. only for diabetes) or covering or more participants (including the patient) involved only clinical preventive services; were intended in a patient’s care to facilitate the appropriate deliv- for use primarily in a setting outside of primary ery of health care services. Organizing care involves care (e.g. specialty care, inpatient care); were not the marshalling of personnel and other resources focused on care coordination (e.g. only access needed to carry out all required patient care activ- focused, only continuity focused, or only general ities, and is often managed by the exchange of quality improvement); were not truly a tool (e.g. information among participants responsible for general information sheets that were not action-ori- different aspects of care” [9]. A  toolkit is defined ented); the VA was already disseminating the tool as “a collection of related information, resources, or strategy widely (e.g. tools to encourage uptake of or tools that together can guide users to develop a secure messaging between patients and providers); plan or organize efforts to follow evidence-based the tool was not applicable to the VA setting; the recommendations or meet evidence-based specific tool was part of a toolkit with interdependent tools practice standards,” and a tool as “an instrument that could not stand alone as individual tools; or (e.g. survey, guidelines, or checklist) that helps users CTAC’s distance-based coach would not be able accomplish a specific task that contributes to meet- to help primary care teams with using the tool. ing a specific evidence-based recommendation or The principal investigator, project manager, and practice standard” [10]. TBM page 493 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Fig 1 | Three-step tool identification, screening and eligibility determination process. Preliminary exclusion criteria included a tool being: (1) condition-specific, (2) covering only clinical preventive services, (3) intended for use outside of primary care, (4) not focused on care coordination, (5) not truly a tool, (6) VA already using the tool or strategy widely, (7) not applicable to the VA setting, (8) part of a toolkit and could not stand alone as an individual tool, or (9) not coachable by CTAC’s distance-based coach. Exclusions based on the tool-rating checklist (Supplementary Appendix) and reviewer discussion and consensus. Excluded tools classified as supporting materials or add- itional resources for the included tools. two additional project staff then sorted remaining discussed these differences, they had the option of tools meeting inclusion criteria into categories and changing their ratings if desired. sub-categories to organize them according to the In the third step, the principal investigator (inter- topics they covered. viewer) and project manager (note-taker) conducted In the second step, two clinicians independently semi-structured interviews with VA providers, clinic reviewed remaining tools, using a newly developed staff, and patients in order to obtain feedback about and piloted tool-rating checklist (Supplementary the tools and learn more about how the tools might Appendix). During the second step, the principal function in a clinic setting. A  total of eight inter- investigator served as the first reviewer of all the views were completed on a broad selection of the tools, with the role of second reviewer split between included tools, five of them individual interviews five clinicians, who were assigned roughly equal with PACT clinic staff from one VA site (primary numbers of tools to review. After the reviews were care provider, registered nurse, lead clinic clerk, completed, the first and second reviewers met to dis- psychologist and pharmacist) and three of them cuss substantial differences in ratings (defined as a group interviews from another site (two Veterans two-point difference on any 3-point rating of a given receiving health care services at the VA, members dimension of the tool, or at least a 3-point difference of the 21-member VISN 22 clinician Community on the 7-point tool summary rating). After reviewers of Practice, and six members of the nine-member page 494 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH VISN 22 Nursing Workgroup). Interviews lasted the pathways by which the selected tools might 1 hr each, and were all conducted by phone except affect dimensions of the VA’s Strategic Analytics for the in-person group interview of the two Veteran for Improvement and Learning (SAIL) domains, patients. A  pre-determined selection of tools, spe- which are an internal dashboard used by VA cifically chosen for each interviewee based on his/ leaders to benchmark VA facilities’ performance her role at the VA, was then presented, via email [19]. As the CTAC project focuses on improv- attachment, an online screen-sharing program, or in ing patients’ experience of care coordination, we person, as applicable. Interviewees provided feed- determined in more detail the mapping of tools back about the tools with interviewer prompting to the Health System Hassles Scale [20], which via questions such as “Do you have a similar tool assesses patients’ challenges using the health- in use  at your facility?”, “Would a tool like this be care system, such as getting medications refilled helpful at your facility?”, and “How would this tool on time, lack of information about why patients be used at your facility?” The group interviews of were referred to a specialist, poor communication VA providers and staff involved a specific clinical between different doctors or clinics, and disagree- scenario and set of tools that were presented to the ments between doctors about the patient’s diag- group, with the group then providing feedback. nosis or treatment, and is included in the AHRQ After these three steps were completed, final Care Coordination Measures Atlas [9]. assignment of tools to categories occurred, and Ethics approval tools were deployed to an online website access- ible via the VA Intranet, using SharePoint®. Using The CTAC project was determined to be nonre- rapid cycles of development, a SharePoint pro- search by the VA Office of Primary Care Services grammer worked with the principal investigator (in accordance with VHA Handbook 1058.05) [21] and project manager to develop the toolkit web- and by the VA Greater Los Angeles Healthcare site’s functionality, usability, and look and feel. System Institutional Review Board. An internal SharePoint site was chosen because of rapidity of deployment and the ease with which RESULTS tools could be added or changed in the future. We Review of 22 toolkits, manuals and resources (the placed special emphasis on developing “tool access original four toolkits plus 18 found through a snow- pages” (Supplementary Appendix), which provide ball search) resulted in identification of 300 poten- prospective users with a preview of the amount of tial tools from these sources (Figure 1). Application time and effort required to deploy the tool, and the of initial inclusion and exclusion criteria narrowed expected benefits of the tool. the list of potential tools to 65. The two-clinician review process and subsequent project team dis- Evaluation of toolkit development process cussion further reduced the number of candidate To evaluate the process of tool selection and toolkit tools to 38. These 38 tools had higher clinician rat- development, we conducted a review of documenta- ings than excluded tools; however, the difference in tion collected during the process, including spread- summary ratings between included and excluded sheets of included and excluded tools, reviewer tools was only statistically significantly different ratings of tools, and field notes from semi-structured for the first reviewer, who reviewed all the tools interviews. We carried out quantitative analyses of himself (reviewer 1: median ± interquartile range, reconciled clinician reviewer scores from the sec- 5 ± 1 for included tools vs. 4 ± 1 for excluded tools, ond step of the tool selection process, calculating p  =  .0005). For the five different clinicians who median and interquartile ranges for tool summary shared the role of second reviewer, the collective ratings, which were on a 1–7 integer scale (a higher rating of included and excluded tools showed more number being more favorable). For inferential sta- variation around median values (reviewer 2: median tistics, we used the Wilcoxon signed-rank test to ± interquartile range, 5 ± 3 for included tools versus assess reviewer agreement on tool ratings, and the 4 ± 2 for excluded tools, p = .096). There was no sig- Wilcoxon rank-sum test for ratings of included ver- nificant difference in summary ratings between the sus excluded tools. Two-tailed p values were consid- two reviewers of each tool (p = .915). ered significant at p < .05. Semi-structured interviews highlighted factors that We also evaluated the degree to which the final might suggest a tool should be included or excluded. set of tools selected for inclusion conformed to For example, one set of tools presented during an evidence-based strategies, and the dimensions interview was focused on motivational interviewing of health care quality that the tools might affect. techniques and supporting curricula and resources. To determine the degree to which tools covered The discussion during the interview revealed that the range of evidence-based strategies for care training in this area and supplemental materials are coordination, we mapped the tools to a compil- already available to VA clinic staff. Other interviews ation of care coordination strategies shown effect- revealed specific challenges a facility might face ive in a systematic review [5]. We also mapped using a tool. For instance, when presented with an TBM page 495 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 1 | Care coordination toolkit tool list with descriptions Tool category and tool name Tool description Managing Referrals to Specialty Care VA Service Agreement Template A template with the purpose of facilitating timely access and patient-centered care for patients by promoting an effective relationship between the PACT team and the spe- cialty care team during the management of a patient’s care. Consult Guides for Primary Care A comprehensive list of clinical conditions and for each, a standard set of information to (Information to Include) be included in a referral request, to help ensure a meaningful visit when the referred patient visits the relevant specialist. While the tool consists of multiple data sets, not all need to be implemented. Relaxation and Meditation Program: Information and tools for implementing a program that teaches patients relaxation and An approach to self-management of meditation skills, with the potential benefit of decreasing the number of referrals to mental health in primary care specialty mental health services. Medication Management VA Formulary (Abbreviated) The Primary Care Service in the VA Palo Alto Healthcare System worked with its pharmacy department to create an abbreviated and searchable VA formulary. This resource was posted on the healthcare system’s external web page and is updated periodically for accuracy. AudioRENEWAL: Phone-based Allows patients to request a renewal of their prescription directly from within the Medication Renewal for Patients AudioCARE telephone refill system 24 hr a day, 7 days a week. Communication with Community A two-page document that includes a letter to help facilitate communication Providers to Co-Manage Veterans’ between a VA provider and an outside healthcare provider regarding management Care of the Veteran patient and instructions for obtaining medications through the VA, and a guide for providers (nonVA) to request a prescription medication that is nonformulary. Medication Tracker for Patients A one-page form with sections to be filled in by a member of the PACT team for the patient. Spaces are provided to fill in a patient’s medication details and healthcare provider name and contact information. The medication list includes details that can help patients remember when to take their medications and what dose to take. Patient After-Visit Summary After-Visit Instructions for patients A customizable document instructing patients about where to go before leaving (paper) the VA the day of their appointment, as well as any follow-up actions which need to be taken. Includes contact information for specialty clinics as well as a map of the campus, which can be used to direct patients to their next destination on campus. After-Visit Summary for patients An electronic tool that produces a customizable, printable patient summary that can be (electronic) provided to a patient after his/her primary care office visit to summarize visit content and subsequent action steps, if appropriate. Patient Activation Materials Patient Agenda-Setting Form A simple, easy to fill-out form to help patients prepare for their healthcare visit. The form can be sent to patients in the mail with a pre-visit packet, or can be given to patients to complete in the waiting room. Patient Treatment Decision Guide A worksheet for patients to use when faced with a medical decision. The form includes questions to ask the provider during the medical appointment, space to take notes, and guidance on how to proceed with making a decision. Tips for Patients: Improving A two-page handout that can be mailed to patients in a pre-visit packet or Communication with your Primary given to patients at check-in, before their appointment. The tool includes tips Care Team for communicating with the primary care team, questions for patients to think about before their appointment and space to write down answers. This tool can help ensure all of a patient’s questions and concerns are addressed during their medical appointment. Pharmacy Safety for Patients A patient handout that explains the role of the pharmacist, pharmacy and patient during the process of obtaining medications. Tips for Patients: Questions to Ask A guide that includes tips, ideas, and questions for patients to use before, during, and Before, During and After your Visit after their appointment that will help facilitate a successful healthcare visit. Provider Contact Information for Patients Clinic Information Pamphlet A template that clinics can customize to create a clinic information pamphlet for patients that includes pertinent information about the clinic. Information may include appoint- ment-making instructions, provider contact information, prescription refill instructions, and other clinic details. Continued page 496 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 1 | Continued Tool category and tool name Tool description My Primary Care Team: Contact and A one-page handout that lists important provider contact information for a patient. Other Information Includes contact details for the primary care provider, PACT nurse, tele-health nurse, specialty care, pharmacy, and mental health. My Primary Care Team: Wallet Card A customizable wallet-sized card that can be printed, populated with a patient’s primary with Contact Information care team information and given to the patient. The back of the card includes a list of information patients should provide when leaving a message at the clinic. Save a Trip to Primary Care A one-page resource for patients, explaining and illustrating the difference between routine, nonurgent, and urgent medical situations. The sheet explains to patients what actions to take in the case of each of these situations, and provides relevant contact information. agenda-setting tool that patients receive at check-in require substantial adaptation of the tool for use in and fill out in the waiting room, one interviewee the clinic, or the involvement or procurement of described check-in kiosks used at the clinic, and new resources such as additional staff, new equip- explained that Veterans do not typically see a clerk ment, or coordination with information technology if they check in via the kiosk. Therefore, having a specialists. An example of an easily implemented clerk hand a paper-based agenda-setting form to the tool is “My Primary Care Team – Wallet Card with patient at check-in would not be a viable approach Contact Information,” a wallet-sized card the clinic in this clinic setting. Based on the feedback from the prepares in advance and has available for patients semi-structured interviews, we were able to reduce when they visit the clinic. The wallet card provides the 38 tools at this stage to the final set of 18 (origi- provider names and contact information along with nally 19, with two tools combined into one; Table 1), the procedure patients should use when leaving a with many of the excluded 19 tools available on the message at the clinic. An example of a challenging toolkit website as additional resources or supporting tool to implement is the VA Care Coordination materials for the included tools. Service Agreement Template, a document that can The final set of 18 tools available in the toolkit be used to codify how a primary care team and a (available on the VA Intranet, see Supplementary specialty care team will coordinate care of shared Appendix for screenshots) covers five categories: patients, and which would require substantial nego- managing referrals to specialty care, medication man- tiation and ongoing meetings between these two agement, patient after-visit summary, patient activa- teams to be filled out meaningfully. tion materials, and provider contact information for Mapping the tools to available evidence-based patients. The tools in the “managing referrals to spe- care coordination strategies [5] showed good, cialty care” category support the appropriate and although not uniform, coverage (Table  2), with efficient use of specialty care and better communica- clear emphasis on providing support for patients tion between primary care and specialty care teams. (13 tools), followed by structured arrangements for The “medication management” category tools can coordinating service provision between providers help improve patients’ understanding of how to take (2 tools), providing support for service providers their medications or help patients obtain access to (2 tools), and structuring the relationships between needed medications more efficiently. The “patient service providers and with patients (1 tool). When after-visit summary” category includes tools that accounting for the additional resources available on offer a convenient way to document for patients the the toolkit website, coverage was improved, with the decisions made during their primary care visit and strategy of using systems to support care coordin- what follow-up is needed. Tools in the category of ation also included. One strategy (improving com- “patient activation materials” help prepare patients munication between service providers) was not for their visit with their primary care team, and are covered by the tools or additional resources. intended to be sent to (or shared with) the patient Table  3 shows dimensions of patient experience in advance of a visit. Finally, the “provider contact that might be favorably affected by use of the tools if information for patients” category tools help to pro- the tools were implemented appropriately. The three vide patients with important contact information, so problems addressed by the most tools were “diffi- they know whom to call for both routine and urgent culty getting questions answered or getting medi- health issues. cal advice between scheduled appointments” (six The 18 tools vary in the level of resources (e.g. tools), “lack of information about your medical con- staff time, information technology support) required ditions” (five tools), and “lack of information about to implement them. Simpler tools can be used “as why your medications have been prescribed to you” is” or with minimal adaptation, and may be imple- (five tools). Mapping tools to SAIL domains showed mented and sustained with existing clinic resources. that the primary domains affected were patient satis- Tools that may be more difficult to implement may faction and specialty care access (Figure 2). TBM page 497 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 2 | Tools and resources in toolkit, mapped to care coordination strategies in Powell Davies et al. [5] Tool category and tool name Care coordination strategy Managing Referrals to Specialty Care VA care coordination service agreement template Structured arrangements for coordinating service provi- sion between providers Consult guides for primary care (information to include) Structured arrangements for coordinating service provi- sion between providers Relaxation and meditation program: an approach to Structuring the relationships between service providers self-management of mental health in primary care and with patients Medication Management VA formulary (abbreviated) Providing support for service providers AudioRENEWAL: Phone-based medication renewal for patients Providing support for patients Communication with community providers to co-manage Providing support for service providers Veterans’ care Medication tracker for patients Providing support for patients Patient After-Visit Summary After-visit instructions for patients (paper) Providing support for patients After-visit summary for patients (electronic) Providing support for patients Patient Activation Materials Patient agenda-setting form Providing support for patients Patient treatment decision guide Providing support for patients Tips for patients: improving communication with your primary care team Providing support for patients Pharmacy safety Providing support for patients Tips for patients: Questions to ask before, during and after your visit Providing support for patients Provider Contact Information for Patients Clinic information pamphlet Providing support for patients My primary care team—contact and other information Providing support for patients My primary care team—wallet card with contact information Providing support for patients Save a trip to primary care Providing support for patients Additional Resources Enhancing communication with patients* Providing support for service providers Care management* Using systems to support care coordination *These are categories of additional resources, with multiple resources within each category. DISCUSSION disseminated in the VA. These aspects of our tool selection process show how the approach adapts We have reported on the development process to organizational context; in another context, our for an online toolkit to support better care coord- structured process could be tailored to different ination in primary care. We were successful in opportunities for improvement. narrowing down a list of 300 potential tools to Figure 2, adapted from the Chronic Care Model 18 that met our inclusion criteria and were ulti- [13], depicts the pathways by which tools and add- mately made available through our toolkit web- itional resources are ultimately expected to affect site. Our two-clinician review process was helpful aspects of care quality (patient satisfaction and spe- in deciding on a penultimate list of tools to be cialty care access) at the VA. Most tool categories reviewed more carefully in semi-structured inter- work to connect the patient to the health system dir- views. Our methods, particularly the tool review ectly, through increasing accessibility, empowering form, may be useful to others who are interested the patient, improving adherence/self-management, in a structured and systematic approach to toolkit or increasing understanding. One tool category, development. “managing referrals to specialty care,” operates Most of the tools that were ultimately included within the health system, but indirectly impacts pro- in the toolkit related to the coordination strategy ductive interactions by improving communication of providing support to patients, reflecting the and reducing inefficiencies that impact patient care. project’s emphasis on improving patient care expe- The tool review process we carried out is subject riences. Our process also aimed to include tools to several limitations. First, we attempted to use a that were complementary to existing VA resources, quantitative tool review process to determine which rather than duplicative of them. For example, many tools to subject to further testing; this review pro- tools and policies related to how primary care teams cess included explicitly rating certain tool features should internally coordinate their care were already page 498 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 3 | Tools in the Care Coordination Toolkit and associated item(s) on the Health System Hassles Scale Health System Hassles Scale items Tool category and tool name 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Managing Referrals to Specialty Care VA Care Coordination Service X X X Agreement Template Consult Guides for Primary Care X X X X X X (Information to Include) Relaxation and Meditation Program: X An approach to self-management of mental health in primary care Medication Management VA Formulary (Abbreviated) X AudioRENEWAL: Phone-based X X Medication Renewal for Patients Communication with Community X X Providers to Co-Manage Veterans’ Care Medication Tracker for Patients X X Patient After-Visit Summary After-Visit Instructions for Patients X X (paper) After Visit Summary for Patients X X X (electronic) Patient Activation Materials Patient Agenda-Setting Form X X X X X X Patient Treatment Decision Guide X X Tips for Patients: Improving X X X X X X Communication with your Primary Care Team Pharmacy Safety for Patients X Tips for Patients: Questions to ask X X X X Before During and After Your Visit Provider Contact Information for Patients Clinic Information Pamphlet X My Primary Care Team - Contact and X Other Information My Primary Care Team - Wallet Card X with Contact Information Save a Trip to Primary Care X X Health System Hassles Scale, response options, and 16 items. During the past 12 months, how much of a problem, if at all, has each of the following been for you? (Response options: A very big problem for you, a big problem for you, a moderate problem for you, a small problem for you, or not a problem at all for you.) 1 Lack of information about your medical conditions. 2 Lack of information about which treatment options are best for your medical condition. 3 Lack of information about why your medications have been prescribed to you. 4 Problems getting your medications refilled on time. 5 Uncertainty about when or how to take your medications. 6 Side effects from your medications. 7 Lack of information about why you have been referred to a specialist. 8 Having to wait a long time to get an appointment for specialty providers or clinics. 9 Poor communication between different healthcare providers. 10 Disagreements between your providers about your diagnoses or the best treatment for you. 11 Lack of information about why you need lab tests or x-rays. 12 Having to wait too long to find out about the results of lab tests or x-rays. 13 Difficulty getting questions answered or getting medical advice between scheduled appointments. 14 Lack of time to discuss all your problems with your health care provider during scheduled appointments. 15 Having your concerns ignored or overlooked by your healthcare providers. 16 Medical appointments that interfere with your work, family, or hobbies. TBM page 499 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Fig 2 | Pathways from tool use to improved dimensions of care quality in the VA. This diagram depicts interactions between the patient and the health system, including primary care (i.e. the “teamlet” which includes one registered nurse care manager, one licensed practical nurse, and one administrative clerk, and one primary care provider; and “the extended PACT team” which can include the addition of a social worker, psychologist, and/or pharmacist). Orange arrows represent tools supporting interactions between parties; bidirectional arrows represent tools requiring input from both parties, whereas uni-directional arrows represent tools where the one party is primarily responsible. Each specific arrow is accompanied by a numbered label that describes the tool and, in italics, the tool’s expected contribution to the overall goal to “facilitate productive interactions.” There are additional yellow markers denoting the SAIL domains that are expected to be positively affected by the productive interactions supported by the tools. and then providing a summary rating for each tool. the toolkit are patient-facing, we believe that many Although tool ratings did provide useful informa- could be used in nonVA settings as is, or with minor tion, we learned that they could not be used by them- adaptations. selves for an ultimate determination of each tool’s The field of toolkit development has been ham- value. Rather, tools had to be viewed in the context pered by limited evidence about what features of of other features of the project, including the unique toolkit development lead to effective strategies for features of the VA setting (including whether the VA implementing evidence-based innovations; in add- already supported similar tools via other means), the ition, toolkit developers face inherent tradeoffs scope of the CTAC project (including the ability of between deployment time, cost, quality, and flexi- the tool to be supported by a distance-based coach), bility [22]. York and colleagues propose a taxonomy and whether the tool was duplicative thematically of of three different approaches to toolkit develop- other tools already selected for inclusion. As a result, ment: a “gold standard” approach where candidate tools that were ultimately included were not uni- tools undergo careful evaluation to ensure that they formly distributed across the various evidence-based improve performance; a “grassroots” approach that strategies for care coordination. Nonetheless, we feel involves the toolkit creator collecting already-created that sufficient variety exists in the toolkit to meet the tools, which then undergo a small amount of vetting; needs of potential end users in the context of other and a “wiki” approach where tools may be posted toolkits available within the VA. Second, we had online without restriction [23]. These different limited resources to find tools (causing us to use a approaches impose various types of burdens on dif- snowball search strategy) and to interview prospec- ferent stakeholders: the gold standard approach may tive users about their appropriateness; a more sys- take a long time to develop and make available to tematic tool identification strategy and a broader users, and result in limited output, but should result group of user interviews might have resulted in a in very user-friendly tools that are likely to enhance different set of tools being selected. However, our care; in contrast, the wiki approach is an inexpensive approach may be in line with typical resource con- dissemination method but imposes a greater bur- straints faced by organizational improvement teams. den on prospective users to sort through the avail- Third, the deliberate attempt to choose tools rele- able options. The grassroots approach, although vant to the VA context and vet them with VA users relying on existing materials, is still labor-intensive, means that some tools (e.g. VA formulary tool) may as it requires a thorough review (and sometimes out- not be appropriate for use outside the VA. However, reach) to collect a comprehensive range of tools. our structured selection process can be used in other Our approach was similar to the grassroots organizations. Additionally, because most tools in approach chosen by York and colleagues for their page 500 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH toolkits, with respect to the harvesting of existing If successful, the CTAC project will improve care tools; however, we differed from the grassroots coordination in VA primary care clinics, and will approach in that we used a more extensive review also provide readily applicable methods for spread- process to ensure saliency of included tools to our ing improvements throughout the VA. In addition, users, reduce redundancy, and ensure a minimum the project will inform VA policymakers regarding level of quality; in addition, we modified tools to what other implementation strategies, including the improve their look and feel prior to posting them. use of distance coaching, may influence the use of As a result, we ended up with a smaller number toolkits. of tools than has been characteristic of prior VA “grassroots” toolkits [23], which may potentially SUPPLEMENTARY MATERIAL be compensated for by making it easier for pro- Supplementary material is available at Translational spective users to find and decide upon their Behavioral Medicine online. preferred tool. Work by DeWalt and colleagues, who carried out Acknowledgments: We thank Michael Ong and Adriana Izquierdo for their help in reviewing the tools as expert clinician reviewers. We also thank a 2-year toolkit development and testing process for Tonya Reznor for programming assistance to deploy the toolkit online, John the AHRQ Health Literacy Universal Precautions Øvretveit for helpful comments on a previous version of this manuscript, Toolkit, is perhaps the closest to the gold standard and individuals who participated in semi-structured interviews for their input. This material is based upon work supported by the Department of articulated above, with draft tools tested in clinical Veterans Affairs, Quality Enhancement Research Initiative through a grant to practice [24] and subsequent practice-based evalua- the Care Coordination QUERI Program (QUE 15-276). The views expressed tions of individual tools and the toolkit collectively in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States [25–27]. Interestingly, the overall demonstration of government. the toolkit in clinical practice recommended that technical assistance be provided for tool imple- Compliance with ethical standards mentation [25]; CTAC intends to test the value of providing such technical assistance for the care Conflict of interest: The authors declare that they have no conflicts of interest. coordination toolkit, in the form of a distance-based coach for half of the project sites, while the other half have access to the toolkit only. Although we did References not have the time to test tools in clinical practice dur- ing the toolkit development phase, we plan to track which tools are adopted (versus never attempted) by 1. Berwick DM, Hackbarth AD. Eliminating waste in US health care. Jama. 2012;307(14):1513–1516. each of the participating sites, and among adopted 2. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and tools, which tools are successfully implemented and cost. Health Aff (Millwood) . 2008;27(3):759–769. 3. Gellad WF. The veterans choice act and dual health system use. J Gen maintained (vs. being abandoned), with follow-up Intern Med. 2016;31(2):153–154. key informant interviews exploring reasons for suc- 4. Yoon J, Scott JY, Phibbs CS, Wagner TH. Recent trends in vet- cess in implementation or barriers. In addition, we erans affairs chronic condition spending. Popul Health Manag. 2011;14(6):293–298. plan to incorporate users’ feedback to refine the 5. Powell Davies G, Williams AM, Larsen K, Perkins D, Roland M, Harris MF. tools in the toolkit as they are used, and add new Coordinating primary health care: an analysis of the outcomes of a sys- tematic review. Med J Aust. 2008;188(8 Suppl): S65–S68. tools when appropriate. 6. Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care– We also intend to evaluate the degree to which the causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40. toolkit matches the needs of prospective users, the 7. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of imple- degree to which users’ readiness to implement a tool mentation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10:21. affects subsequent success in implementation, and 8. Gale RC, Asch SM, Taylor T, et al. The most used and most helpful facil- the extent to which the tools affect patient experi- itators for patient-centered medical home implementation. Implement ence of care. We attempted to include tools that Sci. 2015;10:52. 9. McDonald KM, Schultz E, Albin L, et al. Care Coordination Measures Atlas have varying implementation difficulty, that have Version 4. Rockville, MD: Agency for Healthcare Research and Quality; broad applicability, and that have not yet been uni- 10. AHRQ Publishing and Communications Guidelines, Section 6: Toolkit formly adopted in the VA based on our semi-struc- Guidance. http://www.ahrq.gov/research/publications/pubcomguide/ tured interviews. However, it remains to be seen pcguide6.html Accessibility verified January 4, 2018. 11. Bagalman E. The number of veterans that use VA health care services: a whether the tools will successfully align with clinics’ fact sheet. Washington, DC: Congressional Research Service; 2014. priorities, and there may be a tradeoff in terms of 12. Chang ET, Wang M, Kirsh S, Rubenstein LV. VA High-Risk Populations in the difficulty of the tools being implemented and Primary Care. Mineapolis, MN: Accepted for presentation at Academy Health Annual Research Meeting; 2015. the potential beneficial effect on patient experience 13. Rosland AM, Nelson K, Sun H, et al. The patient-centered medi- of care, with more difficult tools potentially provid- cal home in the Veterans Health Administration. Am J Manag Care. 2013;19(7):e263–272. ing more benefit, but also requiring more resources. 14. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred We realized from the outset that the toolkit can be reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med. 2009;6(7):e1000097. an adjunct to, but not a replacement for, a serious 15. Luck J, Bowman C, York L, et al. Multimethod evaluation of the VA’s peer- commitment on the part of clinic leadership and to-peer Toolkit for patient-centered medical home implementation. J Gen Intern Med. 2014;29(Suppl 2):S572–S578. staff to improve care coordination. TBM page 501 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH 16. Gale RC, Luck J, York L, Asch S. Peer-to-peer toolkits enhance sharing of Design and Control of Workflow Processes: Business Process 22. Reijers HA. best practices across an integrated delivery system. J Patient Cent Res Management for the Service Industry. Chapter 6: Heuristic Workflow Rev. 2016;3:189. Redesign, vol. 2617. Berlin: Springer; 2003. 17. Ganz DA, Huang C, Saliba D, Shier V, Berlowitz D, Lukas CV, Pelczarski 23. York L , Bruce B, Luck J, et al. Online toolkits for metric-driven K, Schoelles K, Wallace LC, Neumann P. Preventing Falls in Hospitals: quality improvement: the veterans health administration A Toolkit for Improving Quality of Care [online]. In. Rockville, MD: Agency Jt Comm J Qual Patient Saf. managed grassroots approach. for Healthcare Research and Quality; 2013. 2013;39(12):561–569. 18. Engaging Patients in Improving Ambulatory Care: A Compendium of 24. DeWalt DA, Broucksou KA, Hawk V, et al. Developing and test- Tools from Maine, Oregon, and Humboldt County, California. https:// Nurs Outlook. ing the health literacy universal precautions toolkit. www.rwjf.org /content/dam/farm/toolkits/toolkits/2013/rwjf404402 2011;59(2):85–94. Accessibility verified January 4, 2018. 25. Mabachi NM, Cifuentes M, Barnard J, et al. Demonstration of the health 19. Carmichael J, Jassar G, Nguyen PA. Healthcare metrics: Where do pharma- literacy universal precautions toolkit: lessons for quality improvement. Am J Health Syst Pharm. 2016;73(19):1537–1547. Ambul Care Manage. 2016;39(3):199–208. cists add value? 20. Parchman ML, Noël PH, Lee S. Primary care attributes, 26. Weiss BD, Brega AG, LeBlanc WG, et al. Improving the effectiveness of health care system hassles, and chronic illness. Med Care. medication review: guidance from the health literacy universal precau- 2005;43(11):1123–1129. J Am Board Fam Med. 2016;29(1):18–23. tions toolkit. 21. Tsan MF, Puglisi T. Health care operations activities that may consti- 27. Brega AG, Freedman MA, LeBlanc WG, et al. Using the health literacy Irb. universal precautions toolkit to improve the quality of patient materials. tute research: the department of veterans affairs’s perspective. 2014;36(1):9–11. J Health Commun. 2015;20(Suppl 2):69–76. page 502 of 502 TBM http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Translational Behavioral Medicine Oxford University Press

Loading next page...
1
 
/lp/ou_press/development-of-a-web-based-toolkit-to-support-improvement-of-care-4DvTmY8W18

References (27)

Copyright
Copyright © 2022 Society of Behavioural Medicine
ISSN
1869-6716
eISSN
1613-9860
DOI
10.1093/tbm/ibx072
Publisher site
See Article on Publisher Site

Abstract

Implications of Healthcare Innovation, Promising practices for the coordination of chronic care exist, Practice: To narrow down the very large num- Implementation, and Policy but how to select and share these practices to support quality ber of available care coordination tools to those (CSHIIP), VA Greater Los Angeles improvement within a healthcare system is uncertain. This study Healthcare System, Sepulveda, relevant to local improvement goals and context, California, USA describes an approach for selecting high-quality tools for an healthcare leaders and managers can adapt and David Geffen School of Medicine, online care coordination toolkit to be used in Veterans Health apply the tool selection process we outline in this University of California at Los Administration (VA) primary care practices. We evaluated tools article. Angeles, Los Angeles, CA, USA in three steps: (1) an initial screening to identify tools relevant to RAND Health, Santa Monica, CA, care coordination in VA primary care, (2) a two-clinician expert USA review process assessing tool characteristics (e.g. frequency Policy: A structured approach to toolkit develop- School of Biomedical Informatics, of problem addressed, linkage to patients’ experience of care, University of Texas Health Science ment can facilitate the spread of care innovations effect on practice workflow, and sustainability with existing Center at Houston, Houston, TX, for quality improvement efforts, but toolkits are resources) and assigning each tool a summary rating, and (3) USA an adjunct to, not a replacement for, a serious 5 semi-structured interviews with VA patients and frontline clini- Department of Psychiatry and commitment on the part of clinic leadership and cians and staff. Of 300 potentially relevant tools identified by Biobehavioral Sciences, University staff to improve care coordination. of California at Los Angeles, Los searching online resources, 65, 38, and 18 remained after steps Angeles, CA, USA one, two and three, respectively. The 18 tools cover five topics: Veterans Evidence-based managing referrals to specialty care, medication management, Research Dissemination and patient after-visit summary, patient activation materials, agenda Research: To ensure that toolkits are effective Implementation Center (VERDICT), setting, patient pre-visit packet, and provider contact information in achieving their desired ends, additional work South Texas VA Health Care for patients. The final toolkit provides access to the 18 tools, as should characterize the development processes System, San Antonio, TX, USA 7 well as detailed information about tools’ expected benefits, and and design features of different toolkits, including University of Texas Health resources required for tool implementation. Future care coordin- the relationship between these processes and fea- Science Center at San Antonio, San ation efforts can benefit from systematically reviewing available tures and subsequent tool adoption, implementa- Antonio, TX, USA tools to identify those that are high quality and relevant. tion, and sustainability. HSR&D Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System, Menlo Park, CA, USA Keywords Division of Primary Care and coordination strategies in the published research lit- Care coordination, Primary care, Quality improve- Population Health, Stanford erature [5], effective methods to identify, document, University, Stanford, CA, USA ment, Toolkit, Distance coaching spread, and sustain evidence-based care coordin- Correspondence to: David ation approaches in routine care are not clear. The A Ganz, David.Ganz@va.gov Coordination Toolkit and Coaching (CTAC) project INTRODUCTION aims to improve patients’ experience of care coord- Cite this as: TBM 2018;8:492–502 Inefficiencies in coordination of chronic care are a doi: 10.1093/tbm/ibx072 ination within VA primary care and between VA significant source of waste in the United States (USD primary care and other outpatient settings, including $25–$45 billion in 2011) [1] and a barrier to achiev- Published by Oxford University Press health care provided in the community, while also on behalf of the Society of Behavioral ing the Triple Aim of improved patient experience Medicine 2018. This work is written by developing better methods for bringing research evi- of care, lower per capita costs, and better population (a) US Government employees(s) and dence on care coordination into routine care. is in the public domain in the US. health [2]. In the Veterans Health Administration To implement research on care coordination in (VA), recent legislation allowing some patients to practice, any coordination strategy must not only be seek care from community medical providers (i.e. evidence-based, but also applicable to the context of outside of VA owned and operated facilities) has fur- the adopting organization. The gap between research ther complicated care coordination in the primary and practice exists in part because research results care practices of a system where patients already for complex problems such as care coordination are have higher rates of comorbidity and mental illness often presented generically, without the supporting than private sector counterparts [3, 4]. Although documentation of the tools required to achieve such a systematic review has shown the benefits of care page 492 to 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH results. Additionally, research results often do not Organizational context apply when conducted under the varied resource The VA is a national integrated healthcare delivery constraints, patient populations, and organizational system serving 5.9 million Veteran patients annu- milieus that are found in practice [6]. These con- ally [11], with a central headquarters in Washington, straints may strongly shape the specific intervention DC and 18 healthcare networks known as Veterans elements that can be applied successfully within any Integrated Service Networks (VISNs); each VISN specific care coordination improvement effort. For includes multiple healthcare “facilities” that are sys- example, though the VA’s shared information technol- tems in themselves, offering inpatient and outpatient ogy infrastructure greatly facilitates internal sharing of care, often across multiple sites. In the VA, most innovations, it also means that innovations in research patients are assigned to a primary care team [12]. studies or available in general repositories such as Since 2010, the VA has focused on transforming its the Agency for Healthcare Research and Quality primary care teams into a patient-centered medi- (AHRQ) Innovations Exchange may not be applic- cal home model with augmented staffing (one reg- able to a local VA initiative because of incompatibility istered nurse care manager, one licensed practical with VA’s electronic health record. In addition, indi- nurse, and one administrative clerk for each primary viduals delivering clinical care may have limited time care provider), called patient-aligned care teams to search for tools, make a comparison between avail- (PACT), and PACTs are tasked with coordinating able options, and decide which tools to adopt. Thus, the patient’s care [13]. VA’s status as an integrated one way to facilitate the uptake of research in prac- healthcare delivery system offers the benefits of a tice is to compile a toolkit—in essence, a “short list” of shared electronic health record and communication action-oriented strategies for improving care coordin- systems, including electronic mail, for VA-owned ation that is compatible with the local context [7]. and operated facilities and selected contractors. In this article, we describe CTAC’s first phase, which involved selecting tools for a toolkit and Toolkit development developing a VA Intranet site to support the tools. The decision of which tools to include in the toolkit We aimed to evaluate candidate tools based on involved three steps: (1) an initial review of content the existing evidence base for effective care coord- to determine if tools met inclusion or exclusion crite- ination strategies, relevance to improving patients’ ria, (2) a two-clinician review process to rate tools on experience of care coordination, tool feasibility and their characteristics, and (3) semi-structured inter- quality, and evaluations by VA patients and frontline views with VA patients and frontline clinicians and VA primary care providers and staff. Within the VA, staff. Figure 1, adapted from the PRISMA statement toolkits are often used in conjunction with other dis- [14], shows the flow of tools through this three-step semination methods, such as learning collaboratives process. [8] and quality improvement coaching, and a later In the first step, the principal investigator and phase of CTAC will compare the additional value project manager identified tools using a snowball of distance-based coaching to implement the tools, approach, starting with two VA toolkits (focused on with access to the toolkit without coaching. PACT and specialty care) [15, 16] and two nonVA toolkits [17, 18]. The focus of this initial review METHODS was to determine the candidate tools’ general rel- evance to the project topic. We excluded tools for Definitions any of the following reasons: tools were restricted Care coordination is defined as “…the deliberate to a particular clinical domain, such as being con- organization of patient care activities between two dition-specific (e.g. only for diabetes) or covering or more participants (including the patient) involved only clinical preventive services; were intended in a patient’s care to facilitate the appropriate deliv- for use primarily in a setting outside of primary ery of health care services. Organizing care involves care (e.g. specialty care, inpatient care); were not the marshalling of personnel and other resources focused on care coordination (e.g. only access needed to carry out all required patient care activ- focused, only continuity focused, or only general ities, and is often managed by the exchange of quality improvement); were not truly a tool (e.g. information among participants responsible for general information sheets that were not action-ori- different aspects of care” [9]. A  toolkit is defined ented); the VA was already disseminating the tool as “a collection of related information, resources, or strategy widely (e.g. tools to encourage uptake of or tools that together can guide users to develop a secure messaging between patients and providers); plan or organize efforts to follow evidence-based the tool was not applicable to the VA setting; the recommendations or meet evidence-based specific tool was part of a toolkit with interdependent tools practice standards,” and a tool as “an instrument that could not stand alone as individual tools; or (e.g. survey, guidelines, or checklist) that helps users CTAC’s distance-based coach would not be able accomplish a specific task that contributes to meet- to help primary care teams with using the tool. ing a specific evidence-based recommendation or The principal investigator, project manager, and practice standard” [10]. TBM page 493 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Fig 1 | Three-step tool identification, screening and eligibility determination process. Preliminary exclusion criteria included a tool being: (1) condition-specific, (2) covering only clinical preventive services, (3) intended for use outside of primary care, (4) not focused on care coordination, (5) not truly a tool, (6) VA already using the tool or strategy widely, (7) not applicable to the VA setting, (8) part of a toolkit and could not stand alone as an individual tool, or (9) not coachable by CTAC’s distance-based coach. Exclusions based on the tool-rating checklist (Supplementary Appendix) and reviewer discussion and consensus. Excluded tools classified as supporting materials or add- itional resources for the included tools. two additional project staff then sorted remaining discussed these differences, they had the option of tools meeting inclusion criteria into categories and changing their ratings if desired. sub-categories to organize them according to the In the third step, the principal investigator (inter- topics they covered. viewer) and project manager (note-taker) conducted In the second step, two clinicians independently semi-structured interviews with VA providers, clinic reviewed remaining tools, using a newly developed staff, and patients in order to obtain feedback about and piloted tool-rating checklist (Supplementary the tools and learn more about how the tools might Appendix). During the second step, the principal function in a clinic setting. A  total of eight inter- investigator served as the first reviewer of all the views were completed on a broad selection of the tools, with the role of second reviewer split between included tools, five of them individual interviews five clinicians, who were assigned roughly equal with PACT clinic staff from one VA site (primary numbers of tools to review. After the reviews were care provider, registered nurse, lead clinic clerk, completed, the first and second reviewers met to dis- psychologist and pharmacist) and three of them cuss substantial differences in ratings (defined as a group interviews from another site (two Veterans two-point difference on any 3-point rating of a given receiving health care services at the VA, members dimension of the tool, or at least a 3-point difference of the 21-member VISN 22 clinician Community on the 7-point tool summary rating). After reviewers of Practice, and six members of the nine-member page 494 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH VISN 22 Nursing Workgroup). Interviews lasted the pathways by which the selected tools might 1 hr each, and were all conducted by phone except affect dimensions of the VA’s Strategic Analytics for the in-person group interview of the two Veteran for Improvement and Learning (SAIL) domains, patients. A  pre-determined selection of tools, spe- which are an internal dashboard used by VA cifically chosen for each interviewee based on his/ leaders to benchmark VA facilities’ performance her role at the VA, was then presented, via email [19]. As the CTAC project focuses on improv- attachment, an online screen-sharing program, or in ing patients’ experience of care coordination, we person, as applicable. Interviewees provided feed- determined in more detail the mapping of tools back about the tools with interviewer prompting to the Health System Hassles Scale [20], which via questions such as “Do you have a similar tool assesses patients’ challenges using the health- in use  at your facility?”, “Would a tool like this be care system, such as getting medications refilled helpful at your facility?”, and “How would this tool on time, lack of information about why patients be used at your facility?” The group interviews of were referred to a specialist, poor communication VA providers and staff involved a specific clinical between different doctors or clinics, and disagree- scenario and set of tools that were presented to the ments between doctors about the patient’s diag- group, with the group then providing feedback. nosis or treatment, and is included in the AHRQ After these three steps were completed, final Care Coordination Measures Atlas [9]. assignment of tools to categories occurred, and Ethics approval tools were deployed to an online website access- ible via the VA Intranet, using SharePoint®. Using The CTAC project was determined to be nonre- rapid cycles of development, a SharePoint pro- search by the VA Office of Primary Care Services grammer worked with the principal investigator (in accordance with VHA Handbook 1058.05) [21] and project manager to develop the toolkit web- and by the VA Greater Los Angeles Healthcare site’s functionality, usability, and look and feel. System Institutional Review Board. An internal SharePoint site was chosen because of rapidity of deployment and the ease with which RESULTS tools could be added or changed in the future. We Review of 22 toolkits, manuals and resources (the placed special emphasis on developing “tool access original four toolkits plus 18 found through a snow- pages” (Supplementary Appendix), which provide ball search) resulted in identification of 300 poten- prospective users with a preview of the amount of tial tools from these sources (Figure 1). Application time and effort required to deploy the tool, and the of initial inclusion and exclusion criteria narrowed expected benefits of the tool. the list of potential tools to 65. The two-clinician review process and subsequent project team dis- Evaluation of toolkit development process cussion further reduced the number of candidate To evaluate the process of tool selection and toolkit tools to 38. These 38 tools had higher clinician rat- development, we conducted a review of documenta- ings than excluded tools; however, the difference in tion collected during the process, including spread- summary ratings between included and excluded sheets of included and excluded tools, reviewer tools was only statistically significantly different ratings of tools, and field notes from semi-structured for the first reviewer, who reviewed all the tools interviews. We carried out quantitative analyses of himself (reviewer 1: median ± interquartile range, reconciled clinician reviewer scores from the sec- 5 ± 1 for included tools vs. 4 ± 1 for excluded tools, ond step of the tool selection process, calculating p  =  .0005). For the five different clinicians who median and interquartile ranges for tool summary shared the role of second reviewer, the collective ratings, which were on a 1–7 integer scale (a higher rating of included and excluded tools showed more number being more favorable). For inferential sta- variation around median values (reviewer 2: median tistics, we used the Wilcoxon signed-rank test to ± interquartile range, 5 ± 3 for included tools versus assess reviewer agreement on tool ratings, and the 4 ± 2 for excluded tools, p = .096). There was no sig- Wilcoxon rank-sum test for ratings of included ver- nificant difference in summary ratings between the sus excluded tools. Two-tailed p values were consid- two reviewers of each tool (p = .915). ered significant at p < .05. Semi-structured interviews highlighted factors that We also evaluated the degree to which the final might suggest a tool should be included or excluded. set of tools selected for inclusion conformed to For example, one set of tools presented during an evidence-based strategies, and the dimensions interview was focused on motivational interviewing of health care quality that the tools might affect. techniques and supporting curricula and resources. To determine the degree to which tools covered The discussion during the interview revealed that the range of evidence-based strategies for care training in this area and supplemental materials are coordination, we mapped the tools to a compil- already available to VA clinic staff. Other interviews ation of care coordination strategies shown effect- revealed specific challenges a facility might face ive in a systematic review [5]. We also mapped using a tool. For instance, when presented with an TBM page 495 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 1 | Care coordination toolkit tool list with descriptions Tool category and tool name Tool description Managing Referrals to Specialty Care VA Service Agreement Template A template with the purpose of facilitating timely access and patient-centered care for patients by promoting an effective relationship between the PACT team and the spe- cialty care team during the management of a patient’s care. Consult Guides for Primary Care A comprehensive list of clinical conditions and for each, a standard set of information to (Information to Include) be included in a referral request, to help ensure a meaningful visit when the referred patient visits the relevant specialist. While the tool consists of multiple data sets, not all need to be implemented. Relaxation and Meditation Program: Information and tools for implementing a program that teaches patients relaxation and An approach to self-management of meditation skills, with the potential benefit of decreasing the number of referrals to mental health in primary care specialty mental health services. Medication Management VA Formulary (Abbreviated) The Primary Care Service in the VA Palo Alto Healthcare System worked with its pharmacy department to create an abbreviated and searchable VA formulary. This resource was posted on the healthcare system’s external web page and is updated periodically for accuracy. AudioRENEWAL: Phone-based Allows patients to request a renewal of their prescription directly from within the Medication Renewal for Patients AudioCARE telephone refill system 24 hr a day, 7 days a week. Communication with Community A two-page document that includes a letter to help facilitate communication Providers to Co-Manage Veterans’ between a VA provider and an outside healthcare provider regarding management Care of the Veteran patient and instructions for obtaining medications through the VA, and a guide for providers (nonVA) to request a prescription medication that is nonformulary. Medication Tracker for Patients A one-page form with sections to be filled in by a member of the PACT team for the patient. Spaces are provided to fill in a patient’s medication details and healthcare provider name and contact information. The medication list includes details that can help patients remember when to take their medications and what dose to take. Patient After-Visit Summary After-Visit Instructions for patients A customizable document instructing patients about where to go before leaving (paper) the VA the day of their appointment, as well as any follow-up actions which need to be taken. Includes contact information for specialty clinics as well as a map of the campus, which can be used to direct patients to their next destination on campus. After-Visit Summary for patients An electronic tool that produces a customizable, printable patient summary that can be (electronic) provided to a patient after his/her primary care office visit to summarize visit content and subsequent action steps, if appropriate. Patient Activation Materials Patient Agenda-Setting Form A simple, easy to fill-out form to help patients prepare for their healthcare visit. The form can be sent to patients in the mail with a pre-visit packet, or can be given to patients to complete in the waiting room. Patient Treatment Decision Guide A worksheet for patients to use when faced with a medical decision. The form includes questions to ask the provider during the medical appointment, space to take notes, and guidance on how to proceed with making a decision. Tips for Patients: Improving A two-page handout that can be mailed to patients in a pre-visit packet or Communication with your Primary given to patients at check-in, before their appointment. The tool includes tips Care Team for communicating with the primary care team, questions for patients to think about before their appointment and space to write down answers. This tool can help ensure all of a patient’s questions and concerns are addressed during their medical appointment. Pharmacy Safety for Patients A patient handout that explains the role of the pharmacist, pharmacy and patient during the process of obtaining medications. Tips for Patients: Questions to Ask A guide that includes tips, ideas, and questions for patients to use before, during, and Before, During and After your Visit after their appointment that will help facilitate a successful healthcare visit. Provider Contact Information for Patients Clinic Information Pamphlet A template that clinics can customize to create a clinic information pamphlet for patients that includes pertinent information about the clinic. Information may include appoint- ment-making instructions, provider contact information, prescription refill instructions, and other clinic details. Continued page 496 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 1 | Continued Tool category and tool name Tool description My Primary Care Team: Contact and A one-page handout that lists important provider contact information for a patient. Other Information Includes contact details for the primary care provider, PACT nurse, tele-health nurse, specialty care, pharmacy, and mental health. My Primary Care Team: Wallet Card A customizable wallet-sized card that can be printed, populated with a patient’s primary with Contact Information care team information and given to the patient. The back of the card includes a list of information patients should provide when leaving a message at the clinic. Save a Trip to Primary Care A one-page resource for patients, explaining and illustrating the difference between routine, nonurgent, and urgent medical situations. The sheet explains to patients what actions to take in the case of each of these situations, and provides relevant contact information. agenda-setting tool that patients receive at check-in require substantial adaptation of the tool for use in and fill out in the waiting room, one interviewee the clinic, or the involvement or procurement of described check-in kiosks used at the clinic, and new resources such as additional staff, new equip- explained that Veterans do not typically see a clerk ment, or coordination with information technology if they check in via the kiosk. Therefore, having a specialists. An example of an easily implemented clerk hand a paper-based agenda-setting form to the tool is “My Primary Care Team – Wallet Card with patient at check-in would not be a viable approach Contact Information,” a wallet-sized card the clinic in this clinic setting. Based on the feedback from the prepares in advance and has available for patients semi-structured interviews, we were able to reduce when they visit the clinic. The wallet card provides the 38 tools at this stage to the final set of 18 (origi- provider names and contact information along with nally 19, with two tools combined into one; Table 1), the procedure patients should use when leaving a with many of the excluded 19 tools available on the message at the clinic. An example of a challenging toolkit website as additional resources or supporting tool to implement is the VA Care Coordination materials for the included tools. Service Agreement Template, a document that can The final set of 18 tools available in the toolkit be used to codify how a primary care team and a (available on the VA Intranet, see Supplementary specialty care team will coordinate care of shared Appendix for screenshots) covers five categories: patients, and which would require substantial nego- managing referrals to specialty care, medication man- tiation and ongoing meetings between these two agement, patient after-visit summary, patient activa- teams to be filled out meaningfully. tion materials, and provider contact information for Mapping the tools to available evidence-based patients. The tools in the “managing referrals to spe- care coordination strategies [5] showed good, cialty care” category support the appropriate and although not uniform, coverage (Table  2), with efficient use of specialty care and better communica- clear emphasis on providing support for patients tion between primary care and specialty care teams. (13 tools), followed by structured arrangements for The “medication management” category tools can coordinating service provision between providers help improve patients’ understanding of how to take (2 tools), providing support for service providers their medications or help patients obtain access to (2 tools), and structuring the relationships between needed medications more efficiently. The “patient service providers and with patients (1 tool). When after-visit summary” category includes tools that accounting for the additional resources available on offer a convenient way to document for patients the the toolkit website, coverage was improved, with the decisions made during their primary care visit and strategy of using systems to support care coordin- what follow-up is needed. Tools in the category of ation also included. One strategy (improving com- “patient activation materials” help prepare patients munication between service providers) was not for their visit with their primary care team, and are covered by the tools or additional resources. intended to be sent to (or shared with) the patient Table  3 shows dimensions of patient experience in advance of a visit. Finally, the “provider contact that might be favorably affected by use of the tools if information for patients” category tools help to pro- the tools were implemented appropriately. The three vide patients with important contact information, so problems addressed by the most tools were “diffi- they know whom to call for both routine and urgent culty getting questions answered or getting medi- health issues. cal advice between scheduled appointments” (six The 18 tools vary in the level of resources (e.g. tools), “lack of information about your medical con- staff time, information technology support) required ditions” (five tools), and “lack of information about to implement them. Simpler tools can be used “as why your medications have been prescribed to you” is” or with minimal adaptation, and may be imple- (five tools). Mapping tools to SAIL domains showed mented and sustained with existing clinic resources. that the primary domains affected were patient satis- Tools that may be more difficult to implement may faction and specialty care access (Figure 2). TBM page 497 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 2 | Tools and resources in toolkit, mapped to care coordination strategies in Powell Davies et al. [5] Tool category and tool name Care coordination strategy Managing Referrals to Specialty Care VA care coordination service agreement template Structured arrangements for coordinating service provi- sion between providers Consult guides for primary care (information to include) Structured arrangements for coordinating service provi- sion between providers Relaxation and meditation program: an approach to Structuring the relationships between service providers self-management of mental health in primary care and with patients Medication Management VA formulary (abbreviated) Providing support for service providers AudioRENEWAL: Phone-based medication renewal for patients Providing support for patients Communication with community providers to co-manage Providing support for service providers Veterans’ care Medication tracker for patients Providing support for patients Patient After-Visit Summary After-visit instructions for patients (paper) Providing support for patients After-visit summary for patients (electronic) Providing support for patients Patient Activation Materials Patient agenda-setting form Providing support for patients Patient treatment decision guide Providing support for patients Tips for patients: improving communication with your primary care team Providing support for patients Pharmacy safety Providing support for patients Tips for patients: Questions to ask before, during and after your visit Providing support for patients Provider Contact Information for Patients Clinic information pamphlet Providing support for patients My primary care team—contact and other information Providing support for patients My primary care team—wallet card with contact information Providing support for patients Save a trip to primary care Providing support for patients Additional Resources Enhancing communication with patients* Providing support for service providers Care management* Using systems to support care coordination *These are categories of additional resources, with multiple resources within each category. DISCUSSION disseminated in the VA. These aspects of our tool selection process show how the approach adapts We have reported on the development process to organizational context; in another context, our for an online toolkit to support better care coord- structured process could be tailored to different ination in primary care. We were successful in opportunities for improvement. narrowing down a list of 300 potential tools to Figure 2, adapted from the Chronic Care Model 18 that met our inclusion criteria and were ulti- [13], depicts the pathways by which tools and add- mately made available through our toolkit web- itional resources are ultimately expected to affect site. Our two-clinician review process was helpful aspects of care quality (patient satisfaction and spe- in deciding on a penultimate list of tools to be cialty care access) at the VA. Most tool categories reviewed more carefully in semi-structured inter- work to connect the patient to the health system dir- views. Our methods, particularly the tool review ectly, through increasing accessibility, empowering form, may be useful to others who are interested the patient, improving adherence/self-management, in a structured and systematic approach to toolkit or increasing understanding. One tool category, development. “managing referrals to specialty care,” operates Most of the tools that were ultimately included within the health system, but indirectly impacts pro- in the toolkit related to the coordination strategy ductive interactions by improving communication of providing support to patients, reflecting the and reducing inefficiencies that impact patient care. project’s emphasis on improving patient care expe- The tool review process we carried out is subject riences. Our process also aimed to include tools to several limitations. First, we attempted to use a that were complementary to existing VA resources, quantitative tool review process to determine which rather than duplicative of them. For example, many tools to subject to further testing; this review pro- tools and policies related to how primary care teams cess included explicitly rating certain tool features should internally coordinate their care were already page 498 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Table 3 | Tools in the Care Coordination Toolkit and associated item(s) on the Health System Hassles Scale Health System Hassles Scale items Tool category and tool name 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Managing Referrals to Specialty Care VA Care Coordination Service X X X Agreement Template Consult Guides for Primary Care X X X X X X (Information to Include) Relaxation and Meditation Program: X An approach to self-management of mental health in primary care Medication Management VA Formulary (Abbreviated) X AudioRENEWAL: Phone-based X X Medication Renewal for Patients Communication with Community X X Providers to Co-Manage Veterans’ Care Medication Tracker for Patients X X Patient After-Visit Summary After-Visit Instructions for Patients X X (paper) After Visit Summary for Patients X X X (electronic) Patient Activation Materials Patient Agenda-Setting Form X X X X X X Patient Treatment Decision Guide X X Tips for Patients: Improving X X X X X X Communication with your Primary Care Team Pharmacy Safety for Patients X Tips for Patients: Questions to ask X X X X Before During and After Your Visit Provider Contact Information for Patients Clinic Information Pamphlet X My Primary Care Team - Contact and X Other Information My Primary Care Team - Wallet Card X with Contact Information Save a Trip to Primary Care X X Health System Hassles Scale, response options, and 16 items. During the past 12 months, how much of a problem, if at all, has each of the following been for you? (Response options: A very big problem for you, a big problem for you, a moderate problem for you, a small problem for you, or not a problem at all for you.) 1 Lack of information about your medical conditions. 2 Lack of information about which treatment options are best for your medical condition. 3 Lack of information about why your medications have been prescribed to you. 4 Problems getting your medications refilled on time. 5 Uncertainty about when or how to take your medications. 6 Side effects from your medications. 7 Lack of information about why you have been referred to a specialist. 8 Having to wait a long time to get an appointment for specialty providers or clinics. 9 Poor communication between different healthcare providers. 10 Disagreements between your providers about your diagnoses or the best treatment for you. 11 Lack of information about why you need lab tests or x-rays. 12 Having to wait too long to find out about the results of lab tests or x-rays. 13 Difficulty getting questions answered or getting medical advice between scheduled appointments. 14 Lack of time to discuss all your problems with your health care provider during scheduled appointments. 15 Having your concerns ignored or overlooked by your healthcare providers. 16 Medical appointments that interfere with your work, family, or hobbies. TBM page 499 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH Fig 2 | Pathways from tool use to improved dimensions of care quality in the VA. This diagram depicts interactions between the patient and the health system, including primary care (i.e. the “teamlet” which includes one registered nurse care manager, one licensed practical nurse, and one administrative clerk, and one primary care provider; and “the extended PACT team” which can include the addition of a social worker, psychologist, and/or pharmacist). Orange arrows represent tools supporting interactions between parties; bidirectional arrows represent tools requiring input from both parties, whereas uni-directional arrows represent tools where the one party is primarily responsible. Each specific arrow is accompanied by a numbered label that describes the tool and, in italics, the tool’s expected contribution to the overall goal to “facilitate productive interactions.” There are additional yellow markers denoting the SAIL domains that are expected to be positively affected by the productive interactions supported by the tools. and then providing a summary rating for each tool. the toolkit are patient-facing, we believe that many Although tool ratings did provide useful informa- could be used in nonVA settings as is, or with minor tion, we learned that they could not be used by them- adaptations. selves for an ultimate determination of each tool’s The field of toolkit development has been ham- value. Rather, tools had to be viewed in the context pered by limited evidence about what features of of other features of the project, including the unique toolkit development lead to effective strategies for features of the VA setting (including whether the VA implementing evidence-based innovations; in add- already supported similar tools via other means), the ition, toolkit developers face inherent tradeoffs scope of the CTAC project (including the ability of between deployment time, cost, quality, and flexi- the tool to be supported by a distance-based coach), bility [22]. York and colleagues propose a taxonomy and whether the tool was duplicative thematically of of three different approaches to toolkit develop- other tools already selected for inclusion. As a result, ment: a “gold standard” approach where candidate tools that were ultimately included were not uni- tools undergo careful evaluation to ensure that they formly distributed across the various evidence-based improve performance; a “grassroots” approach that strategies for care coordination. Nonetheless, we feel involves the toolkit creator collecting already-created that sufficient variety exists in the toolkit to meet the tools, which then undergo a small amount of vetting; needs of potential end users in the context of other and a “wiki” approach where tools may be posted toolkits available within the VA. Second, we had online without restriction [23]. These different limited resources to find tools (causing us to use a approaches impose various types of burdens on dif- snowball search strategy) and to interview prospec- ferent stakeholders: the gold standard approach may tive users about their appropriateness; a more sys- take a long time to develop and make available to tematic tool identification strategy and a broader users, and result in limited output, but should result group of user interviews might have resulted in a in very user-friendly tools that are likely to enhance different set of tools being selected. However, our care; in contrast, the wiki approach is an inexpensive approach may be in line with typical resource con- dissemination method but imposes a greater bur- straints faced by organizational improvement teams. den on prospective users to sort through the avail- Third, the deliberate attempt to choose tools rele- able options. The grassroots approach, although vant to the VA context and vet them with VA users relying on existing materials, is still labor-intensive, means that some tools (e.g. VA formulary tool) may as it requires a thorough review (and sometimes out- not be appropriate for use outside the VA. However, reach) to collect a comprehensive range of tools. our structured selection process can be used in other Our approach was similar to the grassroots organizations. Additionally, because most tools in approach chosen by York and colleagues for their page 500 of 502 TBM Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH toolkits, with respect to the harvesting of existing If successful, the CTAC project will improve care tools; however, we differed from the grassroots coordination in VA primary care clinics, and will approach in that we used a more extensive review also provide readily applicable methods for spread- process to ensure saliency of included tools to our ing improvements throughout the VA. In addition, users, reduce redundancy, and ensure a minimum the project will inform VA policymakers regarding level of quality; in addition, we modified tools to what other implementation strategies, including the improve their look and feel prior to posting them. use of distance coaching, may influence the use of As a result, we ended up with a smaller number toolkits. of tools than has been characteristic of prior VA “grassroots” toolkits [23], which may potentially SUPPLEMENTARY MATERIAL be compensated for by making it easier for pro- Supplementary material is available at Translational spective users to find and decide upon their Behavioral Medicine online. preferred tool. Work by DeWalt and colleagues, who carried out Acknowledgments: We thank Michael Ong and Adriana Izquierdo for their help in reviewing the tools as expert clinician reviewers. We also thank a 2-year toolkit development and testing process for Tonya Reznor for programming assistance to deploy the toolkit online, John the AHRQ Health Literacy Universal Precautions Øvretveit for helpful comments on a previous version of this manuscript, Toolkit, is perhaps the closest to the gold standard and individuals who participated in semi-structured interviews for their input. This material is based upon work supported by the Department of articulated above, with draft tools tested in clinical Veterans Affairs, Quality Enhancement Research Initiative through a grant to practice [24] and subsequent practice-based evalua- the Care Coordination QUERI Program (QUE 15-276). The views expressed tions of individual tools and the toolkit collectively in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States [25–27]. Interestingly, the overall demonstration of government. the toolkit in clinical practice recommended that technical assistance be provided for tool imple- Compliance with ethical standards mentation [25]; CTAC intends to test the value of providing such technical assistance for the care Conflict of interest: The authors declare that they have no conflicts of interest. coordination toolkit, in the form of a distance-based coach for half of the project sites, while the other half have access to the toolkit only. Although we did References not have the time to test tools in clinical practice dur- ing the toolkit development phase, we plan to track which tools are adopted (versus never attempted) by 1. Berwick DM, Hackbarth AD. Eliminating waste in US health care. Jama. 2012;307(14):1513–1516. each of the participating sites, and among adopted 2. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and tools, which tools are successfully implemented and cost. Health Aff (Millwood) . 2008;27(3):759–769. 3. Gellad WF. The veterans choice act and dual health system use. J Gen maintained (vs. being abandoned), with follow-up Intern Med. 2016;31(2):153–154. key informant interviews exploring reasons for suc- 4. Yoon J, Scott JY, Phibbs CS, Wagner TH. Recent trends in vet- cess in implementation or barriers. In addition, we erans affairs chronic condition spending. Popul Health Manag. 2011;14(6):293–298. plan to incorporate users’ feedback to refine the 5. Powell Davies G, Williams AM, Larsen K, Perkins D, Roland M, Harris MF. tools in the toolkit as they are used, and add new Coordinating primary health care: an analysis of the outcomes of a sys- tematic review. Med J Aust. 2008;188(8 Suppl): S65–S68. tools when appropriate. 6. Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care– We also intend to evaluate the degree to which the causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40. toolkit matches the needs of prospective users, the 7. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of imple- degree to which users’ readiness to implement a tool mentation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10:21. affects subsequent success in implementation, and 8. Gale RC, Asch SM, Taylor T, et al. The most used and most helpful facil- the extent to which the tools affect patient experi- itators for patient-centered medical home implementation. Implement ence of care. We attempted to include tools that Sci. 2015;10:52. 9. McDonald KM, Schultz E, Albin L, et al. Care Coordination Measures Atlas have varying implementation difficulty, that have Version 4. Rockville, MD: Agency for Healthcare Research and Quality; broad applicability, and that have not yet been uni- 10. AHRQ Publishing and Communications Guidelines, Section 6: Toolkit formly adopted in the VA based on our semi-struc- Guidance. http://www.ahrq.gov/research/publications/pubcomguide/ tured interviews. However, it remains to be seen pcguide6.html Accessibility verified January 4, 2018. 11. Bagalman E. The number of veterans that use VA health care services: a whether the tools will successfully align with clinics’ fact sheet. Washington, DC: Congressional Research Service; 2014. priorities, and there may be a tradeoff in terms of 12. Chang ET, Wang M, Kirsh S, Rubenstein LV. VA High-Risk Populations in the difficulty of the tools being implemented and Primary Care. Mineapolis, MN: Accepted for presentation at Academy Health Annual Research Meeting; 2015. the potential beneficial effect on patient experience 13. Rosland AM, Nelson K, Sun H, et al. The patient-centered medi- of care, with more difficult tools potentially provid- cal home in the Veterans Health Administration. Am J Manag Care. 2013;19(7):e263–272. ing more benefit, but also requiring more resources. 14. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred We realized from the outset that the toolkit can be reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med. 2009;6(7):e1000097. an adjunct to, but not a replacement for, a serious 15. Luck J, Bowman C, York L, et al. Multimethod evaluation of the VA’s peer- commitment on the part of clinic leadership and to-peer Toolkit for patient-centered medical home implementation. J Gen Intern Med. 2014;29(Suppl 2):S572–S578. staff to improve care coordination. TBM page 501 of 502 Downloaded from https://academic.oup.com/tbm/article/8/3/492/5001928 by DeepDyve user on 14 July 2022 ORIGINAL RESEARCH 16. Gale RC, Luck J, York L, Asch S. Peer-to-peer toolkits enhance sharing of Design and Control of Workflow Processes: Business Process 22. Reijers HA. best practices across an integrated delivery system. J Patient Cent Res Management for the Service Industry. Chapter 6: Heuristic Workflow Rev. 2016;3:189. Redesign, vol. 2617. Berlin: Springer; 2003. 17. Ganz DA, Huang C, Saliba D, Shier V, Berlowitz D, Lukas CV, Pelczarski 23. York L , Bruce B, Luck J, et al. Online toolkits for metric-driven K, Schoelles K, Wallace LC, Neumann P. Preventing Falls in Hospitals: quality improvement: the veterans health administration A Toolkit for Improving Quality of Care [online]. In. Rockville, MD: Agency Jt Comm J Qual Patient Saf. managed grassroots approach. for Healthcare Research and Quality; 2013. 2013;39(12):561–569. 18. Engaging Patients in Improving Ambulatory Care: A Compendium of 24. DeWalt DA, Broucksou KA, Hawk V, et al. Developing and test- Tools from Maine, Oregon, and Humboldt County, California. https:// Nurs Outlook. ing the health literacy universal precautions toolkit. www.rwjf.org /content/dam/farm/toolkits/toolkits/2013/rwjf404402 2011;59(2):85–94. Accessibility verified January 4, 2018. 25. Mabachi NM, Cifuentes M, Barnard J, et al. Demonstration of the health 19. Carmichael J, Jassar G, Nguyen PA. Healthcare metrics: Where do pharma- literacy universal precautions toolkit: lessons for quality improvement. Am J Health Syst Pharm. 2016;73(19):1537–1547. Ambul Care Manage. 2016;39(3):199–208. cists add value? 20. Parchman ML, Noël PH, Lee S. Primary care attributes, 26. Weiss BD, Brega AG, LeBlanc WG, et al. Improving the effectiveness of health care system hassles, and chronic illness. Med Care. medication review: guidance from the health literacy universal precau- 2005;43(11):1123–1129. J Am Board Fam Med. 2016;29(1):18–23. tions toolkit. 21. Tsan MF, Puglisi T. Health care operations activities that may consti- 27. Brega AG, Freedman MA, LeBlanc WG, et al. Using the health literacy Irb. universal precautions toolkit to improve the quality of patient materials. tute research: the department of veterans affairs’s perspective. 2014;36(1):9–11. J Health Commun. 2015;20(Suppl 2):69–76. page 502 of 502 TBM

Journal

Translational Behavioral MedicineOxford University Press

Published: May 23, 2018

There are no references for this article.