After-visit summaries in primary care: mixed methods results from a literature review and stakeholder interviews

After-visit summaries in primary care: mixed methods results from a literature review and... Abstract Background After-visit summary (AVS) documents presenting key information from each medical encounter have become standard in the USA due to federal health care reform. Little is known about how they are used or whether they improve patient care. Methods First, we completed a literature review and described the totality of the literature on AVS by article type and major outcome measures. Next, we used reputational sampling from large-scale US studies on primary care to identify and interview nine stakeholders on their perceptions of AVS across high-performing primary care practices. Interviews were transcribed and coded for AVS use in practice, perceptions of the best/worst features and recommendations for improving AVS utility in routine care. Results The literature review resulted in 17 studies; patients reported higher perceived value of AVS compared with providers, despite poor recall of specific AVS content and varied post-visit use. In key informant interviews, key informants expressed enthusiasm for the potential of using AVS to reinforce key information with patients, especially if AVS were customizable. Despite this potential, key informants found that AVS included incorrect information and did not feel that patients or their practices were using AVS to enhance care. Conclusions There is a gap between the potential of AVS and how providers and patients are using it in routine care. Suggestions for improved use of AVS include increasing customization, establishing care team responsibilities and workflows and ensuring patients with communication barriers have dedicated support to review AVS during visits. Electronic health records, meaningful use, patient-centred care, patient portals, personal health records, primary health care Introduction Spurred by US health care reform and the subsequent Meaningful Use financial incentives, many US health care systems and clinicians have implemented electronic health records (EHRs) that adhere to specific requirements. This includes the requirement to provide a written clinical summary from the EHR to patients after each clinical encounter (1)—referred to as an after-visit summary (AVS). AVS have had a rapid introduction into clinical practice (2), given that the vast majority of US hospitals (94%) and office-based health professionals (77%) met Stage 1 Meaningful Use metrics in 2014 (3,4), of which AVS was a core component. Even with impending changes to the Meaningful Use program in the coming years (5), the current practice of visit summaries in primary care is likely to continue as a part of patient-centred care, especially since consumers are now accustomed to written encounter summaries. By providing patients with a written record of medical decisions and care plans, the use of AVS has the potential to improve patient knowledge, self-management and patient–provider communication. Numerous studies have documented barriers patients face in understanding and remembering information about their treatments and care plans after a visit (6–9). In particular, AVS use may hold great potential in addressing the well-documented barriers to patient–provider communication and shared decision making faced by vulnerable patients with limited health literacy and limited English proficiency (10–14). Despite this potential, there is limited research that explores AVS implementation in clinical practice and how its use has impacted patient and provider outcomes. Because of the paucity of information available, there were two complementary objectives of this study: (i) to explore the existing literature on the current use of AVS and (ii) to gain perspectives from clinical leaders about the current implementation and potential for integrating AVS into clinical practice. In particular, we sought to integrate findings from these objectives, with a specific focus on vulnerable patient populations. Methods Literature review In January 2018, we conducted a comprehensive search on PubMed to identify articles from the queries ‘after visit summary’, ‘visit summary’, ‘visit discharge’ and ‘clinical summary’. Papers were included if they (i) were published in English language and (ii) represented research conducted in the USA (where the term AVS is most commonly used). We excluded papers if (i) they mentioned the term AVS (such as being listed as one of many related EHR tools) but did not provide any data on AVS use specifically or (ii) if they were referring to a broader process of visit communication that did not involve the standardized AVS tool. We also reviewed the references lists of included articles to identify additional studies with a focus on AVS use. Although we had a specific interest in the impact of AVS use among vulnerable patients, we conducted a broad literature review on AVS because there were few studies identified overall. The included articles were categorized by study type, as there was wide variation in the research goals. For example, several studies focused on provider perceptions/use of the AVS, which varied greatly from studies assessing the readability of the document or longer term patient understanding of the information provided. We also categorized the research methods employed as higher (e.g. trials, strong comparison groups) versus lower quality (e.g. case studies, lack of comparison group). Finally, within each type of research study, the small sample size of the included articles allowed us to directly summarize the major findings and provide examples of the key outcomes examined. Key informant interviews To complement this literature review, we also conducted a small qualitative study among leaders in primary care about their perceptions of AVS use in routine practice. Rather than using a random sampling approach that might have captured practices without any current routine AVS use, we instead used purposive sampling to identify a sample of leaders in high-performing primary care practices more likely to be attesting for Meaningful Use certification in their practices. Specifically, we first used reputational sampling from published literature of large demonstration projects that systematically identified high-performing sites in both academic and safety primary care sites (15,16)—identifying and interviewing experts who had recently completed multiple site visits and in-depth observations of primary care practices nationally (including variation by region and practice type). We then used snowball sampling to identify the remaining key informants, ensuring that a significant portion (at least 1/3) of key informants were leaders or had extensive experience working with safety net health care settings, as this was a major objective of our study. This selection process did not target positive versus negative opinions of AVS use specifically, as interviewees had experience with AVS in practice that could have differed substantially from one another. In total, we conducted interviews with key informants from nine primary care sites, concluding after we had reached thematic saturation of the current types of AVS use. We used a semi-structured interview guide to gain perspectives about (i) current AVS practices within their system (e.g. who is responsible for AVS distribution, the process for distributing AVS and how AVS information is customized), (ii) the potential of AVS to improve patient knowledge and outcomes within their system and more broadly across primary care systems nationwide, (iii) strategies to improve the use of AVS and (iv) specific considerations for using AVS for the care of individuals with limited English proficiency and limited health literacy. The interviews were audio-recorded and transcribed for analysis. We used descriptive qualitative methods (17) to organize, categorize and code the transcripts across all of the major interview discussion topics. More specifically, we coded discrete information provided in the interviews into categories (such as the staff member responsible for AVS distribution at each site, the AVS features used the most, the AVS features viewed as least useful), as well as used thematic coding to capture broader ideas about team-based care, workflows and other topics that could influence the impact of AVS use in clinical care. All four co-authors conducted the key informant interviews and reached consensus on the final coding categories and emergent themes, and two of the co-authors (CRL and LT) completed the coding process on all transcripts once the codebook was established. The University of California San Francisco Institutional Review Board deemed this study as not classifying as human subjects research. Results Literature review Our literature review resulted in 263 articles (243 from PubMed, 20 manually identified from reference lists). We excluded 246 articles, resulting in 17 final articles (Table 1). We developed four major categories of studies (not mutually exclusive): Table 1. Summary of articles included in after visit summary (AVS) literature review Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  EHRs, electronic health records; RCT, randomized controlled trial. View Large Table 1. Summary of articles included in after visit summary (AVS) literature review Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  EHRs, electronic health records; RCT, randomized controlled trial. View Large 1. Case studies of implementation (15,18–22); 2. Qualitative/quantitative assessments of patient perceptions (23–30); 3. Qualitative/quantitative assessments of clinician perceptions (18,25,26,28,31); 4. Observational studies or interventional research (25,32,33). A substantial number of these studies used less rigorous methodological designs (such as convenience samples with pre-post self-reported measures); but 8 of the 14 studies (23–29,31) employed in-depth survey, qualitative or experimental methods. Examples or case studies of AVS implementation in real-world practice The articles examining implementation of AVS emphasized team-based approaches that utilized standard workflows. One study encouraged team-based responsibility, with nurses and medical assistants (MAs) delivering the AVS and care plan at the conclusion of the visit (15). Another study discussed the potential to integrate AVS into a health coaching model, using the AVS document as a tool to assess patient understanding (19). In the three content analyses, one study found only half of AVS contained information about follow-up appointments and only a quarter contained tailored AVS sections (18), while the others found that AVS were written with complex language and at a readability level requiring a higher level of education to understand (22,34). Patient perceptions of AVS Patient perspectives on AVS were favourable. In total, four qualitative studies (23,26,28,29) reported that patients used the document to relay information to their families or other physicians (23,28,29). However, patients expressed concerns about the accuracy of their information (26,28,29) and the potential for privacy breaches (28,29). While the overall readability of the AVS was problematic in some cases (26,29), many patients desired more information (such as more detailed information or context about their diagnoses and treatment/disease management) (30). Quantitative studies (24,25,27) echoed these themes: a vast majority of patients found the AVS useful, but only half or fewer reported using them after the visit. Clinician perceptions of AVS The studies examining clinician perceptions were focused on physicians. Overall, physicians had moderately favourable views of the ease and potential of using AVS for patient care and education (25,28,31). However, they expressed concerns about the high complexity of information and the lack of tailoring to the needs of specific patients (25,26,28), particularly with regard to literacy level and language. In addition, physicians expressed concerns about not always having sufficient time during practice to update the problem list or medication list and therefore mentioned errors and extraneous information (e.g. outdated diagnostic codes) (31). Observational or interventional research using AVS Three articles evaluated interventions centred on clinical applications of AVS, most of which did not result in significant findings. There was high variability in whether patients reported using AVS after their initial visits, from a small minority (25) to a majority of patients who received highly personalized versions (32). A randomized controlled trial of AVS content did not find significant differences in patient adherence, satisfaction or recall of medical information when directly comparing AVS documents with varying amounts of content (25). Patients’ recall of the information on the AVS was low (only ~33% of content categories); this recall of information was unexpectedly not related to patients’ health literacy status or the amount of information displayed. Key informant interviews In our key informant interviews, the final sample of nine interviewees represented academic, safety net and private practices (Table 2). The vast majority of participants were using the Epic EHR system in their practice (similar to many other health care settings nationwide (35)), even though we did not use this as a specific inclusion criterion. Despite this, several of the participants were also able to discuss more than one EHR given their experiences with multiple site visits or their previous clinical experience prior to Epic implementation. Table 2. Summary of key informant interviewees by site and role Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  View Large Table 2. Summary of key informant interviewees by site and role Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  View Large Current state of AVS implementation A high-level summary of the current AVS use is found in Table 3. Major findings included the following. Table 3. Summary of current after-visit summary (AVS) implementation by interview site Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  EHRs, electronic health records. View Large Table 3. Summary of current after-visit summary (AVS) implementation by interview site Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  EHRs, electronic health records. View Large Regular distribution of AVS Likely driven by Meaningful Use, most clinics issued a printed AVS at the majority (if not all) of visits. In addition, many clinics used the ‘patient instructions’ section of the AVS to include personalized information like counselling recommendations and guidance for self-management. I would say it’s probably the sections that are most used by the clinician are the blank free text space where you do write out some instructions. Patients satisfied with AVS, but might not be using it Several interviewees talked about positive patient perceptions (mirroring the literature review results above): ‘Patients actually really, really like having the information’. However, few to no interviewees suggested that the patients referred to the AVS post-visit: ‘I think the patient treats it like they would treat any other confusing piece of paper, which is either to throw it away before they leave the clinic or after they get home’. Clinics not using AVS for patient teaching The majority of practices did not use the AVS in a standard way to reinforce specific information with patients, instead printing and handing it out without explanation. I’ve yet to find anyone, anyplace where someone goes over the After Visit Summary with the patient. And I’ve asked many places [even in high-performing sites] because it seems so obvious that you want to do that in terms of closing the loop…. It’s such a terrific way to close the loop, and it’s just surprising. People just don’t do it. Slightly less than half of interviewees did mention highlighting some information on the AVS. Yet this was not done in a standardized way across clinicians or visits. Importance of specific features of the current AVS When considering specific features of the AVS (Table 4), almost all participants expressed that the patient instructions section was most useful because of the ability to customize information easily. The medication list (if accurate) was also mentioned as useful. Finally, upcoming visits and care plans were also highlighted as potentially important (but perhaps not always standard). Table 4. Summary of best and worst features of after-visit summary (AVS) document by interview site   Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.    Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.  View Large Table 4. Summary of best and worst features of after-visit summary (AVS) document by interview site   Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.    Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.  View Large Next steps: overcoming barriers The key informants unanimously felt that AVS could improve clinical outcomes if utilized properly. When asked about future changes in the Meaningful Use program related to the AVS, interviewees did not foresee abandoning this document in practice. I think [the AVS] could be really important. I don’t think it’s important the way it’s used now, but I think it could be extremely important and extremely helpful. Moving forward, improvements in AVS use were related to the following themes: Team-based workflows Interviewees expressed that non-clinicians are well positioned to use the AVS with patients for operational next steps (like follow-up appointments). Within the one clinic with a standard MA workflow already in place, the interviewee commented, ‘MAs really like it. They like being part of the process of closing the loop and just helping the patient with those final details’. In addition, MAs or other staff could likely counsel related to lifestyle (such as diet or exercise) or other content with additional training and/or support. For example, one interviewee stated that the MA could use the AVS more effectively, but only with guidance from a provider: The problem is the MA would have to know which part of the After Visit Summary to go over because you don’t want to go over more than like a couple of things, because people are not going to walk on practice remembering eight or 10 things. Focus within the AVS In addition, there were many comments related to the idea that the AVS ‘seems to want to serve too many purposes’. In addition to multiple content areas like medications and diagnoses, clinicians also wrote in personalized instructions in varying ways. Therefore, the current AVS format was long and complex, especially to find specific necessary information from a single visit. Increased ability to customize the AVS in straightforward ways was viewed as critical. Tailoring by language and literacy Because the AVS was not available in non-English languages or with low-literacy text, interviewees requested adjusting content to improving patient communication. For example: For our folks that speak other languages, we are really limited in terms of written instructions we can provide for them. I don’t have any good workaround for that. If there’s a way to do like the med chart with pictures, not just all words… [The AVS is] basically four pages of words. Conclusions Among a small amount of published literature on the topic, we found that patients perceive AVS positively, but few appear to routinely refer to the document after the visit. Clinicians surveyed in the published literature were less satisfied than patients with AVS. Moreover, beyond this literature review of existing research, we also conducted our own qualitative investigation among primary care leaders about their perceptions of AVS in high-performing clinical practices. Among these key informant interviewees, we found similar implementation experiences across a varied group of primary care practices. While a hardcopy AVS were distributed in virtually all encounters, there was uncertainty about whether patients used AVS and a lack of routine practice to educate patients about AVS content. The customized patient instructions section was viewed as most useful within the AVS, but this could be buried in the midst of other content. Despite such challenges, interviewees expressed overall positivity about the potential of the AVS to improve patient understanding in the future. This is the first study to our knowledge that comprehensively studied the current use of AVS in real-world practices in combination with stakeholder perceptions across multiple health care settings about the best ways to improve AVS use for maximum impact. While interviewees in this study provided recommendations for improving the content of AVS to improve implementation, any content changes would be insufficient without additional workflows to support patient use and understanding. Future research is needed to understand whether and how AVS contribute to improved patient outcomes (e.g. understanding/retention, clinical outcomes) and to directly compare the impact of different workflows of AVS distribution. There is no published literature about electronic delivery of AVS through online patient portals, or comparisons of digital versus printed distribution. In addition, there is a need for research to compare workflows of teach-back (36) using AVS to determine the best modes for patient understanding and retention. Our study supports previous research on patient–provider communication. For example, patients in our literature review expressed high interest in access to information from their medical encounters via AVS, which is similar to many other studies on patient interest in and satisfaction with access to their online medical record information (37,38). Moreover, our findings support previous work that that training and/or tools can improve in-person communication (39), especially for vulnerable patient populations (40,41), but this is the first study to our knowledge of whether the AVS is being used for patient education and teach-back. Moreover, implementation of these improved communication strategies into real-world settings requires overcoming obstacles such as under-staffing and insufficient time during visits. There are several limitations of this study. First, the literature review may have missed studies using a structured process for delivering patient education materials at the conclusion of visits or hospitalizations. In addition, our qualitative sample was small and is not broadly generalizable, and most participants gave feedback on a single EHR product. In addition, the interviewees were all providers without any patient representation. However, we reached thematic saturation with this small but diverse set of interviewees across multiple health care settings. Moving forward, patient summaries of information like AVS will likely continue to play a role in primary care. AVS utility for both patients and clinicians will likely increase as content and design are improved. The growth of the patient-centred medical home and the emphasis on team-based care will likely result in new roles and responsibilities for communication with patients, and AVS may take centre stage in workflow redesign. Over time, as federal policies and incentives for EHR use change, AVS will survive only if clinicians and patients find them relevant and useful. Declaration Funding: The Roundtable on Health Literacy of the National Academies of Sciences, Engineering, and Medicine provided support for our investigations into AVS. CRL is supported by AHRQ R00HS022408. Conflict of interest: The authors report no conflicts of interest. Acknowledgement We would like to thank all the people we interviewed who contributed their time to this project. References 1. Centers for Medicare and Medicaid Services. 2015 Meaningful Use Definitions and Objectives. http://www.healthit.gov/providers-professionals/meaningful-use-definition-objectives ( accessed on 14 May 2015). 2. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med  2010; 363: 501– 4. Google Scholar CrossRef Search ADS PubMed  3. Centers for Medicare and Medicaid Services. 2014 Hospitals Participating in the CMS EHR Incentive Programs. http://dashboard.healthit.gov/quickstats/pages/FIG-Hospitals-EHR-Incentive-Programs.php ( accessed on 14 May 2015). 4. Centers for Medicare and Medicaid Services. 2014. Office-based Health Care Professional Participation in the CMS EHR Incentive Programs. http://thehealthcareblog.com/blog/2016/01/19/ehr-incentive-programs-where-we-go-from-here/ ( accessed on 14 May 2015). 5. Slavitt A, DeSalvo K. EHR incentive programs: where we go next. The CMS Blog. 6. Stevenson FA, Barry CA, Britten N, Barber N, Bradley CP. Doctor-patient communication about drugs: the evidence for shared decision making. Soc Sci Med  2000; 50: 829– 40. Google Scholar CrossRef Search ADS PubMed  7. Roter DL, Hall JA. Studies of doctor-patient interaction. Annu Rev Public Health  1989; 10: 163– 80. Google Scholar CrossRef Search ADS PubMed  8. Sarkar U, Schillinger D, Bibbins-Domingo Ket al.   Patient-physicians’ information exchange in outpatient cardiac care: time for a heart to heart? Patient Educ Couns  2011; 85: 173– 9. Google Scholar CrossRef Search ADS PubMed  9. Hummel J, Evans P. Providing Clinical Summaries to Patients after Each Visit: A Technical Guide . Seattle, WA: Qualis Health, 2012. 10. Peek ME, Odoms-Young A, Quinn MTet al.   Race and shared decision-making: perspectives of African-Americans with diabetes. Soc Sci Med  2010; 71: 1– 9. Google Scholar CrossRef Search ADS PubMed  11. Aboumatar HJ, Carson KA, Beach MC, Roter DL, Cooper LA. The impact of health literacy on desire for participation in healthcare, medical visit communication, and patient reported outcomes among patients with hypertension. J Gen Intern Med  2013; 28: 1469– 76. Google Scholar CrossRef Search ADS PubMed  12. Institute of Medicine. Health Literacy: A Prescription to End Confusion . Washington, DC : National Academies Press (US), 2004. 13. Schillinger D. Literacy and health communication: reversing the ‘inverse care law’. Am J Bioeth  2007; 7: 15– 8. Google Scholar CrossRef Search ADS PubMed  14. Schillinger D, Bindman A, Wang F, Stewart A, Piette J. Functional health literacy and the quality of physician-patient communication among diabetes patients. Patient Educ Couns  2004; 52: 315– 23. Google Scholar CrossRef Search ADS PubMed  15. Sinsky CA, Willard-Grace R, Schutzbank AMet al.   In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med  2013; 11: 272– 8. Google Scholar CrossRef Search ADS PubMed  16. Wagner EH, Gupta R, Coleman K. Practice transformation in the safety net medical home initiative: a qualitative look. Med Care  2014; 52( 11 suppl 4): S18– 22. Google Scholar CrossRef Search ADS PubMed  17. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health  2000; 23: 334– 40. Google Scholar CrossRef Search ADS PubMed  18. Salmon C, O’Conor R, Singh Set al.   Characteristics of outpatient clinical summaries in the United States. Int J Med Inform  2016; 94: 75– 80. Google Scholar CrossRef Search ADS PubMed  19. Bodenheimer T, Laing BY. The teamlet model of primary care. Ann Fam Med  2007; 5: 457– 61. Google Scholar CrossRef Search ADS PubMed  20. Jiggins K. A content analysis of the meaningful use clinical summary: do clinical summaries promote patient engagement? Prim Health Care Res Dev  2015; 17: 1– 14. 21. Kanter M, Martinez O, Lindsay G, Andrews K, Denver C. Proactive office encounter: a systematic approach to preventive and chronic care at every patient encounter. Perm J  2010; 14: 38– 43. Google Scholar PubMed  22. Colorafi K, Moua L, Shaw M, Ricker D, Postma J. Assessing the value of the meaningful use clinical summary for patients and families with pediatric asthma. J Asthma  2017; 55:1–9. 23. Tang PC, Newcomb C. Informing patients: a guide for providing patient health information. J Am Med Inform Assoc  1998; 5: 563– 70. Google Scholar CrossRef Search ADS PubMed  24. Neuberger M, Dontje K, Holzman Get al.   Examination of office visit patient preferences for the after-visit summary (AVS). Perspect Health Inf Manag  2014; 11: 1d. Google Scholar PubMed  25. Pavlik V, Brown AE, Nash S, Gossey JT. Association of patient recall, satisfaction, and adherence to content of an electronic health record (EHR)-generated after visit summary: a randomized clinical trial. J Am Board Fam Med  2014; 27: 209– 18. Google Scholar CrossRef Search ADS PubMed  26. Black H, Gonzalez R, Priolo Cet al.   True “meaningful use”: technology meets both patient and provider needs. Am J Manag Care  2015; 21: e329– 37. Google Scholar PubMed  27. Emani S, Healey M, Ting DYet al.   Awareness and use of the after-visit summary through a patient portal: evaluation of patient characteristics and an application of the theory of planned behavior. J Med Internet Res  2016; 18: e77. Google Scholar CrossRef Search ADS PubMed  28. Federman AD, Sanchez-Munoz A, Jandorf Let al.   Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design. J Am Med Inform Assoc  2016; 24: e61– 68. 29. Belyeu BM, Klein JW, Reisch LMet al.   Patients’ perceptions of their doctors’ notes and after-visit summaries: a mixed methods study of patients at safety-net clinics. Health Expect  2017; 21: 485– 93. Google Scholar CrossRef Search ADS PubMed  30. Clarke MA, Moore JL, Steege LMet al.   Toward a patient-centered ambulatory after-visit summary: identifying primary care patients’ information needs. Health Soc Care Community  2017; 43: 1– 16. 31. Emani S, Ting DY, Healey Met al.   Physician perceptions and beliefs about generating and providing a clinical summary of the office visit. Appl Clin Inform  2015; 6: 577– 90. Google Scholar CrossRef Search ADS PubMed  32. Anbar RD, Anbar JS, Hashim MA. Use of an after-visit summary to augment mental health of children and adolescents. Clin Pediatr (Phila)  2015; 54: 1009– 11. Google Scholar CrossRef Search ADS PubMed  33. Dehen RI, Carter SU, Watanabe M. Impact of after visit summaries on patient return rates at an acupuncture and oriental medicine clinic. Med Acupunct  2014; 26: 221– 5. Google Scholar CrossRef Search ADS PubMed  34. Jiggins K. A content analysis of the meaningful use clinical summary: do clinical summaries promote patient engagement? Prim Health Care Res Dev  2016; 17: 238– 51. Google Scholar CrossRef Search ADS PubMed  35. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and healthcare systems. J Am Med Inform Assoc  2015; 22: 465–71. 36. Use the Teach-Back Method: Tool #5. 2015; http://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/literacy-toolkit/healthlittoolkit2-tool5.html (accessed on 4 February 2018). 37. Ralston JD, Carrell D, Reid Ret al.   Patient web services integrated with a shared medical record: patient use and satisfaction. J Am Med Inform Assoc  2007; 14: 798– 806. Google Scholar CrossRef Search ADS PubMed  38. Tieu L, Sarkar U, Schillinger Det al.   Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res  2015; 17: e275. Google Scholar CrossRef Search ADS PubMed  39. Ha Dinh TT, Bonner A, Clark R, Ramsbotham J, Hines S. The effectiveness of the teach-back method on adherence and self-management in health education for people with chronic disease: a systematic review. JBI Database System Rev Implement Rep  2016; 14: 210– 47. Google Scholar CrossRef Search ADS PubMed  40. Wolff K, Cavanaugh K, Malone Ret al.   The diabetes literacy and numeracy education toolkit (DLNET): materials to facilitate diabetes education and management in patients with low literacy and numeracy skills. Diabetes Educ  2009; 35: 233– 45. Google Scholar CrossRef Search ADS PubMed  41. White RO, Eden S, Wallston KAet al.   Health communication, self-care, and treatment satisfaction among low-income diabetes patients in a public health setting. Patient Educ Couns  2015; 98: 144– 9. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Family Practice Oxford University Press

After-visit summaries in primary care: mixed methods results from a literature review and stakeholder interviews

Loading next page...
 
/lp/ou_press/after-visit-summaries-in-primary-care-mixed-methods-results-from-a-DVemgTZVbG
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
0263-2136
eISSN
1460-2229
D.O.I.
10.1093/fampra/cmy045
Publisher site
See Article on Publisher Site

Abstract

Abstract Background After-visit summary (AVS) documents presenting key information from each medical encounter have become standard in the USA due to federal health care reform. Little is known about how they are used or whether they improve patient care. Methods First, we completed a literature review and described the totality of the literature on AVS by article type and major outcome measures. Next, we used reputational sampling from large-scale US studies on primary care to identify and interview nine stakeholders on their perceptions of AVS across high-performing primary care practices. Interviews were transcribed and coded for AVS use in practice, perceptions of the best/worst features and recommendations for improving AVS utility in routine care. Results The literature review resulted in 17 studies; patients reported higher perceived value of AVS compared with providers, despite poor recall of specific AVS content and varied post-visit use. In key informant interviews, key informants expressed enthusiasm for the potential of using AVS to reinforce key information with patients, especially if AVS were customizable. Despite this potential, key informants found that AVS included incorrect information and did not feel that patients or their practices were using AVS to enhance care. Conclusions There is a gap between the potential of AVS and how providers and patients are using it in routine care. Suggestions for improved use of AVS include increasing customization, establishing care team responsibilities and workflows and ensuring patients with communication barriers have dedicated support to review AVS during visits. Electronic health records, meaningful use, patient-centred care, patient portals, personal health records, primary health care Introduction Spurred by US health care reform and the subsequent Meaningful Use financial incentives, many US health care systems and clinicians have implemented electronic health records (EHRs) that adhere to specific requirements. This includes the requirement to provide a written clinical summary from the EHR to patients after each clinical encounter (1)—referred to as an after-visit summary (AVS). AVS have had a rapid introduction into clinical practice (2), given that the vast majority of US hospitals (94%) and office-based health professionals (77%) met Stage 1 Meaningful Use metrics in 2014 (3,4), of which AVS was a core component. Even with impending changes to the Meaningful Use program in the coming years (5), the current practice of visit summaries in primary care is likely to continue as a part of patient-centred care, especially since consumers are now accustomed to written encounter summaries. By providing patients with a written record of medical decisions and care plans, the use of AVS has the potential to improve patient knowledge, self-management and patient–provider communication. Numerous studies have documented barriers patients face in understanding and remembering information about their treatments and care plans after a visit (6–9). In particular, AVS use may hold great potential in addressing the well-documented barriers to patient–provider communication and shared decision making faced by vulnerable patients with limited health literacy and limited English proficiency (10–14). Despite this potential, there is limited research that explores AVS implementation in clinical practice and how its use has impacted patient and provider outcomes. Because of the paucity of information available, there were two complementary objectives of this study: (i) to explore the existing literature on the current use of AVS and (ii) to gain perspectives from clinical leaders about the current implementation and potential for integrating AVS into clinical practice. In particular, we sought to integrate findings from these objectives, with a specific focus on vulnerable patient populations. Methods Literature review In January 2018, we conducted a comprehensive search on PubMed to identify articles from the queries ‘after visit summary’, ‘visit summary’, ‘visit discharge’ and ‘clinical summary’. Papers were included if they (i) were published in English language and (ii) represented research conducted in the USA (where the term AVS is most commonly used). We excluded papers if (i) they mentioned the term AVS (such as being listed as one of many related EHR tools) but did not provide any data on AVS use specifically or (ii) if they were referring to a broader process of visit communication that did not involve the standardized AVS tool. We also reviewed the references lists of included articles to identify additional studies with a focus on AVS use. Although we had a specific interest in the impact of AVS use among vulnerable patients, we conducted a broad literature review on AVS because there were few studies identified overall. The included articles were categorized by study type, as there was wide variation in the research goals. For example, several studies focused on provider perceptions/use of the AVS, which varied greatly from studies assessing the readability of the document or longer term patient understanding of the information provided. We also categorized the research methods employed as higher (e.g. trials, strong comparison groups) versus lower quality (e.g. case studies, lack of comparison group). Finally, within each type of research study, the small sample size of the included articles allowed us to directly summarize the major findings and provide examples of the key outcomes examined. Key informant interviews To complement this literature review, we also conducted a small qualitative study among leaders in primary care about their perceptions of AVS use in routine practice. Rather than using a random sampling approach that might have captured practices without any current routine AVS use, we instead used purposive sampling to identify a sample of leaders in high-performing primary care practices more likely to be attesting for Meaningful Use certification in their practices. Specifically, we first used reputational sampling from published literature of large demonstration projects that systematically identified high-performing sites in both academic and safety primary care sites (15,16)—identifying and interviewing experts who had recently completed multiple site visits and in-depth observations of primary care practices nationally (including variation by region and practice type). We then used snowball sampling to identify the remaining key informants, ensuring that a significant portion (at least 1/3) of key informants were leaders or had extensive experience working with safety net health care settings, as this was a major objective of our study. This selection process did not target positive versus negative opinions of AVS use specifically, as interviewees had experience with AVS in practice that could have differed substantially from one another. In total, we conducted interviews with key informants from nine primary care sites, concluding after we had reached thematic saturation of the current types of AVS use. We used a semi-structured interview guide to gain perspectives about (i) current AVS practices within their system (e.g. who is responsible for AVS distribution, the process for distributing AVS and how AVS information is customized), (ii) the potential of AVS to improve patient knowledge and outcomes within their system and more broadly across primary care systems nationwide, (iii) strategies to improve the use of AVS and (iv) specific considerations for using AVS for the care of individuals with limited English proficiency and limited health literacy. The interviews were audio-recorded and transcribed for analysis. We used descriptive qualitative methods (17) to organize, categorize and code the transcripts across all of the major interview discussion topics. More specifically, we coded discrete information provided in the interviews into categories (such as the staff member responsible for AVS distribution at each site, the AVS features used the most, the AVS features viewed as least useful), as well as used thematic coding to capture broader ideas about team-based care, workflows and other topics that could influence the impact of AVS use in clinical care. All four co-authors conducted the key informant interviews and reached consensus on the final coding categories and emergent themes, and two of the co-authors (CRL and LT) completed the coding process on all transcripts once the codebook was established. The University of California San Francisco Institutional Review Board deemed this study as not classifying as human subjects research. Results Literature review Our literature review resulted in 263 articles (243 from PubMed, 20 manually identified from reference lists). We excluded 246 articles, resulting in 17 final articles (Table 1). We developed four major categories of studies (not mutually exclusive): Table 1. Summary of articles included in after visit summary (AVS) literature review Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  EHRs, electronic health records; RCT, randomized controlled trial. View Large Table 1. Summary of articles included in after visit summary (AVS) literature review Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  Category  Authors, year  Setting; population studied/described  Study design and/or primary outcome  Examples or case studies of AVS implementation in real-world practice  Bodenheimer and Laing, 2007 (19)  General primary care practice; case study of clinical workflows and practices  Description of best practices; AVS as a component of closing the loop within a team-based care model  Colorafi et al., 2017 (22)  Paediatric asthma patients’ clinical summaries; 20 clinical summaries (12 written by physicians and 8 by nurse practitioners) analysed  Content analysis, descriptive statistics and readability measurements; average reading level ninth grade, not easy to understand, changes necessary to improve communication between providers, patients and their families  Jiggins et al., 2015 (20)  Urban primary care practice; 100 older adults receiving care from 10 family clinicians  Content analysis of AVS; AVS features and content  Kanter et al., 2010 (21)  Integrated care system; patients with outpatient visits within 13 medical centres  Description of proactive office encounter intervention and correlated metrics; Clinical quality and care closure performance measures  Salmon et al., 2016 (18)  Primary care practices; 13 practices in 11 states using an EHR  Content evaluation of AVS; length, structure, readability, suitability, understandability and actionability of AVS  Sinsky et al., 2013 (15)  Primary care practices; 23 high-performing practices identified via an expert panel  Innovations in primary care content and workflow, including AVS workflow by non-clinician staff  Patient perceptions of AVS  Belyeu et al., 2017 (29)  Urban safety-net primary clinics; 27 patients with poorly controlled diabetes  Qualitative focus groups; patients found AVS content useful, want more detail and enjoy sharing clinical documents to teach others but were frustrated with inaccuracies  Black et al., 2015 (26)  Academic health system; 21 patients with asthma receiving care from and 13 clinicians providing care in primary care and asthma clinics serving low-income urban neighbourhoods  Qualitative analysis of focus groups; perspectives about AVS content and uses  Clarke et al., 2017 (30)  Family and community medicine clinics, 29 patients with acute or chronic illness  Semi-structured interviews reviewing example AVS; patients identified most important AVS features  Emani et al., 2016 (27)  Academic medical centre; 5370 patients enrolled in patient portal  Cross-sectional survey; awareness, access and perspectives of AVS use  Federman et al., 2016 (28)  Hospital- and community-based primary care practices; 39 patients across 4 sites  Individual semi-structured interviews and focus groups; previous AVS use and perspectives on AVS content, formatting, accuracy and privacy  Neuberger et al., 2014 (24)  Academic medical centre; 209 patients with a primary care visit  Cross-sectional survey and interview; accuracy, perspectives and use of AVS  Pavlik et al., 2014 (25)  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; patient recall of AVS information, satisfaction with care and adherence to clinician recommendation  Tang and Newcomb, 1998 (23)  Academic hospital; 20 patients with acute and chronic care visits  Qualitative focus groups; preferences for AVS and satisfaction with care  Clinician perceptions of AVS  Black et al., 2015 (26)  Academic health system; primary care and specialty care clinicians and nurses  Qualitative analysis of focus groups; perspectives about AVS content and uses  Emani et al., 2015 (31)  2 academic medical centres; 853 clinicians participating in Meaningful Use program  Cross-sectional survey; perspectives and expectations of AVS use, including ease of use, workload and effects on health outcomes  Federman et al., 2016 (28))  Hospital- and community-based primary care practices; 38 physicians and 18 nurse practitioners  Focus groups with primary care providers; perspectives on AVS content, formatting, workflow and concerns  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  Qualitative interviews to inform RCT; clinician perspectives about facilitators and barriers to AVS use  Salmon et al. 2016 (18)  Primary care practices; 13 medical directors or representatives of 13 practices in 11 states using an EHR  Interviews of medical directors; perceived helpfulness of AVS  Observational or interventional research using AVS  Anbar et al., 2015 (32)  Academic paediatric hospital; 285 child and adolescent patients who had received counselling for medical treatment  Descriptive study with follow-up; patient-reported AVS use following visit  Dehen et al., 2014 (33)  Integrative medicine clinic; 200 patients receiving care before or after, AVS implementation  Pre–post intervention study; patient return visits  Pavlik et al., 2014 (25))  Academic hospital; 272 adult primary care patients with ≥1 chronic condition  RCT; use of AVS and recall of information after visit  EHRs, electronic health records; RCT, randomized controlled trial. View Large 1. Case studies of implementation (15,18–22); 2. Qualitative/quantitative assessments of patient perceptions (23–30); 3. Qualitative/quantitative assessments of clinician perceptions (18,25,26,28,31); 4. Observational studies or interventional research (25,32,33). A substantial number of these studies used less rigorous methodological designs (such as convenience samples with pre-post self-reported measures); but 8 of the 14 studies (23–29,31) employed in-depth survey, qualitative or experimental methods. Examples or case studies of AVS implementation in real-world practice The articles examining implementation of AVS emphasized team-based approaches that utilized standard workflows. One study encouraged team-based responsibility, with nurses and medical assistants (MAs) delivering the AVS and care plan at the conclusion of the visit (15). Another study discussed the potential to integrate AVS into a health coaching model, using the AVS document as a tool to assess patient understanding (19). In the three content analyses, one study found only half of AVS contained information about follow-up appointments and only a quarter contained tailored AVS sections (18), while the others found that AVS were written with complex language and at a readability level requiring a higher level of education to understand (22,34). Patient perceptions of AVS Patient perspectives on AVS were favourable. In total, four qualitative studies (23,26,28,29) reported that patients used the document to relay information to their families or other physicians (23,28,29). However, patients expressed concerns about the accuracy of their information (26,28,29) and the potential for privacy breaches (28,29). While the overall readability of the AVS was problematic in some cases (26,29), many patients desired more information (such as more detailed information or context about their diagnoses and treatment/disease management) (30). Quantitative studies (24,25,27) echoed these themes: a vast majority of patients found the AVS useful, but only half or fewer reported using them after the visit. Clinician perceptions of AVS The studies examining clinician perceptions were focused on physicians. Overall, physicians had moderately favourable views of the ease and potential of using AVS for patient care and education (25,28,31). However, they expressed concerns about the high complexity of information and the lack of tailoring to the needs of specific patients (25,26,28), particularly with regard to literacy level and language. In addition, physicians expressed concerns about not always having sufficient time during practice to update the problem list or medication list and therefore mentioned errors and extraneous information (e.g. outdated diagnostic codes) (31). Observational or interventional research using AVS Three articles evaluated interventions centred on clinical applications of AVS, most of which did not result in significant findings. There was high variability in whether patients reported using AVS after their initial visits, from a small minority (25) to a majority of patients who received highly personalized versions (32). A randomized controlled trial of AVS content did not find significant differences in patient adherence, satisfaction or recall of medical information when directly comparing AVS documents with varying amounts of content (25). Patients’ recall of the information on the AVS was low (only ~33% of content categories); this recall of information was unexpectedly not related to patients’ health literacy status or the amount of information displayed. Key informant interviews In our key informant interviews, the final sample of nine interviewees represented academic, safety net and private practices (Table 2). The vast majority of participants were using the Epic EHR system in their practice (similar to many other health care settings nationwide (35)), even though we did not use this as a specific inclusion criterion. Despite this, several of the participants were also able to discuss more than one EHR given their experiences with multiple site visits or their previous clinical experience prior to Epic implementation. Table 2. Summary of key informant interviewees by site and role Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  View Large Table 2. Summary of key informant interviewees by site and role Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  Site no  Key informant site  Role  Location  1  Safety net practice  Director of quality  Oregon  2  Safety net practice  Internist  Colorado  3  Safety net academic practice  Director of primary care excellence  California  4  Safety net academic practice  Associate division chief of primary care  New York  5  Academic practice  Medical director  Massachusetts  6  Academic practice  Division chief  Massachusetts  7  Private practice  Internist  Wisconsin  8  Integrated delivery system  Director of primary care excellence  Washington  9  Integrated delivery system  Director of quality  California  View Large Current state of AVS implementation A high-level summary of the current AVS use is found in Table 3. Major findings included the following. Table 3. Summary of current after-visit summary (AVS) implementation by interview site Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  EHRs, electronic health records. View Large Table 3. Summary of current after-visit summary (AVS) implementation by interview site Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  Site  Current AVS workflow  Team member distributing AVS  Site 1: Safety net practice  • Paper copy of AVS (~2 pages) given to patients at end of visit • Standard MA work to check out patients (reviews mostly administrative tasks like scheduling visits/ lab draws) • Clinicians spend more time with patients with higher needs like those with limited health literacy • Clinic has an interdisciplinary committee to work on standardized non-jargon AVS language to meet patient literacy needs  Medical assistant (MA)  Site 2: Safety net practice  • Before recent Epic implementation, clinic used handwritten after-visit note (contained specialty appointment and lab info) • 80–90% of patients currently receive AVS • AVS prepared by clinician; MA gives to patient and highlights important parts  MA  Site 3: Academic, safety net practice  • Across many sites, clinics either are not using AVS or simply printing and handing AVS to patient without explanation • Not common for visit discharge to be viewed as an important process and little attention paid to AVS in patient-centred medical home implementation  Unclear role  Site 4: Academic, safety net practice  • Clinicians give printed AVS to almost all patients • Clinicians circle specific sections for patients to review • Patients who need the information on AVS the most have the longest and most difficult documents (>4 pages long)  Clinician  Site 5: Academic practice  • Standard process for AVS printing at every visit • Majority of patients turn down option to print AVS when asked • Many clinicians customize patient instructions in AVS with simpler language • AVS meaningful for some high-risk patients, who are prioritized to have a formal care planning process documented within AVS  Front desk  Site 6: Academic practice  • AVS printed at front desk at all visits (not using MAs or other staff) • Patients like the AVS, but clinicians do not (partly because it does not fit in well to workflow)  Front desk  Site 7: Private practice  • Standard practice for physician to print and review AVS with patients  Clinician  Site 8: Integrated delivery system  • Rarely sees ‘active teach-back’ using AVS • Patients probably not currently using AVS after it is printed • Unclear who in primary care team owns this responsibility  Unclear role  Site 9: Integrated delivery system  • AVS is one of many ways to communicate with patient • Clinicians can customize with free text or EHR shortcuts • AVS is cluttered and long, but has useful information  Both MAs and clinicians  EHRs, electronic health records. View Large Regular distribution of AVS Likely driven by Meaningful Use, most clinics issued a printed AVS at the majority (if not all) of visits. In addition, many clinics used the ‘patient instructions’ section of the AVS to include personalized information like counselling recommendations and guidance for self-management. I would say it’s probably the sections that are most used by the clinician are the blank free text space where you do write out some instructions. Patients satisfied with AVS, but might not be using it Several interviewees talked about positive patient perceptions (mirroring the literature review results above): ‘Patients actually really, really like having the information’. However, few to no interviewees suggested that the patients referred to the AVS post-visit: ‘I think the patient treats it like they would treat any other confusing piece of paper, which is either to throw it away before they leave the clinic or after they get home’. Clinics not using AVS for patient teaching The majority of practices did not use the AVS in a standard way to reinforce specific information with patients, instead printing and handing it out without explanation. I’ve yet to find anyone, anyplace where someone goes over the After Visit Summary with the patient. And I’ve asked many places [even in high-performing sites] because it seems so obvious that you want to do that in terms of closing the loop…. It’s such a terrific way to close the loop, and it’s just surprising. People just don’t do it. Slightly less than half of interviewees did mention highlighting some information on the AVS. Yet this was not done in a standardized way across clinicians or visits. Importance of specific features of the current AVS When considering specific features of the AVS (Table 4), almost all participants expressed that the patient instructions section was most useful because of the ability to customize information easily. The medication list (if accurate) was also mentioned as useful. Finally, upcoming visits and care plans were also highlighted as potentially important (but perhaps not always standard). Table 4. Summary of best and worst features of after-visit summary (AVS) document by interview site   Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.    Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.  View Large Table 4. Summary of best and worst features of after-visit summary (AVS) document by interview site   Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.    Best features of AVS  Worst features of AVS  Site 1: Safety net practice  • Patient instructions [Our committee has] ‘been paying special attention to things like the white space and exactly how much is on [the AVS]… we’ve created dot phrases to change [wording for] referrals and for procedure orders that explain more of what the thing is or what type of doctor and persons are being referred to [in plain language].’  • Too much information ‘I just go in and wipe all of that extra stuff out [to make the AVS more useful]’.  Site 2: Safety net practice  • Medication list ‘I think giving the patients an easy-to-read list of their medications is probably the most important thing’. • Patient instructions ‘I think after that, just a brief outline of what their instructions and what the goals of that appointment were’.  • Too much information, not usable ‘I think we do need to redesign our after visit summary, for sure… I just feel like the order it’s presented in is a little bit difficult for patients to understand and it’s all clunky’.  Site 3: Academic, safety net practice  • Patient instructions ‘They [need] the essential components of the care plan, particularly any changes in the care plan, and not all the other stuff that it has’.  • Too much information ‘Well, number one is they’re [need to be] really short…. It really should be a like a page’. • Does not show changes over time ‘It doesn’t indicate that instead of doing A, you’re going to do B…. If it doesn’t do that, then it’s really not helpful…. It really has to show the change in the care plan’.  Site 4: Academic, safety net practice  • Medications, upcoming visits, patient instructions ‘[I want to be able to say] “Here are your meds in a clear and organized way. Here are the tests or the upcoming appointments that you have. Here’s individualized instructions for you” ’. • Patient instructions ‘You can also attach educational pieces out of the EMR and that often has some pictures…. While it comes at the end of the after visit summary, I feel like it’s sort of a separate piece’.  • Too much information ‘I feel like the document is big and cumbersome and bulky and I think that most of us feel like, ‘Oh my god. Too much. This is useless’. … I think people have taken it and tried to target the things they wanted individual patients to look at. Not even giving them the other pages’. • Problem list not useful ‘I think the least valuable is the problems [list]’.  Site 5: Academic practice  • Care goals ‘The [care] goal section of Epic [is what we are focusing on now]’. • Medication lists, patient instructions ‘I do actually think we need to have their meds every time…. They need educational materials and meds and goals and care plans’.  • Too much information ‘Unfortunately, all this other stuff comes out too [when you print the AVS]…. Whether you could just say, ‘I only want these two sections here’. I don’t know if that’s possible’. • Problem list not useful ‘The other thing is we don’t really need to hand the patient their problem list every time’.  Site 6: Academic practice  • Medication list ‘I guess the medication has probably been the single most useful because that takes a lot of reconciliation. It’s so hard and there are so often mistakes with it’. • Patient instructions ‘I think [adding] some of the patient education information would be very nice’.  • Too much information, Not usable ‘ [The AVS] would have to be set up in a better way from the usability perspective right now because right now, for me to go find the information like that, it’s hidden’.  Site 7: Private practice  • Patient instructions ‘Patient information…. I’ve got lots of SmartPhrases that are relevant to different topics’.  • Too much information ‘It’s a lot of junk on there that just gets thrown in… I can see people get overwhelmed with information’. • Problem list not useful ‘The problem list… that’s a long thing but I think that’s maybe unnecessary’.  Site 8: Integrated delivery system  • Patient instructions, care goals ‘after visit summary include lots of behavior or often include behavioral recommendations [in customized patient instructions and care goal/plan sections]. That’s one of the beauties of putting it in the hands of the health coach’.  • Too much information ‘The [AVS] that I get here… have far more information than I would ever want to know’.  Site 9: Integrated delivery system  • Patient instructions ‘I think each doctor has their own ways of educational links they call up… which you might not put in your notes but you should tell the patient verbally or put it in instructions [section of AVS]’. • Medication list ‘I think it’s the utility here having the med list on the after visit summary as well as in our patient portal’.  • Too much information ‘Some of [the AVS] are way too long, and even if you wrote down the language to the sixth grade level… some of these things read like War and Peace’.  View Large Next steps: overcoming barriers The key informants unanimously felt that AVS could improve clinical outcomes if utilized properly. When asked about future changes in the Meaningful Use program related to the AVS, interviewees did not foresee abandoning this document in practice. I think [the AVS] could be really important. I don’t think it’s important the way it’s used now, but I think it could be extremely important and extremely helpful. Moving forward, improvements in AVS use were related to the following themes: Team-based workflows Interviewees expressed that non-clinicians are well positioned to use the AVS with patients for operational next steps (like follow-up appointments). Within the one clinic with a standard MA workflow already in place, the interviewee commented, ‘MAs really like it. They like being part of the process of closing the loop and just helping the patient with those final details’. In addition, MAs or other staff could likely counsel related to lifestyle (such as diet or exercise) or other content with additional training and/or support. For example, one interviewee stated that the MA could use the AVS more effectively, but only with guidance from a provider: The problem is the MA would have to know which part of the After Visit Summary to go over because you don’t want to go over more than like a couple of things, because people are not going to walk on practice remembering eight or 10 things. Focus within the AVS In addition, there were many comments related to the idea that the AVS ‘seems to want to serve too many purposes’. In addition to multiple content areas like medications and diagnoses, clinicians also wrote in personalized instructions in varying ways. Therefore, the current AVS format was long and complex, especially to find specific necessary information from a single visit. Increased ability to customize the AVS in straightforward ways was viewed as critical. Tailoring by language and literacy Because the AVS was not available in non-English languages or with low-literacy text, interviewees requested adjusting content to improving patient communication. For example: For our folks that speak other languages, we are really limited in terms of written instructions we can provide for them. I don’t have any good workaround for that. If there’s a way to do like the med chart with pictures, not just all words… [The AVS is] basically four pages of words. Conclusions Among a small amount of published literature on the topic, we found that patients perceive AVS positively, but few appear to routinely refer to the document after the visit. Clinicians surveyed in the published literature were less satisfied than patients with AVS. Moreover, beyond this literature review of existing research, we also conducted our own qualitative investigation among primary care leaders about their perceptions of AVS in high-performing clinical practices. Among these key informant interviewees, we found similar implementation experiences across a varied group of primary care practices. While a hardcopy AVS were distributed in virtually all encounters, there was uncertainty about whether patients used AVS and a lack of routine practice to educate patients about AVS content. The customized patient instructions section was viewed as most useful within the AVS, but this could be buried in the midst of other content. Despite such challenges, interviewees expressed overall positivity about the potential of the AVS to improve patient understanding in the future. This is the first study to our knowledge that comprehensively studied the current use of AVS in real-world practices in combination with stakeholder perceptions across multiple health care settings about the best ways to improve AVS use for maximum impact. While interviewees in this study provided recommendations for improving the content of AVS to improve implementation, any content changes would be insufficient without additional workflows to support patient use and understanding. Future research is needed to understand whether and how AVS contribute to improved patient outcomes (e.g. understanding/retention, clinical outcomes) and to directly compare the impact of different workflows of AVS distribution. There is no published literature about electronic delivery of AVS through online patient portals, or comparisons of digital versus printed distribution. In addition, there is a need for research to compare workflows of teach-back (36) using AVS to determine the best modes for patient understanding and retention. Our study supports previous research on patient–provider communication. For example, patients in our literature review expressed high interest in access to information from their medical encounters via AVS, which is similar to many other studies on patient interest in and satisfaction with access to their online medical record information (37,38). Moreover, our findings support previous work that that training and/or tools can improve in-person communication (39), especially for vulnerable patient populations (40,41), but this is the first study to our knowledge of whether the AVS is being used for patient education and teach-back. Moreover, implementation of these improved communication strategies into real-world settings requires overcoming obstacles such as under-staffing and insufficient time during visits. There are several limitations of this study. First, the literature review may have missed studies using a structured process for delivering patient education materials at the conclusion of visits or hospitalizations. In addition, our qualitative sample was small and is not broadly generalizable, and most participants gave feedback on a single EHR product. In addition, the interviewees were all providers without any patient representation. However, we reached thematic saturation with this small but diverse set of interviewees across multiple health care settings. Moving forward, patient summaries of information like AVS will likely continue to play a role in primary care. AVS utility for both patients and clinicians will likely increase as content and design are improved. The growth of the patient-centred medical home and the emphasis on team-based care will likely result in new roles and responsibilities for communication with patients, and AVS may take centre stage in workflow redesign. Over time, as federal policies and incentives for EHR use change, AVS will survive only if clinicians and patients find them relevant and useful. Declaration Funding: The Roundtable on Health Literacy of the National Academies of Sciences, Engineering, and Medicine provided support for our investigations into AVS. CRL is supported by AHRQ R00HS022408. Conflict of interest: The authors report no conflicts of interest. Acknowledgement We would like to thank all the people we interviewed who contributed their time to this project. References 1. Centers for Medicare and Medicaid Services. 2015 Meaningful Use Definitions and Objectives. http://www.healthit.gov/providers-professionals/meaningful-use-definition-objectives ( accessed on 14 May 2015). 2. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med  2010; 363: 501– 4. Google Scholar CrossRef Search ADS PubMed  3. Centers for Medicare and Medicaid Services. 2014 Hospitals Participating in the CMS EHR Incentive Programs. http://dashboard.healthit.gov/quickstats/pages/FIG-Hospitals-EHR-Incentive-Programs.php ( accessed on 14 May 2015). 4. Centers for Medicare and Medicaid Services. 2014. Office-based Health Care Professional Participation in the CMS EHR Incentive Programs. http://thehealthcareblog.com/blog/2016/01/19/ehr-incentive-programs-where-we-go-from-here/ ( accessed on 14 May 2015). 5. Slavitt A, DeSalvo K. EHR incentive programs: where we go next. The CMS Blog. 6. Stevenson FA, Barry CA, Britten N, Barber N, Bradley CP. Doctor-patient communication about drugs: the evidence for shared decision making. Soc Sci Med  2000; 50: 829– 40. Google Scholar CrossRef Search ADS PubMed  7. Roter DL, Hall JA. Studies of doctor-patient interaction. Annu Rev Public Health  1989; 10: 163– 80. Google Scholar CrossRef Search ADS PubMed  8. Sarkar U, Schillinger D, Bibbins-Domingo Ket al.   Patient-physicians’ information exchange in outpatient cardiac care: time for a heart to heart? Patient Educ Couns  2011; 85: 173– 9. Google Scholar CrossRef Search ADS PubMed  9. Hummel J, Evans P. Providing Clinical Summaries to Patients after Each Visit: A Technical Guide . Seattle, WA: Qualis Health, 2012. 10. Peek ME, Odoms-Young A, Quinn MTet al.   Race and shared decision-making: perspectives of African-Americans with diabetes. Soc Sci Med  2010; 71: 1– 9. Google Scholar CrossRef Search ADS PubMed  11. Aboumatar HJ, Carson KA, Beach MC, Roter DL, Cooper LA. The impact of health literacy on desire for participation in healthcare, medical visit communication, and patient reported outcomes among patients with hypertension. J Gen Intern Med  2013; 28: 1469– 76. Google Scholar CrossRef Search ADS PubMed  12. Institute of Medicine. Health Literacy: A Prescription to End Confusion . Washington, DC : National Academies Press (US), 2004. 13. Schillinger D. Literacy and health communication: reversing the ‘inverse care law’. Am J Bioeth  2007; 7: 15– 8. Google Scholar CrossRef Search ADS PubMed  14. Schillinger D, Bindman A, Wang F, Stewart A, Piette J. Functional health literacy and the quality of physician-patient communication among diabetes patients. Patient Educ Couns  2004; 52: 315– 23. Google Scholar CrossRef Search ADS PubMed  15. Sinsky CA, Willard-Grace R, Schutzbank AMet al.   In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med  2013; 11: 272– 8. Google Scholar CrossRef Search ADS PubMed  16. Wagner EH, Gupta R, Coleman K. Practice transformation in the safety net medical home initiative: a qualitative look. Med Care  2014; 52( 11 suppl 4): S18– 22. Google Scholar CrossRef Search ADS PubMed  17. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health  2000; 23: 334– 40. Google Scholar CrossRef Search ADS PubMed  18. Salmon C, O’Conor R, Singh Set al.   Characteristics of outpatient clinical summaries in the United States. Int J Med Inform  2016; 94: 75– 80. Google Scholar CrossRef Search ADS PubMed  19. Bodenheimer T, Laing BY. The teamlet model of primary care. Ann Fam Med  2007; 5: 457– 61. Google Scholar CrossRef Search ADS PubMed  20. Jiggins K. A content analysis of the meaningful use clinical summary: do clinical summaries promote patient engagement? Prim Health Care Res Dev  2015; 17: 1– 14. 21. Kanter M, Martinez O, Lindsay G, Andrews K, Denver C. Proactive office encounter: a systematic approach to preventive and chronic care at every patient encounter. Perm J  2010; 14: 38– 43. Google Scholar PubMed  22. Colorafi K, Moua L, Shaw M, Ricker D, Postma J. Assessing the value of the meaningful use clinical summary for patients and families with pediatric asthma. J Asthma  2017; 55:1–9. 23. Tang PC, Newcomb C. Informing patients: a guide for providing patient health information. J Am Med Inform Assoc  1998; 5: 563– 70. Google Scholar CrossRef Search ADS PubMed  24. Neuberger M, Dontje K, Holzman Get al.   Examination of office visit patient preferences for the after-visit summary (AVS). Perspect Health Inf Manag  2014; 11: 1d. Google Scholar PubMed  25. Pavlik V, Brown AE, Nash S, Gossey JT. Association of patient recall, satisfaction, and adherence to content of an electronic health record (EHR)-generated after visit summary: a randomized clinical trial. J Am Board Fam Med  2014; 27: 209– 18. Google Scholar CrossRef Search ADS PubMed  26. Black H, Gonzalez R, Priolo Cet al.   True “meaningful use”: technology meets both patient and provider needs. Am J Manag Care  2015; 21: e329– 37. Google Scholar PubMed  27. Emani S, Healey M, Ting DYet al.   Awareness and use of the after-visit summary through a patient portal: evaluation of patient characteristics and an application of the theory of planned behavior. J Med Internet Res  2016; 18: e77. Google Scholar CrossRef Search ADS PubMed  28. Federman AD, Sanchez-Munoz A, Jandorf Let al.   Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design. J Am Med Inform Assoc  2016; 24: e61– 68. 29. Belyeu BM, Klein JW, Reisch LMet al.   Patients’ perceptions of their doctors’ notes and after-visit summaries: a mixed methods study of patients at safety-net clinics. Health Expect  2017; 21: 485– 93. Google Scholar CrossRef Search ADS PubMed  30. Clarke MA, Moore JL, Steege LMet al.   Toward a patient-centered ambulatory after-visit summary: identifying primary care patients’ information needs. Health Soc Care Community  2017; 43: 1– 16. 31. Emani S, Ting DY, Healey Met al.   Physician perceptions and beliefs about generating and providing a clinical summary of the office visit. Appl Clin Inform  2015; 6: 577– 90. Google Scholar CrossRef Search ADS PubMed  32. Anbar RD, Anbar JS, Hashim MA. Use of an after-visit summary to augment mental health of children and adolescents. Clin Pediatr (Phila)  2015; 54: 1009– 11. Google Scholar CrossRef Search ADS PubMed  33. Dehen RI, Carter SU, Watanabe M. Impact of after visit summaries on patient return rates at an acupuncture and oriental medicine clinic. Med Acupunct  2014; 26: 221– 5. Google Scholar CrossRef Search ADS PubMed  34. Jiggins K. A content analysis of the meaningful use clinical summary: do clinical summaries promote patient engagement? Prim Health Care Res Dev  2016; 17: 238– 51. Google Scholar CrossRef Search ADS PubMed  35. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and healthcare systems. J Am Med Inform Assoc  2015; 22: 465–71. 36. Use the Teach-Back Method: Tool #5. 2015; http://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/literacy-toolkit/healthlittoolkit2-tool5.html (accessed on 4 February 2018). 37. Ralston JD, Carrell D, Reid Ret al.   Patient web services integrated with a shared medical record: patient use and satisfaction. J Am Med Inform Assoc  2007; 14: 798– 806. Google Scholar CrossRef Search ADS PubMed  38. Tieu L, Sarkar U, Schillinger Det al.   Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res  2015; 17: e275. Google Scholar CrossRef Search ADS PubMed  39. Ha Dinh TT, Bonner A, Clark R, Ramsbotham J, Hines S. The effectiveness of the teach-back method on adherence and self-management in health education for people with chronic disease: a systematic review. JBI Database System Rev Implement Rep  2016; 14: 210– 47. Google Scholar CrossRef Search ADS PubMed  40. Wolff K, Cavanaugh K, Malone Ret al.   The diabetes literacy and numeracy education toolkit (DLNET): materials to facilitate diabetes education and management in patients with low literacy and numeracy skills. Diabetes Educ  2009; 35: 233– 45. Google Scholar CrossRef Search ADS PubMed  41. White RO, Eden S, Wallston KAet al.   Health communication, self-care, and treatment satisfaction among low-income diabetes patients in a public health setting. Patient Educ Couns  2015; 98: 144– 9. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Family PracticeOxford University Press

Published: May 28, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off