Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Algorithm-based decision support for symptom self-management among adults with Cancer: results of usability testing

Algorithm-based decision support for symptom self-management among adults with Cancer: results of... Background: It is essential that cancer patients understand anticipated symptoms, how to self-manage these symptoms, and when to call their clinicians. However, patients are often ill-prepared to manage symptoms at home. Clinical decision support (CDS) is a potentially innovative way to provide information to patients where and when they need it. The purpose of this project was to design and evaluate a simulated model of an algorithm- based CDS program for self-management of cancer symptoms. Methods: This study consisted of three phases; development of computable algorithms for self-management of cancer symptoms using a modified ADAPTE process, evaluation of a simulated model of the CDS program, and identification of design objectives and lessons learned from the evaluation of patient-centered CDS. In phase 1, algorithms for pain, constipation and nausea/vomiting were developedbyanexpertpanel.Inphase 2, we conducted usability testing of a simulated symptom assessment and management intervention for self-care (SAMI-Self-Care) CDS program involving focus groups, interviews and surveys with cancer patients, their caregivers and clinicians. The Acceptability E-scale measured acceptability of the program. In phase 3, we developed design objectives and identified barriers to uptake of patient-centered CDS based on the data gathered from stakeholders. Results: In phase 1, algorithms were reviewed and approved through a consensus meeting and majority vote. In phase 2, 24 patients & caregivers and 13 clinicians participated in the formative evaluation. Iterative changes were made in a simulated SAMI-Self-Care CDS program. Acceptability scores were high among patients, caregivers and clinicians. In phase 3, we formulated CDS design objectives, which included: 1) ensure patient safety, 2) communicate clinical concepts effectively, 3) promote communication with clinicians, 4) support patient activation, and 5) facilitate navigation and use. We identified patient barriers and clinician concerns to using CDS for symptom self-management, which were consistent with the chronic care model, a theoretical framework used to enhance patient-clinician communication and patient self- management. Conclusion: Patient safety and tool navigation were critical features of CDS for patient self-management. Insights gleaned from this study may be used to inform the development of CDS resources for symptom self-management in patients with other chronic conditions. Keywords: Rule-based clinical decision support, Symptom management, Patient engagement, Patient self-management, Cancer * Correspondence: [email protected] The Phyllis F. Cantor Center, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, MA 02115, USA Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 2 of 20 Background called the clinic to ask for help. Ruland et al. [20] devel- Patient-centered healthcare is one of six aims to improve oped a Web-based resource that assisted patients with the United States healthcare system [1]. One facet of assessing symptoms, finding information, communicat- patient-centered care is engaging patients and encour- ing with clinic personnel and provided self-management aging self-management, especially in the context of advice. This tool was evaluated and showed a slight de- chronic illness. Self-management programs that provide crease in symptom distress for patients [20] but the CDS coaching and education, and are supported by timely component was not evaluated independently for its ef- information, facilitate patient engagement [2, 3]. fect. A follow-up study determined that communication Given the frequency of symptoms from cancer and its with nurses was the most valued component [25]. treatment, it is essential that patients (and their caregivers) The present study extends the literature by reporting understand how to self-manage symptoms and when to call on the design and formative evaluation of an algorithm- their clinicians for advice. McCorkle and colleagues [3–5] based simulated model of a patient-centered CDS pro- noted that most patients with cancer try to self-manage gram that facilitates cancer symptom self-management, their care and that developing system-level interventions to provides advice on when patients should contact their support self-management is essential for quality cancer clinicians, and includes coaching information about what care. Evidence-based strategies to assist patients with self- to tell them. The goals of the project were to develop management include: education, telephone consultations, computable algorithms for pain, constipation and nau- Internet tools for tracking disease-specific parameters, and sea/vomiting in phase 1, conduct iterative usability test- coaching [3, 6–13]. Such interventions targeted information ing of the simulated CDS program called the Symptom management, medications, psychological consequences of Assessment and Management Intervention for Self- illness, lifestyle, social support, communication, accessing Care (SAMI-Self-Care) with patients, their caregivers services, and setting goals [14–16]. Self-management inter- and clinicians in phase 2, and develop design objectives ventions decreased the severity of pain [17], fatigue [18], and identify barriers to uptake of patient-centered CDS and depression [19]. Web-based interventions, featuring based on the data gathered from stakeholders, which in- self-monitoring, education and coaching of patients com- cluded members of an expert panel, patients, caregivers, bined with summaries of patient-reported data delivered to and their clinicians in phase 3. clinicians, led to improvement in symptom distress and quality of life [12, 16, 20]. Evidence suggests that reporting Methods symptoms to clinicians alone may not be sufficient. A pro- We employed a convergent, parallel, mixed methods de- ject in which patients receiving chemotherapy reported on sign, [26] in which we collected qualitative and quantitative symptoms using an automated phone system failed to im- data in parallel, analyzed it separately, and then merged all pact care. Even though symptoms of moderate to severe in- data during interpretation, to evaluate the acceptability of a tensity were reported, clinicians did not provide patients simulated algorithm-based CDS program for cancer with management guidance [10]. symptom self-management. This study was conducted at Considering the variability of clinician response to Dana-Farber Cancer Institute (DFCI) and approved by the symptom reports, an optimal approach to enhance self- Institutional Review Board (IRB), Protocol number-12-300. management includes providing patient-specific, real- time, actionable information. Supplying Clinical Decision Phase 1: Algorithm-based CDS intervention development Support (CDS) directly to patients may enhance their In order to develop the patient-centered CDS algorithms, ability to self-manage symptoms. CDS “provides individ- we used a modified ADAPTE process [27–29]consisting uals with person-specific information, intelligently fil- of five steps; 1) identify expert panels to develop and tered or presented at appropriate times, to enhance evaluate the computable algorithms, 2) develop groups to health” [21]. Four studies have described rule-based CDS work on each symptom and synthesize the literature, 3) tools for cancer patients. [12, 20, 22, 23] In two studies, convene groups to translate evidence-based information CDS was used to identify the presence of a symptom into computable algorithms for each symptom, 4) conduct using an algorithm with a single decision node that gen- peer review on the content of the algorithms before con- erated general recommendations for symptom manage- vening a multidisciplinary consensus meeting, and 5) hold ment [12, 22]. In studies by Berry and colleagues [24] a multidisciplinary panel meeting to review, modify and and Weaver, [23] the focus was self-care support and approve the algorithms. identifying symptoms to report to clinicians. Berry and The study team identified areas of expertise that were colleagues [12] found that Web-based coaching of pa- critical for developing and evaluating an algorithm-based tients to report symptom experiences verbally resulted CDS tool for symptom self-management. Individuals with in more frequent reports during clinic visits, but the in- the desired expertise were identified through existing pro- vestigators did not document how often a participant fessional relationships or through recommendations of Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 3 of 20 colleagues. The identified individuals were invited to join is recommended as a standard approach to the develop- the panel by the principal investigator (MEC) and in- ment process as it allows one to ascertain whether partici- cluded: stakeholders with expertise in clinical care (oncol- pants understand items on a questionnaire and allows for ogists, palliative care experts, psychiatrists, and oncology iterative changes before finalizing questions [34]. nurses), information systems (experts in CDS, patient data This patient-vetted algorithm content was given to the collection, health communication, and graphic design), Dana-Farber Harvard Comprehensive Cancer Center care delivery process (experts in workflow, quality im- Health Communication Core. The Core developed a provement, and health equity), as well as patients and mockup of the SAMI-Self-Care using interactive PDF caregivers. files to simulate the function of the CDS tool based on Our expert panel drew from evidence-based resources expert panel input and clinical informatics literature [35, and worked with CDS experts to develop computable al- 36]. The PDF files imitated the functionality of a Web- gorithms that would enable self-management of pain, site and allowed participants to experience the “look and constipation, and nausea/vomiting. Overall, the expert feel” of a potential operational CDS system on an iPad. panel met four times for 4 h each time to review the al- This approach provided a practical and economical ap- gorithms developed by the research team. The symp- proach to usability testing, especially during formative toms chosen were the most common reasons for urgent stages of development when iterative refinement is ne- care among cancer patients and identified as important cessary. The Health Communication Core also designed targets for symptom management by patients and their reports containing self-management recommendations caregivers in a previous study that explored patient pref- to elicit feedback from participants regarding report erences for CDS to enhance clinical care [30]. Our content and appearance. A series of focus groups or in- process of developing algorithms was based on our pre- terviews were conducted to elicit feedback about the vious work that adapted evidence-based guidelines for CDS program. Using a multi-methods approach (i.e. clinicians [31]. Similar to our previous work, multi- focus groups and interviews) is appropriate when the in- disciplinary groups were formed for each symptom and tent is to gather information that is focused on specific serial meetings were held to develop the initial algo- goals and questions, which in this case was participant rithms. Once the algorithms were completed, a group response about the usability of an algorithm-based CDS meeting with the members of the research team and the program [37]. We used a combination of methods to expert panel was held to review, modify and approve the provide flexibility to participants to be able to participate algorithms. For each symptom, characteristics were iden- in the usability sessions. We used the same facilitator to tified to direct self-management. Characteristics of pain conduct the focus groups and interviews and used an included descriptors of neuropathic or somatic pain, and interview guide to elicit information about the usability a measure of intensity (moderate, severe). Features of and acceptability of the SAMI-Self-Care CDS program. nausea/vomiting included chemotherapy-induced, Based on feedback from the expert panel, patients, radiation-induced, chemoradiation-induced, acid reflux, caregivers, and clinicians, we iteratively created three vertigo, recent narcotic increase, anticipation of chemo- successive versions of the SAMI-Self-Care program. therapy, and constipation. Characteristics of constipation Figures 1, 2, 3, 4 show the iterative development of the included frequency and consistency of stool. self-management report that was generated for partici- pants based on responses to the questions and Phase 2: Usability testing algorithm-based CDS. Feedback from the usability test- In order to refine the algorithm content and the simulated ing was used to inform the changes that were made to SAMI-Self-Care tool display and function iteratively, we the CDS program and the self-management report (see used a collaborative-participatory approach. This ap- results). proach engaged stakeholders in a shared co-designing process and helped ensure that the product will meet real- Participant recruitment life needs and be adopted [32, 33]. The team formulated a Patients were identified through medical records at DFCI scenario for a symptom management dilemma that would under a waiver of consent authorization, and names were allow patient users to traverse the self-care management submitted to their clinicians for approval to contact pa- algorithms so that self-management recommendations tients, as required by the IRB. Eligible patients were age ≥ could be generated. Cognitive testing was performed for 18, English speaking, and had received cancer treatment each algorithm by reviewing the content displayed on an within the past 6-months. All approved patients were sent iPad with patients and their caregivers to ensure that these an initial letter describing the study and inviting them to questions were clear, understandable, and relevant, and participate. Patients received two follow-up calls if they that the response options were appropriate prior to initiat- didn’t respond to the letter. Patients identified caregivers ing the iterative usability testing. Cognitive testing of items to invite to participate in the study. Eligible caregivers Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 4 of 20 Fig. 1 Initial SAMI-Self Care report were age ≥ 18 years and provided care to a patient who re- participants and were held in a convenient conference ceived cancer treatment within the past 6-months. Delib- room at DFCI. Meetings for clinicians lasted about 30 min erate efforts were made to sample across a range of whereas those for patients and caregivers lasted 45– diagnoses, educational backgrounds, ages, genders, and 60 min. One symptom was addressed per session. Using a races. All participants signed consent forms at the time of script, the facilitator queried patients, caregivers and clini- participation. Each participant received $50 for complet- cians about the algorithm content of the SAMI-SC pro- ing the study. grams for constipation, pain, and nausea/vomiting, and Clinicians in ambulatory oncology included physicians, asked about the “look and feel” of the materials for pain nurse practitioners, physician assistants, and registered and nausea/vomiting. The topics for patients and care- nurses. Clinicians were identified through clinic rosters. givers focused on the: 1) visual appeal, 2) format and navi- An invitation to participate in the study was emailed to gation, 3) understandability of written content and eligible clinicians. Two follow-up emails were sent if no terminology, and 4) wording of self-management sugges- response was received. Clinicians were recruited for either tions. The topics for the clinicians focused on 1) a review focus groups or interviews depending on their schedules. of the patient self-management algorithms, 2) review of Clinicians received an invitation letter describing the study the simulated SAMI-Self-Care CDS program, and 3) feed- but signed consent was waived by the IRB. Each clinician back about ways to improve the algorithms and/or simu- received $100 for completing the study. lated model. Patients/caregivers and clinicians reported demographic data using a standardized survey. Focus group and interview processes The number of sessions per symptom was determined Focus groups and interviews were held separately for cli- by evidence of content saturation and absence of new nicians and patients/caregivers. Focus groups included up common responses [38]. With patients/caregivers, we to five participants; interviews included one or two conducted six sessions for pain, seven sessions for nausea/ Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 5 of 20 Fig. 2 First revision for SAMI-Self Care report vomiting, and three sessions for constipation. With clini- clinician participants regarding: 1) understandability of cians, we conducted two sessions for pain, three sessions language, 2) helpfulness of suggestions provided for for nausea/vomiting, and one session for constipation. All symptom self-management, 3) usefulness of reports, and discussions were audio-recorded and transcribed. 4) overall satisfaction with the program. The a priori tar- get for acceptability of SAMI-Self-Care was a mean Survey instruments score of ≥4 on each item since this score indicated a The Acceptability E-scale measured the acceptability of positive acceptability response. In addition to the the CDS program [39]. This scale has been used in adult Acceptability E-scale, participants were asked what other patients with various types of cancers from medical on- types of information would have been helpful for symp- cology, radiation oncology and stem cell transplant to tom self-management. The scale took less than 2 min to measure acceptability of Web-based systems, including complete and was written at a 5th grade reading level. computerized symptom and quality of life assessments, patient educational materials and patient-centered deci- Analysis sion support. Psychometric testing was conducted in 627 Patient/caregiver and clinician demographic data were adults with cancer [40, 41]. Cronbach alpha reliability summarized as descriptive statistics. Acceptability coefficient was 0.75 and factor analysis revealed that the E-scale results were summarized as item means and scale was unidimensional [39]. Five-point Likert-type standard deviations. items (1–5; larger number equals higher acceptability Audio-recorded session content was transcribed and and a score of 3 indicates a neutral response) were pre- reviewed by three study team members (MEC, DFL, sented to elicit feedback from both patient/caregiver and MMN), coded inductively, and grouped into common Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 6 of 20 Fig. 3 Second revision for SAMI-Self Care report responses to identify revisions that needed to be made in organize the data collected from the expert panel, patients, the CDS tool. Common responses were then combined caregivers and clinicians and discuss the themes that were and reconciled though discussion. A list of suggested re- identified through inductive content analysis [42]. These visions to improve SAMI-Self-Care algorithms and the data were then shared and discussed with other members simulated model was created and ranked based on audio of the research team to generate a final list of design ob- files and notes. Critical revisions were implemented im- jectives, patient barriers, and clinician concerns to use of mediately, while other less critical revisions were moni- the CDS program. tored for repetition. This process was repeated, after each iteration of CDS tool development, until no new Analyses responses or suggestions for changes emerged from the Audio-recorded session content was transcribed and usability testing. Phase 2 analyses focused on the revi- reviewed by three study team members (MEC, DFL, sions that needed to be make to the CDS tool. MMN), coded, and grouped into themes. Themes were then combined and reconciled though discussion. In- Phase 3: Design objectives and barriers to uptake of ductive analyses of the coded qualitative data for phase 3 patient-centered CDS focused on interpreting patient barriers to using the Once all the data were collected and the usability testing CDS program, defining clinician concerns about use of completed, three investigators (MEC, DFL, MMN) CDS by patients/caregivers, and identifying strategies to reviewed all of the qualitative data in order to identify de- overcome the areas of concern. It was noted during the sign objectives and barriers to uptake of the patient- analysis that the themes that emerged from the qualita- centered CDS program. Eight sessions were needed to tive data were consistent with the Chronic Care Model Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 7 of 20 Fig. 4 Third revision for SAMI-Self Care report [3, 43]. Thus, five design objectives detailed below (in non-pharmacological and self-management approaches Results) were developed to address these concerns. As a so that a comprehensive approach to enable self- result, content analysis was then used to regroup the management would be embedded into the algorithms. themes to fit within the Chronic Care Model. Once the development of the algorithms and usability testing was completed, a multidisciplinary consensus Results meeting was held with the expert panel, research team and Phase 1: Rule-based CDS intervention development co-investigators to review, modify and approve the final Nine potential participants were approached for mem- versions. In order to iteratively improve the program, the bership in the expert panel, and all accepted the invita- expert panel focused on several issues. The primary issue tion. Altogether, 16 members of the research team and was patient safety. Safety was addressed by identifying po- expert panel contributed to algorithm development. tential serious causes of each symptom and directing pa- Work sessions were conducted in person and by Web tients to seek contact with their clinicians. Accordingly, all conferencing so that the flow charts could be displayed three algorithms began with the identification of “red flag more easily for discussion. Branching logic was used to symptoms” that caused patients a forced exit of the CDS develop the algorithms and these were displayed as flow tool and a directive to call a clinician immediately for guid- charts that were shared among all team members. Re- ance. A second issue was how to make the tool accessible search staff provided support to access current guide- to patients across a range of health literacies. Our solution lines that were available for the targeted symptoms and was to provide information that patients could elect to re- to assist with literature reviews. Each work group had view or skip based on their needs. A third issue was how someone assigned who was expert in pharmacological, to provide medication-related advice that was aligned with Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 8 of 20 therapies recommended by a patient’s clinician. To address Table 2 Cancer Patient and Caregiver Demographics (N = 24) this issue, we asked the patient whether the medication we Characteristic n % were recommending for the problem (e.g. senna for consti- Role pation) was approved by his/her clinician. If it was ap- Patient 15 63 proved, we inquired whether the patient had taken that Caregiver 9 37 medication. If not, the patient was advised to take it. If the Gender patient had not been prescribed the recommended therapy, Female 13 54 he or she was advised to contact his/her clinician and inquire if this therapy could be appropriate. The resulting Age algorithms had a large number of decision nodes ranging Median/Range 55 21–69 from 51 to 257 (moderate and severe pain combined). A <50 5 21 decision node is a point in the algorithm where the logic ≥ 50 19 79 branches into two or more directions. A higher number of Race decision nodes reflect greater algorithmic complexity [44]. Caucasian 20 83 Table 1 provides information about the number of decision nodes and red flag questions that were present within each Black/African American 2 8 symptom algorithm. A simple majority vote was taken at Other 2 8 the end of the discussion surrounding each symptom algo- Ethnicity rithm to determine whether agreement was reached regard- Hispanic 4 17 ing modifications and approval. Table 1 provides an Non-Hispanic 18 75 overview of the number of red flags that were required for Did Not Report 2 8 each symptoms and the complexity of the algorithms. Education Phase 2: Usability testing High School or Less 3 12 Participant sample Some College or More 21 88 One-hundred-and-three patients were screened to identify Income the 24 patients and caregivers for study inclusion (Table 2). $49,999 or Less 4 17 Twenty-six patients were deemed ineligible by their clini- $50,000 or More 19 79 cians due to the incapacitating nature of their condition and 14 did not meet other eligibility criteria or could not Did Not Report 1 4 be contacted. Seventeen potential subjects declined par- Cancer Type (n = 15) ticipation; 22 indicated interest but were unable to attend Hematologic Malignancies 5 33 scheduled sessions or did not respond to scheduling re- Solid Tumor Malignancies 10 67 quests. Participating patients and caregivers were predom- Types of Solid Tumor inately Caucasian, almost evenly split between men and Breast 1 7 women and most had some college level education. Forty-four clinicians were contacted to identify 13 who Gastrointestinal 3 20 participated in focus groups and interviews (Table 3). Of Genitourinary 1 7 the clinicians who did not participate, 20 did not Gynecologic 1 7 respond, 2 declined, and 9 indicated interest but were Head and Neck 1 7 unable to attend scheduled sessions. Clinician partici- Neuro-oncology 1 7 pants were predominately female and Caucasian. Thoracic 2 13 Internet Use to Obtain Health Information Table 1 Number of decision nodes and red flag questions in Never/Rarely 0 0 each symptom algorithm Sometimes 10 42 Symptom Decisional Nodes Red Flag Questions Nausea & Vomiting 54 6 Often/Very Often 13 54 Pain 257 2 Missing 1 4 Severe Pain 125 Moderate Pain 122 Focus groups and interviews Patients and caregivers provided feedback about the vis- Constipation 51 4 a ual appeal of the program, especially related to the color Red flag questions are the same for both the moderate and severe pain pathways scheme, text density, and font size. Participant feedback Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 9 of 20 Table 3 Oncology Clinician Demographics (N = 13) clinicians immediately, whereas others felt that it was ac- ceptable to use the suggestions and then call their clini- Characteristic n % cians for refractory pain. Tables 4 and 5 provide an Gender overview of comments provided by participants to im- Female 11 85 prove the usability of the program. Race Caucasian 11 84 Black/African American 1 8 Acceptability surveys Participants completed surveys regarding the accept- Asian 1 8 ability of SAMI-Self-Care for “nausea/vomiting” and Ethnicity “pain” after each round of system development and Non-Hispanic 11 85 viewing the simulated interface. Nine patients and Did Not Report 2 15 their caregivers completed surveys, four assessing pain Training and five assessing nausea and vomiting. Two rounds Physicians 3 23 of assessment for usability and acceptability were con- ducted with patients to reach or surpass the predeter- Nurse Practitioners 4 31 mined threshold for acceptability. In general, patients Physician Assistants 2 15 found the system easy to understand and helpful for Registered Nurses 4 31 self-management. Cancer Specialty Area Thirteen clinicians completed surveys, nine assessed Hematologic Malignancies 4 31 pain and three assessed nausea and vomiting, and one Solid Tumor Malignancies 9 69 assessed constipation. Two rounds of assessment for us- ability and acceptability were conducted with clinicians. Types of Solid Tumor The simulated program for nausea/vomiting and consti- Gastrointestinal 2 15 pation reached and surpassed the threshold for accept- Genitourinary 1 8 ability. However, the helpfulness of suggestions for the Neuro-oncology 1 8 pain algorithm was scored lower than our target thresh- Head and Neck 1 8 old of 4.0. For the most part, clinicians also viewed the Thoracic 2 15 system for pain as usable and helpful. Subsequent dis- cussions revealed that some clinicians felt that patients Radiation Oncology 1 8 with a pain score ≥ 9 should contact their clinicians and General Practice 1 8 not pursue self-management. The algorithm was subse- Prior Use of Patient-Focused Information Tools quently changed to reflect that patients with severe pain, Never/Rarely 8 61 should contact their clinicians, but time limitations pre- Sometimes/Often/Very Often 5 39 vented the modified algorithm from being reevaluated. sought to make the display inviting and easy to read. Some of the issues raised by clinicians who viewed the Phase 3: Design Objectives and Barriers to Patient- program were similar to issues identified by patients and centered CDS their caregivers. For example, clinicians made sugges- Design objectives tions about use of language, decreasing the density of From patient, caregiver, and stakeholder feedback, we text and providing graphic images to enhance under- identified five design objectives that were relevant for standing. For the most part, patients, caregivers and cli- the development of algorithm-based patient-centered nicians had similar comments to enhance the usability CDS, which included: ensure patient safety, communi- of the program. However, clinicians provided additional cate clinical concepts effectively, promote communica- feedback about the content of the algorithms, and to en- tion with clinicians, support patient activation, and sure patient safety. They suggested adding questions at facilitate navigation (see Table 6). This table includes the beginning of the algorithms that would identify po- information about the design objective, specific stake- tentially dangerous symptoms that needed immediate at- holder barriers related to this objective, solutions that tention and prompt patients to exit the program and call were identified to address the barrier and changes that their clinicians. We also found that clinicians disagreed were made in the user interface. A discussion surround- about what were best practices for self-management in ing each of these design objectives is found in the pain management. Some clinicians felt that patients with following section followed by some barriers to patient- pain levels of ≥ 9 should not self-manage and call their centered CDS that emerged from our findings. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 10 of 20 Table 4 Results of Usability Testing for the SAMI-Self-Care CDS Program: Patient Perspectives Usability Testing Themes CDS Tool Content* CDS Tool Examples Component** Visual appeal Comments about introductory pages Visual appeal “Some pages seem overwhelming.” that had a lot of content and design Understanding Pain severity question Written content “What does ‘bearable pain’ mean?” of terminology and terminology Medication questions for all Written content “What does ‘able and willing’ [to take a medication] mean?” symptoms and terminology Nausea and vomiting question Written content “Position change- does that mean when I lift my head up?” and terminology Medication questions Written content “Unclear about the word ‘dose’ in the questions.” and terminology Medication question Written content “Did you take the dose you were due for? Does that mean and terminology the dose time already passed or the next dose I am due for?” Pain quality question Written content “Define what type of pain you are referring too”,isit “pokey and terminology pain, electrical current, shock pains burning pains, etc.” Pain quality question Written content “I wouldn’t have categorized numbness as pain...I’m glad and terminology it’s there.” Pain severity question Written content “I like ‘faces’ as part of the pain scale. They make the pain and terminology measure more clear.” Constipation definition Written content “Definitions were too wordy, for example, constipation and terminology definition had too much information.” Pain medication list Written content “I have trouble understanding meaning or relevance to and terminology words such as Morphine or Opioids.” Constipation medication list Written content “Is Senna tea the same as Senna medication?” and terminology Nausea and vomiting red flag safety Written content “What are two glasses of water per day?” questions and terminology All symptom medication questions Written content “Need to add a time frame to the question: ‘Did you take and terminology your medication?’” General content related to Written content Simplicity of terminology required for some patients with introduction of program and terminology little medical sophistication makes clinical concepts difficult and definition of all symptoms to communicate and can be tedious for more medically sophisticated patients. Nausea and vomiting questions Written content “Why are you asking me about acid reflux and then position and terminology change? Are they related?” Nausea and vomiting red flag safety Written content “Why is bone marrow transplant question asked?” questions and terminology Pain red flag safety questions Written content Reason for why some questions are asked is not and terminology understood, e.g., “Not everyone has back pain.” Nausea and vomiting questions Written content “Some patients may be getting some agents that aren’t and terminology considered chemotherapy but the patient thinks they are getting chemo.” Constipation questions and Written content “I didn’t know that morphine and opioids can cause medication lists and terminology constipation.” Pain and nausea and vomiting Written content “Don’t you want to know exact time and date of medication questions and terminology [a medication] dose?” Pain questions and medication list Written content Word “narcotic” brought up negative feelings it was “a and terminology scary word.” Pain question and medication list Written content “I know narcotic is bad for you.” and terminology General comment from bilingual Written content “Is this available in Spanish?” participants and terminology Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 11 of 20 Table 4 Results of Usability Testing for the SAMI-Self-Care CDS Program: Patient Perspectives (Continued) Usability Testing Themes CDS Tool Content* CDS Tool Examples Component** Format and navigation Pain and nausea and vomiting Format and “Can we input all the medications we are taking into medication questions navigation the system?” All symptom assessment questions Format and “Add checkboxes to make this [the entry of symptoms] navigation easier.” Suggestions to improve All symptom questions Written content “Create an option of ‘I don’t know.” and terminology General comment about iPad Format and “Are there instructions for those who are not computer functionality navigation users to know how to use this function?” (Referring to functionality of hovering over a definition for more information.) General comment on iPad Format and “Use ‘back’ instead of ‘previous.’” navigation Wording of General comment for instructions for Written content “I don’t want to bother my care team.” self-management suggestions call clinicians on report and terminology General comment for instructions to Written content “When should I contact my care team?” call clinicians on report and terminology General comment for instructions to Written content “What should I tell my care team?” call clinicians on report and terminology Other Patient safety Pain report suggestions Written content “Is it safe for me to take this medication?” for report Nausea and vomiting red flag safety Written content “Are we allowed to drink 2 large glasses a liquid per day? questions for safety Shouldn’t we ask the doctor first?” questions Constipation report suggestions Written content “Is it safe for me to initiate the proposed intervention?” for suggestions Resources Constipation suggestions Other concerns “Do I have suggested medications in my home?” General comment about iPad Format and “Some people will lack the technology to access the navigation system.” * CDS tool content refers to what aspect of the CDS tool that the comment sought to improve (i.e. medication vs. pain severity question) ** CDS tool component refers to what aspect of the CDS tool that the component that the comment sought to improve (i.e. written content vs. visual appeal) Ensure patient safety graphics to support concepts in the text when appropriate. To ensure that the use of SAMI-Self-Care would not We also designated a section of the display for explana- overlook a life-threatening condition, we identified a set tions or lists so that patients who needed more informa- of screening questions for each symptom that sought to tion could easily access it. Formative testing of questions direct patients with potentially dangerous conditions to through the cognitive interviewing process helped us to stop using the algorithm and seek contact with their cli- recognize that we needed to make questions as explicit nicians. To ensure that the advice provided through and detailed as possible so patients were not left to specu- SAMI-Self-Care was sound, we derived algorithm logic late about how to respond. Finally, we used established and recommendations from evidence-based clinical interface development practices for screen appearance, guidelines. When appropriate, we also inquired if a rec- layout, and text fonts to enhance readability [45]. ommended therapy had already been suggested by the patient’s clinician. If the therapy had not been suggested, we encouraged the patients to check with their clinician Promote communication with clinicians to request approval before initiating a new medication. To encourage communication with patients’ clinicians, we included explicit suggestions regarding when to con- Communicate clinical concepts effectively tact their clinicians and a script for what to say to them. In order to communicate clinical concepts effectively, we We also advised patients to notify their clinicians within conducted cognitive testing of content with patients to en- a specified time frame about interventions they may sure an adequate level of understanding, and we added have followed from SAMI-Self-Care. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 12 of 20 Table 5 Results of Usability Testing for the SAMI-Self-Care CDS Program: Clinician Perspectives Usability Testing CDS Tool Context* CDS Tool Component** Examples Themes Visual appeal General comment related to Format “Use larger fonts and colors as a way to distinguish the look and feel of the system instructions from question.” Nausea and vomiting Format “Give a visual description of what a 16-oz container might look like, e.g., a Poland spring water bottle.” Understanding of Pain severity question Written content and preference Disapproval of wording, “bearable pain.” terminology for terminology Nausea and vomiting question Written content “Clearly indicate what issue is being evaluated, e.g., [for] position change, are we asking about getting up quickly or vertigo?” Pain question Written content “Add timeframes, e.g., did taking the pain mediation offer you relief after 30 min?” Nausea and vomiting Format “Add graphics such as [a picture of] fire in the esophagus, which doesn’t need a definition.” All symptoms Written content and format “Medication lists might be overwhelming for some patients.” Pain Written content “Offer educational explanation such as risk factors regarding why the patients shouldn’t take certain medication, e.g., for ibuprofen explain why stomach protection is needed for those 65 or older.” Format and Comment about introduction Format “Select a symptom that is bothering the patients the navigation and orientation to the program most, and then come back to evaluate other symptoms.” Nausea and vomiting question Written content and algorithms. “Prioritize question order based on frequency of This related to the issue that issues experienced by the patients to reduce number chemotherapy induced nausea of questions patients have to answer and to avoid may be more common than patients having to answer questions to symptoms position-induced nausea majority might not experience.” Comment to improve the look and feel Format and navigation “Try to reduce the number of clicks needed to move of the program the system forward, e.g., they shouldn’t have to select the symptom and press next to move forward.” Provide an introduction to Format and navigation “Tell patients upfront the different symptoms or questions so patients will medications the program will ask about.” know what to expect and why General comment related to Format and navigation “Add skip patterns for those who might have used sequencing of questions the system before.” General comment about Written content “Work on lessening redundancy of the questions.” sequence of questions Wording of Symptom reports Written content “Be clear with instructions regarding communication self-management w/ clinicians.” suggestions Symptom reports for red flag questions. Written content “Clearly indicate to the patient to call now, so they do not mistakenly think the report has been automatically sent to their clinician and that someone will follow up.” Symptom reports Written content “Educate the patient on how to use the paging service.” Symptom reports Written content “Don’t put ‘during normal business hours’ because it sounds like we’re telling patients to stop bothering us.” Symptom reports Written content “List phone number of clinician on the report or a paging service for after hours.” Other Patient safety All symptoms red flag Algorithm content Identify all red flag/emergency issues. questions developed for safety Pain red flag safety questions Algorithm content “Ask about new or severe pain not just one [or the other] and, [a] ‘yes’ [response] should mean call your doctor right away.” Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 13 of 20 Table 5 Results of Usability Testing for the SAMI-Self-Care CDS Program: Clinician Perspectives (Continued) Usability Testing CDS Tool Context* CDS Tool Component** Examples Themes General comment about reports Written content Clinicians worried that they may not be informed that are generated for self-management about patient problems. All symptoms red flag questions added Algorithm content Clinicians concerned they may miss or overlook critical situations. General comment about the report Written content “Include notification to patient that they should always call provider with questions.” General comment Written content “Wouldn’t want the patient to use the program instead of getting care.” Resources General concern about use of iPad Format Concern that some patients will not have access to computers. Pain self-management for severe pain Written content about when Lack of consensus among clinicians regarding clinical Best care practices to call their clinicians best practices. Nausea and vomiting acid reflux Algorithm content Some providers recommend medication like TUMS, but GI doctors may avoid it because it creates acid. Pain and nausea and vomiting Written content wanted more Recommended some medications be added on to comprehensive lists for lists [of medications already included]. medications Pain medication question Written content “Change dosing criteria for long acting to 8–12 h.” Pain medication lists Written content “Certain medications on the list not used across the board causing worry, e.g., fentanyl or tapentadol.” * CDS tool content refers to what aspect of the CDS tool that the comment sought to improve (i.e. medication vs. pain severity question) ** CDS tool component refers to what aspect of the CDS tool that the component that the comment sought to improve (i.e. written content vs. visual appeal) Support patient activation that emerged were reorganized around the components We sought to engage patients by providing personalized of this model which included: patient safety, cultural and actionable instructions. As appropriate, we asked pa- competency, care coordination, resource availability, and tients about resource availability and their willingness to acceptance of technology. follow interventions prior to making recommendations. Barriers pertaining to cultural competency were the Through the educational display section, we provided most common and included challenges related to un- content to help patients understand why certain questions familiar terminology, lack of reference to a clinical were being asked and why specific recommendations were framework, and unusual health beliefs. Although many provided. Helpful features included suggestions for of the concerns identified by patients were shared by cli- pharmacologic and non-pharmacologic therapies. nicians, a few items were unique to patients and their caregivers, including expressing uncertainty about some Facilitate navigation and use self-management suggestions and whether the actions To expedite SAMI-Self-Care use we identified methods would be acceptable to their clinicians, expressing nega- to optimize traversal of the algorithms and minimize the tive health beliefs associated with the use of narcotic burden of data entry (see Table 6). We included a pro- medications, and not having the suggested medications gress bar for patients to know their position in the available at home. There also were concerns about com- process, and we added capabilities to change responses munication with their clinicians, availability of technol- or to pause a session. We also added display content to ogy in the community, and use of the technology. enable a patient to recognize the context of every inter- action through tabs illustrating the symptom being ad- Discussion dressed and text clarifying the context of each question. We developed a simulated algorithm-based CDS pro- Finally, we ensured that all pathways could be traversed, gram to support symptom self-management among were unique and not redundant. adults with cancer and their caregivers. Our CDS pro- gram is more complex than other previously developed Barriers that need to be addressed to promote patient- programs and provides patents with specific information, centered CDS tailored to their situation regarding when to call their Barriers to promote patient-centered CDS regarding clinician, what to tell them, and how to self-manage symptom self-management that emerged appeared to be their symptoms. In order to accomplish this task, we consistent with the Chronic Care Model so the themes have many more decisional nodes than previous patient- Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 14 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS Design Principle Design Principle Details Examples of Solutions Change in User Interface 1 Ensure Patient Safety 1a Build algorithm content based on Use of the NCCN guidelines for cancer pain management Based on published best practices, evidence-based content used established clinical guidelines to guide algorithm content [63] for developing symptom management algorithms Iterative review process of algorithm content and recommendations by multidisciplinary expert panel members 1b Identify at the beginning of a session Additional characteristics of symptoms that suggest potentially Questions added that identify severity and trigger “call now” potentially serious conditions for which dangerous or life threatening conditions identified. e.g., “in pain advice. continued use of algorithm could be algorithm, besides enquiring about new or increased pain, CDS updated for immediate exit and to contact clinician if red harmful or life threatening adding a question that asksabout cramping or squeezing in flag was triggered chest or stomach”. Distinguished nuances between pain symptoms, (e.g. new pain Any time red flag is triggered, patient provided with specific (e.g. fracture) and chronic pain) suggestions on the screen. Disclaimer needed to ensure safety (e.g. “In case of emergency, Warning placed on the welcome page of the program. call your doctor or 911 immediately. Do not use this program Bold font used as a way to capture patients’ attention. for medical emergencies.”) There should be gradation of severity indicating what issues Visual cues added to the report to help prioritize should the patient address first self-management strategies Colors (red-orange-green) and fonts used to ensure patient reviewed specific aspect of the report. e.g. call clinician now Report provided at the end can be viewed on the screen or as a printed report 1c Inquire about appropriateness of Provide guidance on why particular intervention should not be Content modified to provide reasons why a particular recommendations prior to offering advice implemented (e.g. taking ibuprofen) intervention would not be permissible (e.g. stomach ulcers) Provide language to ensure any questions are directed to care Report content updated to contact clinician if uncertainty about team at all times concerns on implementing recommendations 2 Communicate 2a Test word selection with intended Cognitive testing of terms and its interpretation Modified wording utilized in assessment and recommendations Clinical Concepts Effectively end-users to improve understanding of concepts 2b Develop explicit, detailed questions Remove ambiguity of decision points Added specificity of timeframes to questions to improve meaning (, e.g., “Did taking short acting pain medication give you relief from your pain within 30 min of taking it?”) Reference specific medications and dosages as appropriate Designed explicit decisions points to enable machine processing 2c Enhance communication with Improve system use by reducing content Added “faces” and word anchors as part of the pain scale graphics, especially for clinical concepts Created content at a 5th grade reading level Inserted images to re-enforce concepts (e.g., stop sign for emer gency, picture to show acid reflux) 2d Provide lists to enable patient to Utilization of system could be improved by equipping patients Provided lists of most common medications in defined classes in identify specific items such as medications with necessary information a designated area of the screen for lookup as needed Included generic and brand names of medications for ease of recognition 2e Provide educational information Using CDS as a way to reinforce and provide education Educational content added in final summary reports customized to promote understanding on why certain questions are being asked to their symptoms Provided rationale of why certain questions were asked and promoted understanding Improve utility by improving layout of content Used large and “heavier” font size to make text more visible Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 15 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS (Continued) Design Principle Design Principle Details Examples of Solutions Change in User Interface 2f Enhance readability with font style, font Reduced text density size, content density, selective highlighting Used a plain white background of words Provided bolding to emphasize words 3 Promote Communication 3a Provide explicit instructions for patients Urgency of establishing clinical contact based on severity Additional features added to generate report immediately on with Clinicians regarding contacting clinicians about their of the symptom needed(e.g. call right away vs. waiting 24 h) screen if patient triggered any of the emergency “red flag” concerns questions and highlighted the importance of calling clinician NOW. Post assessment report that provides guidance on what should Immediate instructions provided to the patient, on calling be done and when. clinician, onscreen of the program and not just within the report. Added explicit language on what patients should say when calling clinician. Initial reports lost the message about the importance of Report restructured to reinforce importance of contacting communicating with the clinician clinicians and keeping them informed of regimen changes. E.g. tell your doctor or nurse you are taking 200 mg of ibuprofen as needed. Clearly communicate recommendations Report modified into sections of: do now, do next and more suggestions, to help streamline and prioritize suggestions for what the patient can do and when Lack of specification of which symptoms are 3 symptoms patients can choose in current system listed at the available for assessment at beginning of the program beginning of the program. Patients advised to contact clinicians if experiencing symptoms not addressed by the system. 3b Encourage patients to notify their Reinforce the importance of notifying clinicians about any Provided instructions about what patients should specifically tell clinical care team about interventions that interventions that have been initiated within the their clinicians about interventions they have followed recommendations 4 Support Patient 4a Determine what resources are available Improve efficiency of the system and utilization by modifying Added questions to determine what interventions had already Activation to the patient question based on what patients have available to them been prescribed Inquired if a prescription was already available for a recommended medication as a way to align with current therapy of the patient’s care team 4b Identify health beliefs that may impact Modifying how content is framed Content modified conveying meaning acceptable by patients. interpretation of content and modify (e.g. pain medication vs. narcotics) content accordingly 4c Determine what patients are willing to Improving look and feel of the system that quickly provides Provided explicit, detailed instructions that include dosage do prior to making recommendations information and allows the patient to take an active role in amounts, frequencies, medication list and lifestyle suggestions their care Prioritized display in patient report to quickly and easily inform the patient on what they should do next 4d Provide explicit, detailed, actionable Inquired about what patients were willing to do prior to instructions to the extent possible recommending an intervention, e.g., use of enemas for constipation 4e Personalize content, e.g., used Create an opportunity relate to the patient and provide Changed the text to make it personable and user friendly, e.g., possessive pronouns such as “my” or “your” self-management techniques used possessive pronouns such as “my” or “your” where where appropriate appropriate 5 Facilitate Navigation and 5a Designate consistent presentation areas Make it easy for patients to find information within the site Posted medical terms with definitions in a specific area on the Use on screen for repeated display of a specific screen so end user can easily and quickly access information as type of information needed Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 16 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS (Continued) Design Principle Design Principle Details Examples of Solutions Change in User Interface Avoided “pop-ups” because they felt to be interruptive and harder to navigate for a limited computer proficient user 5b Provide comprehensive set of Ensure all possible decision points are covered Guidelines and best practices used for comprehensive coverage selection options to ensure all possible selections covered for every decision node 5c Streamline data entry Improve flow and provide feedback quickly Introduced check boxes to cut down on number of questions required to determine what advice to provide and improved efficiency 5d Optimize workflow through questions Inquiring about symptom characteristics at the beginning Enabled selection of an item on a page to advance to the next of the algorithm page as appropriate Directed patients to highly specific interventions 5e Optimize workflow through questions Inquiring about symptom characteristics at the beginning Facilitated patients starting at the appropriate place in the of the algorithm algorithm by inquiring what interventions have already been attempted Introduced check boxes to reduce number of question and reduce redundancy 5f Track progress for patient Promote efficient workflow Added progress bar showing numeric value, not just graphic representing progress Included “Go Back” function to allow patient to modify earlier responses 5g Accommodate patient changes and Offered multiple ways to start over such as: “Back to Start button” pauses as well as tabs with symptom names Included “Take a Break” button to allow patient to pause the program and come back to it again 5h Provide context for all interactions so Added tabs as a way to indicate to the patient which algorithm that patient recognizes where he/she is they were in within an algorithm Provided headers to supply context for each page anchoring the patient on where they are in a given algorithm 5i Ensure completeness and uniqueness of Provided brief overview of different topics that were covered to pathways through algorithm orient patients at the beginning of a session Re-enforced context and inter-relatedness of questions by showing question and answer from the previous page Ensured that questions allow for a single non-redundant, unique pathway for all possible scenarios Ensured that every pathway led to advice 5j Create tools that will function across Assessed target patient population to determine that 85% of patients had Created CDS tool design to function on Web, smart phone, multiple platforms access or knew how to obtain access to computers or smart phones or iPad Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 17 of 20 centered CDS systems [12, 22]. Acceptability testing re- similar to Bates and colleagues, [35] our Design Objectives sults among patients were favorable, suggesting that pa- focused on streamlining the user interaction by decreasing tients found the SAMI-Self Care prototype a satisfactory the number of questions needed to inform the algorithm, approach to for symptom self-management. Providing targeting anticipated needs with advice in real-time, pro- CDS directly to patients may be a valuable tool. Future viding supplemental information as needed but without studies should explore the best modalities for providing interruption, and relying on a user-centered iterative de- access to these tools including the Web, tablet com- sign [43]. Ozkaynak and colleagues [52] noted that an im- puters, or mobile phones [46]. We anticipate that the portant difference between patient and clinician design for SAMI-Self-Care tool could be implemented on a Web- CDS is integrating technology into the user’s work flow. based platform that could be accessible across com- Designing CDS around clinician work flow is much easier puters, tablets and smartphones. A link to this content than for patient work flow as clinicians function within a could be inserted in the homepage of a patient’s oncol- health system, whereas work flow for patients spans mul- ogy treatment group or sent out directly to patients tiple health care settings and includes home, routines of through electronic mail. daily living and communication with family, clinicians and We noted that the acceptability results from clinicians the health care system [52–55]. The CDS program that for pain were less favorable than for nausea/vomiting and was evaluated in this study attempts to bridge the gap be- constipation, and were below our desired threshold of 4.0 tween care delivered in health care settings and self-care for helpfulness of the report and suggestions. This result in the home. There is a need for applications that improve required obtaining a larger sample size to gather responses outcomes across settings and patient populations [56]. Fu- about usability testing as compared to other symptoms. In ture studies are needed to assess the impact of this CDS spite of our efforts to adhere to pain guidelines, [47]some program on patient outcomes. clinicians were reluctant to support self-management be- Through this process, we identified patient barriers to cause they wanted to remain aware of their patients’ use of CDS and clinicians’ concerns about patients using symptoms and to be directly involved in management. CDS for self-management. From these barriers and con- This finding is interesting as implementation of pain cerns, we derived Design Objectives for CDS for symptom guidelines has been a challenge. Evidence suggests that self-management, which included ensuring patient safety, pain management practices have not changed over the last communicating clinical concepts effectively, promoting twenty years and many patients with cancer continue to communication with clinicians, supporting patient activa- experience severe uncontrolled pain [48, 49]. Further tion and facilitating navigation and use. These objectives study is needed to understand clinician attitudes toward may be useful to inform the design of CDS systems for pain management and barriers to encouraging patient symptom self-management of other conditions. self-care management. One concern that clinicians The barriers we identified for CDS for cancer symptom expressed is that they wanted to be aware of patients who self-management had similarities and differences to barriers had severe levels of pain. The addition of a system that associated with self-management in patients with non- alerts clinicians when patient symptoms pass a predeter- malignant conditions [57, 58]. Similar barriers included lack mined level of severity would address this concern. Clee- of knowledge, poor communication between patients and land and colleagues [13] found that automated symptom clinicians, and logistical issues in obtaining care. However, severity alerts after surgery for lung cancer reduced symp- in contrast to the patients who did not have cancer, the tom distress. In a recent study that examined clinician cancer patients did not report barriers related to physical preferences for CDS, the use of an alerting system for in- limitations, financial constraints, a need for social and emo- creased symptom severity was identified as a desirable tional support, or challenges adhering to treatment [57, 58]. component [50]. Other differences in concerns that we noted between pa- Our design objectives were specifically developed in this tients with cancer and those with non-malignant conditions study to support development of effective patient-focused were that cancer patients and their caregivers expressed CDS. To date, CDS design objectives have focused on use concerns related to ensuring patient safety and negative by clinicians. Thus, even though our CDS system was de- health beliefs about use of narcotic pain medications. Shu- signed for patients, we sought to include features known macher and colleagues [59, 60] examined pain management to be associated with effective CDS systems for clinicians processes among cancer patients and their caregivers and [35, 36, 51]. The system features identified in our study found similar concerns related to ensuring patient safety, that were similar to clinician CDS included: graphics to especially in the context of narcotic pain medications. Pa- enhance understanding of content; explicit, actionable rec- tients and their caregivers had little interaction with clini- ommendations provided at the point of decision-making; cians in the home setting and had to master complex tasks presentation of advice that cultivated trust by providing an related to taking their medications and reported that they explanation of medical logic if needed [36]. In addition, felt that they had to be the final safety check. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 18 of 20 Limitations Author contributions MEC participated in conception of the project, data collection, analyses and The sample for this study was drawn from one institu- interpretation of data, writing and revising the manuscript, final approval of tion using purposive sampling so it is not representative the manuscript and is accountable for the quality of work. JLA participated of all cancer patients. All of the patients and their care- in conception of the project, analyses and interpretation of data, writing and revising the manuscript, final approval of the manuscript and is accountable givers in this study indicated that they used the Internet for the quality of work. DLB participated in conception of the project, for seeking health information at least sometimes. This interpretation of data, writing and revising the manuscript, final approval of rate is higher than the 72% of Internet users found in the manuscript and is accountable for the quality of work. MSR participated in conception of the project, interpretation of data, writing and revising the the 2014 Pew Research survey [61]. Our rates for Inter- manuscript, final approval of the manuscript and is accountable for the net usage may be higher as the majority of our sample quality of work. IMB participated in conception of the project, interpretation had greater than a high school education. Factors associ- of data, writing and revising the manuscript, final approval of the manuscript and is accountable for the quality of work. Patient stakeholder, JP, ated with lower access of the Internet are older age, participated in conception of the project, interpretation of data, writing and lower education and lower income [62]. Another limita- revising the manuscript, final approval of the manuscript and is accountable tion is that the majority of the sample was Caucasian for the quality of work. MMN participated in data collection, analyses and interpretation of data, writing and revising the manuscript, final approval of and had higher levels of education. Further testing of the manuscript and is accountable for the quality of work. DFL participated this approach in a more diverse group of patients with in conception of the project, analyses and interpretation of data, writing and cancer is warranted. Although the use of a simulated revising the manuscript, final approval of the manuscript and is accountable for the quality of work. All authors read and approved the final manuscript. model provided a practical and economical approach to iterative development of the CDS tool, this approach did Funding not allow for real time navigation through the algo- This work was supported by The Patient Centered Outcomes Research rithms for patients. This would be the next step in the Institute Grant PI-12-001. development process. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Conclusions The boundaries of health care are expanding and pa- Ethics approval and consent to participate This study was approved by the Dana-Farber Cancer Institute Institutional Re- tients and their caregivers often have to self-manage view board (12–300). Informed consent was obtained from all individual par- complex cancer care at home. Thus, patient-centered ticipants included in the study. decision support that meets this need is important and timely. Our system provides tailored information that in- Competing interests The authors declare that they have no competing interests. forms patients when to call their clinicians, provides a script about what to tell tem about their symptom and specific suggestions about how the self manage their Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in symptoms at home. Patients and their caregivers rated published maps and institutional affiliations. SAMI-Self-Care as highly acceptable and found the rec- ommendations helpful. The patient-centered CDS design Author details The Phyllis F. Cantor Center, Dana-Farber Cancer Institute, 450 Brookline objectives we derived for cancer symptom self- Avenue, Boston, MA 02115, USA. Department of Psychosocial Oncology and management may be applicable for the self-management Palliative Care, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, of other conditions. MA 02115, USA. The Phyllis F. Cantor Center and the Department of Medicine, Dana-Farber Cancer Institute, 450 Brookline Ave, LW-512, Boston, MA 02115, USA. Lowe Center for Thoracic Oncology, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, MA 02115, USA. Klesis Healthcare Abbreviations and Department of Family Medicine, Durham, NC 27705, USA. Department CDS: Clinical decision support; SAMI-Self-Care: Symptom Assessment and of Family Medicine, Duke University Medical Center, 2100 Erwin Road, Management Intervention - Self Care Durham, NC 27710, USA. Received: 4 August 2017 Accepted: 27 April 2018 Acknowledgements We acknowledged the significant contributions of the expert panel in selecting the patient-focused decision-support project and refining the con- tent and design of the decision support tools specified through this project. References In addition to the co-authors of this manuscript, members of this expert 1. Institute of Medicine Committee on Quality of Health Care in A. Crossing panel included Annmarie Uliano, Richard Boyajian, MS; Jane Brzozowski, MS; the quality chasm: a new health system for the 21st century. Washington and Drs. Aymen Elfiky, Anne Gross, Joseph Jacobson, Christopher Lathan, (DC): National Academies Press (US) Copyright 2001 by the National and Larry Shulman. We also acknowledge the contributions of the Dana- Academy of Sciences. All rights reserved; 2001. Farber Harvard Comprehensive Cancer Center Health Communication Core 2. Patient Self-Management Support Programs: An Evaluation. Final Contract of for their contributions to the design and development of the SAMI-SC Report [http://www.orau.gov/ahrq/sms_report_08.asp?p=browse_guide]. prototype. The authors also wish to thank the participants who shared their Accessed Feb 2016. valuable time and effort. We appreciate the data collection contributions of 3. McCorkle R, Ercolano E, Lazenby M, Schulman-Green D, Schilling LS, Lorig K, research coordinator Maribel Melendez and Taylor Hendel for manuscript Wagner EH. Self-management: enabling and empowering patients living preparation. with cancer as a chronic illness. CA Cancer J Clin. 2011;61(1):50–62. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 19 of 20 4. Schulman-Green D, Bradley EH, Knobf MT, Prigerson H, DiGiovanna MP, 25. Ruland CM, Maffei RM, Borosund E, Krahn A, Andersen T, Grimsbo GH. McCorkle R. Self-management and transitions in women with advanced Evaluation of different features of an eHealth application for personalized breast cancer. J Pain Symptom Manag. 2011;42(4):517–25. illness management support: cancer patients' use and appraisal of 5. Schulman-Green D, Bradley EH, Nicholson NR, Jr., George E, Indeck A, usefulness. Int J Med Inform. 2013;82(7):593–603. McCorkle R: One step at a time: self-management and transitions among 26. Creswell JW, Plano Clark VL. Designing and conducting mixed methods women with ovarian cancer. Oncol Nurs Forum 2012, 39(4):354–360. research, 2nd edn. Thousand Oaks: Sage Publications; 2011. 6. Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management 27. Fervers B, Burgers JS, Haugh MC, Latreille J, Mlika-Cabanne N, Paquet L, Coulombe M, Poirier M, Burnand B. Adaptation of clinical guidelines: of chronic disease in primary care. JAMA. 2002;288(19):2469–75. 7. Warsi A, Wang PS, LaValley MP, Avorn J, Solomon DH. Self-management literature review and proposition for a framework and procedure. Int J Qual education programs in chronic disease: a systematic review and methodological Health Care. 2006;18(3):167–76. critique of the literature. Arch Intern Med. 2004;164(15):1641–9. 28. Amer YS, Elzalabany MM, Omar TI, Ibrahim AG, Dowidar NL. The 'Adapted 8. Lorig KR, Sobel DS, Stewart AL, Brown BW Jr, Bandura A, Ritter P, Gonzalez ADAPTE': an approach to improve utilization of the ADAPTE guideline VM, Laurent DD, Holman HR. Evidence suggesting that a chronic disease adaptation resource toolkit in the Alexandria Center for Evidence-Based self-management program can improve health status while reducing Clinical Practice Guidelines. J Eval Clin Pract. 2015;21(6):1095–106. hospitalization: a randomized trial. Med Care. 1999;37(1):5–14. 29. Fervers B, Burgers JS, Voellinger R, Brouwers M, Browman GP, Graham ID, 9. McKay HGG, Russell E, Feil EG, Boles SM, Barrera M Jr. Internet-based Harrison MB, Latreille J, Mlika-Cabane N, Paquet L, et al. Guideline diabetes self-management and support: initial outcomes from the diabetes adaptation: an approach to enhance efficiency in guideline development network project. Rehabil Psychol. 2002;47(1):31–48. and improve utilisation. BMJ Qual Saf. 2011;20(3):228–36. 30. Ruegg TA. A nurse practitioner-led urgent care center: meeting the needs 10. Mooney KH, Beck SL, Friedman RH, Farzanfar R, Wong B. Automated of the patient with cancer. Clin J Oncol Nurs. 2013;17(4):E52–7. monitoring of symptoms during ambulatory chemotherapy and oncology providers' use of the information: a randomized controlled clinical trial. 31. Cooley ME, Lobach DF, Johns E, Halpenny B, Saunders TA, Del Fiol G, Rabin Support Care Cancer. 2014;22(9):2343–50. MS, Calarese P, Berenbaum IL, Zaner K, et al. Creating computable 11. Ruland CM, Andersen T, Jeneson A, Moore S, Grimsbo GH, Borosund E, Ellison algorithms for symptom management in an outpatient thoracic oncology MC. Effects of an internet support system to assist cancer patients in reducing setting. J Pain Symptom Manag. 2013;46(6):911–24. e911 symptom distress: a randomized controlled trial. Cancer Nurs. 2013;36(1):6–17. 32. Scariot CA, Heemann A, Padovani S. Understanding the collaborative- 12. Berry DL, Hong F, Halpenny B, Partridge A, Fox E, Fann JR, Wolpin S, Lober WB, participatory design. Work (Reading, Mass). 2012;41(Suppl 1):2701–5. Bush N, Parvathaneni U, et al. The electronic self report assessment and 33. Revere D, Dixon BE, Hills R, Williams JL, Grannis SJ. Leveraging health intervention for cancer: promoting patient verbal reporting of symptom and information exchange to improve population health reporting processes: quality of life issues in a randomized controlled trial. BMC Cancer. 2014;14:513. lessons in using a collaborative-participatory design process. EGEMS (Wash, DC). 2014;2(3):1082. 13. Cleeland CS, Wang XS, Shi Q, Mendoza TR, Wright SL, Berry MD, Malveaux D, 34. Collins D. Pretesting survey instruments: an overview of cognitive methods. Shah PK, Gning I, Hofstetter WL, et al. Automated symptom alerts reduce Qual Life Res Int J Qual Life Asp Treat Care Rehab. 2003;12(3):229–38. postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. J Clin Oncol Off J Am Soc Clin Oncol. 2011;29(8):994–1000. 35. Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, 14. Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management Khorasani R, Tanasijevic M, Middleton B. Ten commandments for effective approaches for people with chronic conditions: a review. Patient Educ clinical decision support: making the practice of evidence-based medicine a Couns. 2002;48(2):177–87. reality. J Am Med Inform Assoc. 2003;10(6):523–30. 15. Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ. 36. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice Effectiveness of web-based interventions on patient empowerment: a using clinical decision support systems: a systematic review of trials to systematic review and meta-analysis. J Med Internet Res. 2010;12(2):e23. identify features critical to success. BMJ. 2005;330(7494):765. 37. Crabtree BF, Yanoshik MK, Miller WL, O'Connor PJ. Successful focus groups: 16. Gustafson DH, Hawkins R, McTavish F, Pingree S, Chen WC, Volrathongchai K, advancing the state of the art. Thousand Oaks: SAGE Publications, Inc.; 1993. Stengle W, Stewart JA, Serlin RC. Internet-based interactive support for Cancer patients: are integrated systems better? J Commun. 2008;58(2):238–57. 38. Virzi R. Refining the test phase of usability evaluation: how many subjects is 17. Marie N, Luckett T, Davidson PM, Lovell M, Lal S. Optimal patient enough? Hum Factors. 1992;34(4):457–68. education for cancer pain: a systematic review and theory-based meta- 39. Tariman JD, Berry DL, Halpenny B, Wolpin S, Schepp K. Validation and analysis. Support Care Cancer. 2013;21(12):3529–37. testing of the acceptability E-scale for web-based patient-reported 18. Yun YH, Lee KS, Kim YW, Park SY, Lee ES, Noh DY, Kim S, Oh JH, Jung SY, outcomes in cancer care. Appl Nurs Res. 2011;24(1):53–8. Chung KW et al: Web-based tailored education program for disease-free 40. Berry DL, Halpenny B, Wolpin S, Davison BJ, Ellis WJ, Lober WB, McReynolds J, cancer survivors with cancer-related fatigue: a randomized controlled trial. J Wulff J. Development and evaluation of the personal patient profile-prostate Clin Oncol Off J Am Soc Clin Oncol 2012, 30(12):1296–1303. (P3P), a web-based decision support system for men newly diagnosed with localized prostate cancer. J Med Internet Res. 2010;12(4):e67. 19. Lewis FM, Brandt PA, Cochrane BB, Griffith KA, Grant M, Haase JE, Houldin AD, Post-White J, Zahlis EH, Shands ME. The enhancing connections 41. Mullen KH, Berry DL, Zierler BK. Computerized symptom and quality-of-life program: a six-state randomized clinical trial of a cancer parenting program. assessment for patients with cancer part II: acceptability and usability. Oncol Nurs Forum. 2004;31(5):E84–9. J Consult Clin Psychol. 2015;83(1):12–23. 20. Ruland CM, Jeneson A, Andersen T, Andersen R, Slaughter L, Bente Schjodt 42. Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008; O, Moore SM. Designing tailored internet support to assist cancer patients 62(1):107–15. in illness management. AMIA Annu Symp Proc. 2007;2007:635–9. 43. Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care 21. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A model in the new millennium: thus far, the evidence on the chronic care roadmap for national action on clinical decision support. J Am Med Inform model is encouraging, but we need better tools to help practices improve Assoc. 2007;14(2):141–5. their systems. Health affairs (Project Hope). 2009;28(1):75–85. 22. Head BA, Keeney C, Studts JL, Khayat M, Bumpous J, Pfeifer M. Feasibility 44. Lobach DF, Johns EB, Halpenny B, Saunders TA, Brzozowski J, Del Fiol G, and acceptance of a telehealth intervention to promote symptom Berry DL, Braun IM, Finn K, Wolfe J et al: Increasing complexity in rule-based management during treatment for head and neck Cancer. J Support Oncol. clinical decision support: the symptom assessment and management 2011;9(1):e1–e11. intervention. JMIR Med Inform 2016, 4(4):e36. 45. Johnson CM, Johnson TR, Zhang J. A user-centered framework for 23. Weaver A, Young AM, Rowntree J, Townsend N, Pearson S, Smith J, Gibson O, redesigning health care interfaces. J Biomed Inform. 2005;38(1):75–87. Cobern W, Larsen M, Tarassenko L. Application of mobile phone technology for managing chemotherapy-associated side-effects. Ann Oncol. 2007;18(11): 46. Mirkovic J, Kaufman DR, Ruland CM. Supporting cancer patients in illness 1887–92. management: usability evaluation of a mobile app. JMIR mHealth and 24. Berry DL, Hong F, Halpenny B, Partridge AH, Fann JR, Wolpin S, Lober WB, uHealth. 2014;2(3):e33. doi:https://doi.org/10.2196/mhealth.3359. Bush NE, Parvathaneni U, Back AL, et al. Electronic self-report assessment for 47. Gordon DB, Dahl JL, Miaskowski C, McCarberg B, Todd KH, Paice JA, Lipman cancer and self-care support: results of a multicenter randomized trial. J Clin AG, Bookbinder M, Sanders SH, Turk DC, et al. American pain society Oncol Off J Am Soc Clin Oncol. 2014;32(3):199–205. recommendations for improving the quality of acute and cancer pain Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 20 of 20 management: American pain society quality of care task force. Arch Intern Med. 2005;165(14):1574–80. 48. Isaac T, Stuver SO, Davis RB, Block S, Weeks JC, Berry DL, Weingart SN. Incidence of severe pain in newly diagnosed ambulatory patients with stage IV cancer. Pain Res Manag. 2012, 17;(5):347–52. 49. Zhao F, Chang VT, Cleeland C, Cleary JF, Mitchell EP, Wagner LI, Fisch MJ. Determinants of pain severity changes in ambulatory patients with cancer: an analysis from eastern cooperative oncology group trial E2Z02. J Clin Oncol. 2014;32(4):312–9. 50. Berry DL, Nayak MM, Abrahm JL, Braun I, Rabin MS, Cooley ME. Clinician perspectives on symptom and quality of life experiences of patients during cancer therapies: implications for eHealth. Psycho-Oncology. 2017; 51. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012; 45(6):1202–16. 52. Ozkaynak M, Brennan PF, Hanauer DA, Johnson S, Aarts J, Zheng K, Haque SN. Patient-centered care requires a patient-oriented workflow model. J Am Med Inform Assoc. 2013;20(e1):e14–6. 53. Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK. Promise of and potential for patient-facing technologies to enable meaningful use. Am J Prev Med. 2011;40(5 Suppl 2):S162–72. 54. Clauser SB, Wagner EH, Aiello Bowles EJ, Tuzzio L, Greene SM. Improving modern cancer care through information technology. Am J Prev Med. 2011; 40(5 Suppl 2):S198–207. 55. Valdez RS, Holden RJ, Novak LL, Veinot TC. Transforming consumer health informatics through a patient work framework: connecting patients to context. J Am Med Inform Assoc. 2015;22(1):2–10. 56. McCoy AB, Wright A, Eysenbach G, Malin BA, Patterson ES, Xu H, Sittig DF. State of the art in clinical informatics: evidence and examples. Yearb Med inform. 2013;8(1):13–9. 57. Bayliss EA, Steiner JF, Fernald DH, Crane LA, Main DS. Descriptions of barriers to self-care by persons with comorbid chronic diseases. Ann Fam Med. 2003;1(1):15–21. 58. Fort MP, Alvarado-Molina N, Pena L, Mendoza Montano C, Murrillo S, Martinez H. Barriers and facilitating factors for disease self-management: a qualitative analysis of perceptions of patients receiving care for type 2 diabetes and/or hypertension in San Jose, Costa Rica and Tuxtla Gutierrez, Mexico. BMC Fam Pract. 2013;14:131. 59. Schumacher KL, Plano Clark VL, West CM, Dodd MJ, Rabow MW, Miaskowski C. Pain medication management processes used by oncology outpatients and family caregivers part II: home and lifestyle contexts. J Pain Symptom Manag. 2014;48(5):784–96. 60. Schumacher KL, Plano Clark VL, West CM, Dodd MJ, Rabow MW, Miaskowski C. Pain medication management processes used by oncology outpatients and family caregivers part I: health systems contexts. J Pain Symptom Manag. 2014;48(5):770–83. 61. Fox S, Duggan M. Health online 2013. Pew Research Center. Internet, Science, and Tech. Internet. 15 Jan 2013. (http://www.pewinternet.org/2013/01/15/ health-online-2013/) 62. Smith A, PRCI. Science and technology: technology adoption by lower income populations. San Diego: American Public Human Services Association, IT Solution Management; 2013. 63. NCCN. Clinical practice guidelines in oncology. Adult Cancer Pain. Version 2. 2014 [http://williams.medicine.wisc.edu/pain.pdf http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Medical Informatics and Decision Making Springer Journals

Algorithm-based decision support for symptom self-management among adults with Cancer: results of usability testing

Download PDF
20 pages

Loading next page...
 
/lp/springer_journal/algorithm-based-decision-support-for-symptom-self-management-among-T2J8uMWMb3

References (79)

Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s).
Subject
Medicine & Public Health; Health Informatics; Information Systems and Communication Service; Management of Computing and Information Systems
eISSN
1472-6947
DOI
10.1186/s12911-018-0608-8
Publisher site
See Article on Publisher Site

Abstract

Background: It is essential that cancer patients understand anticipated symptoms, how to self-manage these symptoms, and when to call their clinicians. However, patients are often ill-prepared to manage symptoms at home. Clinical decision support (CDS) is a potentially innovative way to provide information to patients where and when they need it. The purpose of this project was to design and evaluate a simulated model of an algorithm- based CDS program for self-management of cancer symptoms. Methods: This study consisted of three phases; development of computable algorithms for self-management of cancer symptoms using a modified ADAPTE process, evaluation of a simulated model of the CDS program, and identification of design objectives and lessons learned from the evaluation of patient-centered CDS. In phase 1, algorithms for pain, constipation and nausea/vomiting were developedbyanexpertpanel.Inphase 2, we conducted usability testing of a simulated symptom assessment and management intervention for self-care (SAMI-Self-Care) CDS program involving focus groups, interviews and surveys with cancer patients, their caregivers and clinicians. The Acceptability E-scale measured acceptability of the program. In phase 3, we developed design objectives and identified barriers to uptake of patient-centered CDS based on the data gathered from stakeholders. Results: In phase 1, algorithms were reviewed and approved through a consensus meeting and majority vote. In phase 2, 24 patients & caregivers and 13 clinicians participated in the formative evaluation. Iterative changes were made in a simulated SAMI-Self-Care CDS program. Acceptability scores were high among patients, caregivers and clinicians. In phase 3, we formulated CDS design objectives, which included: 1) ensure patient safety, 2) communicate clinical concepts effectively, 3) promote communication with clinicians, 4) support patient activation, and 5) facilitate navigation and use. We identified patient barriers and clinician concerns to using CDS for symptom self-management, which were consistent with the chronic care model, a theoretical framework used to enhance patient-clinician communication and patient self- management. Conclusion: Patient safety and tool navigation were critical features of CDS for patient self-management. Insights gleaned from this study may be used to inform the development of CDS resources for symptom self-management in patients with other chronic conditions. Keywords: Rule-based clinical decision support, Symptom management, Patient engagement, Patient self-management, Cancer * Correspondence: [email protected] The Phyllis F. Cantor Center, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, MA 02115, USA Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 2 of 20 Background called the clinic to ask for help. Ruland et al. [20] devel- Patient-centered healthcare is one of six aims to improve oped a Web-based resource that assisted patients with the United States healthcare system [1]. One facet of assessing symptoms, finding information, communicat- patient-centered care is engaging patients and encour- ing with clinic personnel and provided self-management aging self-management, especially in the context of advice. This tool was evaluated and showed a slight de- chronic illness. Self-management programs that provide crease in symptom distress for patients [20] but the CDS coaching and education, and are supported by timely component was not evaluated independently for its ef- information, facilitate patient engagement [2, 3]. fect. A follow-up study determined that communication Given the frequency of symptoms from cancer and its with nurses was the most valued component [25]. treatment, it is essential that patients (and their caregivers) The present study extends the literature by reporting understand how to self-manage symptoms and when to call on the design and formative evaluation of an algorithm- their clinicians for advice. McCorkle and colleagues [3–5] based simulated model of a patient-centered CDS pro- noted that most patients with cancer try to self-manage gram that facilitates cancer symptom self-management, their care and that developing system-level interventions to provides advice on when patients should contact their support self-management is essential for quality cancer clinicians, and includes coaching information about what care. Evidence-based strategies to assist patients with self- to tell them. The goals of the project were to develop management include: education, telephone consultations, computable algorithms for pain, constipation and nau- Internet tools for tracking disease-specific parameters, and sea/vomiting in phase 1, conduct iterative usability test- coaching [3, 6–13]. Such interventions targeted information ing of the simulated CDS program called the Symptom management, medications, psychological consequences of Assessment and Management Intervention for Self- illness, lifestyle, social support, communication, accessing Care (SAMI-Self-Care) with patients, their caregivers services, and setting goals [14–16]. Self-management inter- and clinicians in phase 2, and develop design objectives ventions decreased the severity of pain [17], fatigue [18], and identify barriers to uptake of patient-centered CDS and depression [19]. Web-based interventions, featuring based on the data gathered from stakeholders, which in- self-monitoring, education and coaching of patients com- cluded members of an expert panel, patients, caregivers, bined with summaries of patient-reported data delivered to and their clinicians in phase 3. clinicians, led to improvement in symptom distress and quality of life [12, 16, 20]. Evidence suggests that reporting Methods symptoms to clinicians alone may not be sufficient. A pro- We employed a convergent, parallel, mixed methods de- ject in which patients receiving chemotherapy reported on sign, [26] in which we collected qualitative and quantitative symptoms using an automated phone system failed to im- data in parallel, analyzed it separately, and then merged all pact care. Even though symptoms of moderate to severe in- data during interpretation, to evaluate the acceptability of a tensity were reported, clinicians did not provide patients simulated algorithm-based CDS program for cancer with management guidance [10]. symptom self-management. This study was conducted at Considering the variability of clinician response to Dana-Farber Cancer Institute (DFCI) and approved by the symptom reports, an optimal approach to enhance self- Institutional Review Board (IRB), Protocol number-12-300. management includes providing patient-specific, real- time, actionable information. Supplying Clinical Decision Phase 1: Algorithm-based CDS intervention development Support (CDS) directly to patients may enhance their In order to develop the patient-centered CDS algorithms, ability to self-manage symptoms. CDS “provides individ- we used a modified ADAPTE process [27–29]consisting uals with person-specific information, intelligently fil- of five steps; 1) identify expert panels to develop and tered or presented at appropriate times, to enhance evaluate the computable algorithms, 2) develop groups to health” [21]. Four studies have described rule-based CDS work on each symptom and synthesize the literature, 3) tools for cancer patients. [12, 20, 22, 23] In two studies, convene groups to translate evidence-based information CDS was used to identify the presence of a symptom into computable algorithms for each symptom, 4) conduct using an algorithm with a single decision node that gen- peer review on the content of the algorithms before con- erated general recommendations for symptom manage- vening a multidisciplinary consensus meeting, and 5) hold ment [12, 22]. In studies by Berry and colleagues [24] a multidisciplinary panel meeting to review, modify and and Weaver, [23] the focus was self-care support and approve the algorithms. identifying symptoms to report to clinicians. Berry and The study team identified areas of expertise that were colleagues [12] found that Web-based coaching of pa- critical for developing and evaluating an algorithm-based tients to report symptom experiences verbally resulted CDS tool for symptom self-management. Individuals with in more frequent reports during clinic visits, but the in- the desired expertise were identified through existing pro- vestigators did not document how often a participant fessional relationships or through recommendations of Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 3 of 20 colleagues. The identified individuals were invited to join is recommended as a standard approach to the develop- the panel by the principal investigator (MEC) and in- ment process as it allows one to ascertain whether partici- cluded: stakeholders with expertise in clinical care (oncol- pants understand items on a questionnaire and allows for ogists, palliative care experts, psychiatrists, and oncology iterative changes before finalizing questions [34]. nurses), information systems (experts in CDS, patient data This patient-vetted algorithm content was given to the collection, health communication, and graphic design), Dana-Farber Harvard Comprehensive Cancer Center care delivery process (experts in workflow, quality im- Health Communication Core. The Core developed a provement, and health equity), as well as patients and mockup of the SAMI-Self-Care using interactive PDF caregivers. files to simulate the function of the CDS tool based on Our expert panel drew from evidence-based resources expert panel input and clinical informatics literature [35, and worked with CDS experts to develop computable al- 36]. The PDF files imitated the functionality of a Web- gorithms that would enable self-management of pain, site and allowed participants to experience the “look and constipation, and nausea/vomiting. Overall, the expert feel” of a potential operational CDS system on an iPad. panel met four times for 4 h each time to review the al- This approach provided a practical and economical ap- gorithms developed by the research team. The symp- proach to usability testing, especially during formative toms chosen were the most common reasons for urgent stages of development when iterative refinement is ne- care among cancer patients and identified as important cessary. The Health Communication Core also designed targets for symptom management by patients and their reports containing self-management recommendations caregivers in a previous study that explored patient pref- to elicit feedback from participants regarding report erences for CDS to enhance clinical care [30]. Our content and appearance. A series of focus groups or in- process of developing algorithms was based on our pre- terviews were conducted to elicit feedback about the vious work that adapted evidence-based guidelines for CDS program. Using a multi-methods approach (i.e. clinicians [31]. Similar to our previous work, multi- focus groups and interviews) is appropriate when the in- disciplinary groups were formed for each symptom and tent is to gather information that is focused on specific serial meetings were held to develop the initial algo- goals and questions, which in this case was participant rithms. Once the algorithms were completed, a group response about the usability of an algorithm-based CDS meeting with the members of the research team and the program [37]. We used a combination of methods to expert panel was held to review, modify and approve the provide flexibility to participants to be able to participate algorithms. For each symptom, characteristics were iden- in the usability sessions. We used the same facilitator to tified to direct self-management. Characteristics of pain conduct the focus groups and interviews and used an included descriptors of neuropathic or somatic pain, and interview guide to elicit information about the usability a measure of intensity (moderate, severe). Features of and acceptability of the SAMI-Self-Care CDS program. nausea/vomiting included chemotherapy-induced, Based on feedback from the expert panel, patients, radiation-induced, chemoradiation-induced, acid reflux, caregivers, and clinicians, we iteratively created three vertigo, recent narcotic increase, anticipation of chemo- successive versions of the SAMI-Self-Care program. therapy, and constipation. Characteristics of constipation Figures 1, 2, 3, 4 show the iterative development of the included frequency and consistency of stool. self-management report that was generated for partici- pants based on responses to the questions and Phase 2: Usability testing algorithm-based CDS. Feedback from the usability test- In order to refine the algorithm content and the simulated ing was used to inform the changes that were made to SAMI-Self-Care tool display and function iteratively, we the CDS program and the self-management report (see used a collaborative-participatory approach. This ap- results). proach engaged stakeholders in a shared co-designing process and helped ensure that the product will meet real- Participant recruitment life needs and be adopted [32, 33]. The team formulated a Patients were identified through medical records at DFCI scenario for a symptom management dilemma that would under a waiver of consent authorization, and names were allow patient users to traverse the self-care management submitted to their clinicians for approval to contact pa- algorithms so that self-management recommendations tients, as required by the IRB. Eligible patients were age ≥ could be generated. Cognitive testing was performed for 18, English speaking, and had received cancer treatment each algorithm by reviewing the content displayed on an within the past 6-months. All approved patients were sent iPad with patients and their caregivers to ensure that these an initial letter describing the study and inviting them to questions were clear, understandable, and relevant, and participate. Patients received two follow-up calls if they that the response options were appropriate prior to initiat- didn’t respond to the letter. Patients identified caregivers ing the iterative usability testing. Cognitive testing of items to invite to participate in the study. Eligible caregivers Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 4 of 20 Fig. 1 Initial SAMI-Self Care report were age ≥ 18 years and provided care to a patient who re- participants and were held in a convenient conference ceived cancer treatment within the past 6-months. Delib- room at DFCI. Meetings for clinicians lasted about 30 min erate efforts were made to sample across a range of whereas those for patients and caregivers lasted 45– diagnoses, educational backgrounds, ages, genders, and 60 min. One symptom was addressed per session. Using a races. All participants signed consent forms at the time of script, the facilitator queried patients, caregivers and clini- participation. Each participant received $50 for complet- cians about the algorithm content of the SAMI-SC pro- ing the study. grams for constipation, pain, and nausea/vomiting, and Clinicians in ambulatory oncology included physicians, asked about the “look and feel” of the materials for pain nurse practitioners, physician assistants, and registered and nausea/vomiting. The topics for patients and care- nurses. Clinicians were identified through clinic rosters. givers focused on the: 1) visual appeal, 2) format and navi- An invitation to participate in the study was emailed to gation, 3) understandability of written content and eligible clinicians. Two follow-up emails were sent if no terminology, and 4) wording of self-management sugges- response was received. Clinicians were recruited for either tions. The topics for the clinicians focused on 1) a review focus groups or interviews depending on their schedules. of the patient self-management algorithms, 2) review of Clinicians received an invitation letter describing the study the simulated SAMI-Self-Care CDS program, and 3) feed- but signed consent was waived by the IRB. Each clinician back about ways to improve the algorithms and/or simu- received $100 for completing the study. lated model. Patients/caregivers and clinicians reported demographic data using a standardized survey. Focus group and interview processes The number of sessions per symptom was determined Focus groups and interviews were held separately for cli- by evidence of content saturation and absence of new nicians and patients/caregivers. Focus groups included up common responses [38]. With patients/caregivers, we to five participants; interviews included one or two conducted six sessions for pain, seven sessions for nausea/ Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 5 of 20 Fig. 2 First revision for SAMI-Self Care report vomiting, and three sessions for constipation. With clini- clinician participants regarding: 1) understandability of cians, we conducted two sessions for pain, three sessions language, 2) helpfulness of suggestions provided for for nausea/vomiting, and one session for constipation. All symptom self-management, 3) usefulness of reports, and discussions were audio-recorded and transcribed. 4) overall satisfaction with the program. The a priori tar- get for acceptability of SAMI-Self-Care was a mean Survey instruments score of ≥4 on each item since this score indicated a The Acceptability E-scale measured the acceptability of positive acceptability response. In addition to the the CDS program [39]. This scale has been used in adult Acceptability E-scale, participants were asked what other patients with various types of cancers from medical on- types of information would have been helpful for symp- cology, radiation oncology and stem cell transplant to tom self-management. The scale took less than 2 min to measure acceptability of Web-based systems, including complete and was written at a 5th grade reading level. computerized symptom and quality of life assessments, patient educational materials and patient-centered deci- Analysis sion support. Psychometric testing was conducted in 627 Patient/caregiver and clinician demographic data were adults with cancer [40, 41]. Cronbach alpha reliability summarized as descriptive statistics. Acceptability coefficient was 0.75 and factor analysis revealed that the E-scale results were summarized as item means and scale was unidimensional [39]. Five-point Likert-type standard deviations. items (1–5; larger number equals higher acceptability Audio-recorded session content was transcribed and and a score of 3 indicates a neutral response) were pre- reviewed by three study team members (MEC, DFL, sented to elicit feedback from both patient/caregiver and MMN), coded inductively, and grouped into common Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 6 of 20 Fig. 3 Second revision for SAMI-Self Care report responses to identify revisions that needed to be made in organize the data collected from the expert panel, patients, the CDS tool. Common responses were then combined caregivers and clinicians and discuss the themes that were and reconciled though discussion. A list of suggested re- identified through inductive content analysis [42]. These visions to improve SAMI-Self-Care algorithms and the data were then shared and discussed with other members simulated model was created and ranked based on audio of the research team to generate a final list of design ob- files and notes. Critical revisions were implemented im- jectives, patient barriers, and clinician concerns to use of mediately, while other less critical revisions were moni- the CDS program. tored for repetition. This process was repeated, after each iteration of CDS tool development, until no new Analyses responses or suggestions for changes emerged from the Audio-recorded session content was transcribed and usability testing. Phase 2 analyses focused on the revi- reviewed by three study team members (MEC, DFL, sions that needed to be make to the CDS tool. MMN), coded, and grouped into themes. Themes were then combined and reconciled though discussion. In- Phase 3: Design objectives and barriers to uptake of ductive analyses of the coded qualitative data for phase 3 patient-centered CDS focused on interpreting patient barriers to using the Once all the data were collected and the usability testing CDS program, defining clinician concerns about use of completed, three investigators (MEC, DFL, MMN) CDS by patients/caregivers, and identifying strategies to reviewed all of the qualitative data in order to identify de- overcome the areas of concern. It was noted during the sign objectives and barriers to uptake of the patient- analysis that the themes that emerged from the qualita- centered CDS program. Eight sessions were needed to tive data were consistent with the Chronic Care Model Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 7 of 20 Fig. 4 Third revision for SAMI-Self Care report [3, 43]. Thus, five design objectives detailed below (in non-pharmacological and self-management approaches Results) were developed to address these concerns. As a so that a comprehensive approach to enable self- result, content analysis was then used to regroup the management would be embedded into the algorithms. themes to fit within the Chronic Care Model. Once the development of the algorithms and usability testing was completed, a multidisciplinary consensus Results meeting was held with the expert panel, research team and Phase 1: Rule-based CDS intervention development co-investigators to review, modify and approve the final Nine potential participants were approached for mem- versions. In order to iteratively improve the program, the bership in the expert panel, and all accepted the invita- expert panel focused on several issues. The primary issue tion. Altogether, 16 members of the research team and was patient safety. Safety was addressed by identifying po- expert panel contributed to algorithm development. tential serious causes of each symptom and directing pa- Work sessions were conducted in person and by Web tients to seek contact with their clinicians. Accordingly, all conferencing so that the flow charts could be displayed three algorithms began with the identification of “red flag more easily for discussion. Branching logic was used to symptoms” that caused patients a forced exit of the CDS develop the algorithms and these were displayed as flow tool and a directive to call a clinician immediately for guid- charts that were shared among all team members. Re- ance. A second issue was how to make the tool accessible search staff provided support to access current guide- to patients across a range of health literacies. Our solution lines that were available for the targeted symptoms and was to provide information that patients could elect to re- to assist with literature reviews. Each work group had view or skip based on their needs. A third issue was how someone assigned who was expert in pharmacological, to provide medication-related advice that was aligned with Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 8 of 20 therapies recommended by a patient’s clinician. To address Table 2 Cancer Patient and Caregiver Demographics (N = 24) this issue, we asked the patient whether the medication we Characteristic n % were recommending for the problem (e.g. senna for consti- Role pation) was approved by his/her clinician. If it was ap- Patient 15 63 proved, we inquired whether the patient had taken that Caregiver 9 37 medication. If not, the patient was advised to take it. If the Gender patient had not been prescribed the recommended therapy, Female 13 54 he or she was advised to contact his/her clinician and inquire if this therapy could be appropriate. The resulting Age algorithms had a large number of decision nodes ranging Median/Range 55 21–69 from 51 to 257 (moderate and severe pain combined). A <50 5 21 decision node is a point in the algorithm where the logic ≥ 50 19 79 branches into two or more directions. A higher number of Race decision nodes reflect greater algorithmic complexity [44]. Caucasian 20 83 Table 1 provides information about the number of decision nodes and red flag questions that were present within each Black/African American 2 8 symptom algorithm. A simple majority vote was taken at Other 2 8 the end of the discussion surrounding each symptom algo- Ethnicity rithm to determine whether agreement was reached regard- Hispanic 4 17 ing modifications and approval. Table 1 provides an Non-Hispanic 18 75 overview of the number of red flags that were required for Did Not Report 2 8 each symptoms and the complexity of the algorithms. Education Phase 2: Usability testing High School or Less 3 12 Participant sample Some College or More 21 88 One-hundred-and-three patients were screened to identify Income the 24 patients and caregivers for study inclusion (Table 2). $49,999 or Less 4 17 Twenty-six patients were deemed ineligible by their clini- $50,000 or More 19 79 cians due to the incapacitating nature of their condition and 14 did not meet other eligibility criteria or could not Did Not Report 1 4 be contacted. Seventeen potential subjects declined par- Cancer Type (n = 15) ticipation; 22 indicated interest but were unable to attend Hematologic Malignancies 5 33 scheduled sessions or did not respond to scheduling re- Solid Tumor Malignancies 10 67 quests. Participating patients and caregivers were predom- Types of Solid Tumor inately Caucasian, almost evenly split between men and Breast 1 7 women and most had some college level education. Forty-four clinicians were contacted to identify 13 who Gastrointestinal 3 20 participated in focus groups and interviews (Table 3). Of Genitourinary 1 7 the clinicians who did not participate, 20 did not Gynecologic 1 7 respond, 2 declined, and 9 indicated interest but were Head and Neck 1 7 unable to attend scheduled sessions. Clinician partici- Neuro-oncology 1 7 pants were predominately female and Caucasian. Thoracic 2 13 Internet Use to Obtain Health Information Table 1 Number of decision nodes and red flag questions in Never/Rarely 0 0 each symptom algorithm Sometimes 10 42 Symptom Decisional Nodes Red Flag Questions Nausea & Vomiting 54 6 Often/Very Often 13 54 Pain 257 2 Missing 1 4 Severe Pain 125 Moderate Pain 122 Focus groups and interviews Patients and caregivers provided feedback about the vis- Constipation 51 4 a ual appeal of the program, especially related to the color Red flag questions are the same for both the moderate and severe pain pathways scheme, text density, and font size. Participant feedback Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 9 of 20 Table 3 Oncology Clinician Demographics (N = 13) clinicians immediately, whereas others felt that it was ac- ceptable to use the suggestions and then call their clini- Characteristic n % cians for refractory pain. Tables 4 and 5 provide an Gender overview of comments provided by participants to im- Female 11 85 prove the usability of the program. Race Caucasian 11 84 Black/African American 1 8 Acceptability surveys Participants completed surveys regarding the accept- Asian 1 8 ability of SAMI-Self-Care for “nausea/vomiting” and Ethnicity “pain” after each round of system development and Non-Hispanic 11 85 viewing the simulated interface. Nine patients and Did Not Report 2 15 their caregivers completed surveys, four assessing pain Training and five assessing nausea and vomiting. Two rounds Physicians 3 23 of assessment for usability and acceptability were con- ducted with patients to reach or surpass the predeter- Nurse Practitioners 4 31 mined threshold for acceptability. In general, patients Physician Assistants 2 15 found the system easy to understand and helpful for Registered Nurses 4 31 self-management. Cancer Specialty Area Thirteen clinicians completed surveys, nine assessed Hematologic Malignancies 4 31 pain and three assessed nausea and vomiting, and one Solid Tumor Malignancies 9 69 assessed constipation. Two rounds of assessment for us- ability and acceptability were conducted with clinicians. Types of Solid Tumor The simulated program for nausea/vomiting and consti- Gastrointestinal 2 15 pation reached and surpassed the threshold for accept- Genitourinary 1 8 ability. However, the helpfulness of suggestions for the Neuro-oncology 1 8 pain algorithm was scored lower than our target thresh- Head and Neck 1 8 old of 4.0. For the most part, clinicians also viewed the Thoracic 2 15 system for pain as usable and helpful. Subsequent dis- cussions revealed that some clinicians felt that patients Radiation Oncology 1 8 with a pain score ≥ 9 should contact their clinicians and General Practice 1 8 not pursue self-management. The algorithm was subse- Prior Use of Patient-Focused Information Tools quently changed to reflect that patients with severe pain, Never/Rarely 8 61 should contact their clinicians, but time limitations pre- Sometimes/Often/Very Often 5 39 vented the modified algorithm from being reevaluated. sought to make the display inviting and easy to read. Some of the issues raised by clinicians who viewed the Phase 3: Design Objectives and Barriers to Patient- program were similar to issues identified by patients and centered CDS their caregivers. For example, clinicians made sugges- Design objectives tions about use of language, decreasing the density of From patient, caregiver, and stakeholder feedback, we text and providing graphic images to enhance under- identified five design objectives that were relevant for standing. For the most part, patients, caregivers and cli- the development of algorithm-based patient-centered nicians had similar comments to enhance the usability CDS, which included: ensure patient safety, communi- of the program. However, clinicians provided additional cate clinical concepts effectively, promote communica- feedback about the content of the algorithms, and to en- tion with clinicians, support patient activation, and sure patient safety. They suggested adding questions at facilitate navigation (see Table 6). This table includes the beginning of the algorithms that would identify po- information about the design objective, specific stake- tentially dangerous symptoms that needed immediate at- holder barriers related to this objective, solutions that tention and prompt patients to exit the program and call were identified to address the barrier and changes that their clinicians. We also found that clinicians disagreed were made in the user interface. A discussion surround- about what were best practices for self-management in ing each of these design objectives is found in the pain management. Some clinicians felt that patients with following section followed by some barriers to patient- pain levels of ≥ 9 should not self-manage and call their centered CDS that emerged from our findings. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 10 of 20 Table 4 Results of Usability Testing for the SAMI-Self-Care CDS Program: Patient Perspectives Usability Testing Themes CDS Tool Content* CDS Tool Examples Component** Visual appeal Comments about introductory pages Visual appeal “Some pages seem overwhelming.” that had a lot of content and design Understanding Pain severity question Written content “What does ‘bearable pain’ mean?” of terminology and terminology Medication questions for all Written content “What does ‘able and willing’ [to take a medication] mean?” symptoms and terminology Nausea and vomiting question Written content “Position change- does that mean when I lift my head up?” and terminology Medication questions Written content “Unclear about the word ‘dose’ in the questions.” and terminology Medication question Written content “Did you take the dose you were due for? Does that mean and terminology the dose time already passed or the next dose I am due for?” Pain quality question Written content “Define what type of pain you are referring too”,isit “pokey and terminology pain, electrical current, shock pains burning pains, etc.” Pain quality question Written content “I wouldn’t have categorized numbness as pain...I’m glad and terminology it’s there.” Pain severity question Written content “I like ‘faces’ as part of the pain scale. They make the pain and terminology measure more clear.” Constipation definition Written content “Definitions were too wordy, for example, constipation and terminology definition had too much information.” Pain medication list Written content “I have trouble understanding meaning or relevance to and terminology words such as Morphine or Opioids.” Constipation medication list Written content “Is Senna tea the same as Senna medication?” and terminology Nausea and vomiting red flag safety Written content “What are two glasses of water per day?” questions and terminology All symptom medication questions Written content “Need to add a time frame to the question: ‘Did you take and terminology your medication?’” General content related to Written content Simplicity of terminology required for some patients with introduction of program and terminology little medical sophistication makes clinical concepts difficult and definition of all symptoms to communicate and can be tedious for more medically sophisticated patients. Nausea and vomiting questions Written content “Why are you asking me about acid reflux and then position and terminology change? Are they related?” Nausea and vomiting red flag safety Written content “Why is bone marrow transplant question asked?” questions and terminology Pain red flag safety questions Written content Reason for why some questions are asked is not and terminology understood, e.g., “Not everyone has back pain.” Nausea and vomiting questions Written content “Some patients may be getting some agents that aren’t and terminology considered chemotherapy but the patient thinks they are getting chemo.” Constipation questions and Written content “I didn’t know that morphine and opioids can cause medication lists and terminology constipation.” Pain and nausea and vomiting Written content “Don’t you want to know exact time and date of medication questions and terminology [a medication] dose?” Pain questions and medication list Written content Word “narcotic” brought up negative feelings it was “a and terminology scary word.” Pain question and medication list Written content “I know narcotic is bad for you.” and terminology General comment from bilingual Written content “Is this available in Spanish?” participants and terminology Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 11 of 20 Table 4 Results of Usability Testing for the SAMI-Self-Care CDS Program: Patient Perspectives (Continued) Usability Testing Themes CDS Tool Content* CDS Tool Examples Component** Format and navigation Pain and nausea and vomiting Format and “Can we input all the medications we are taking into medication questions navigation the system?” All symptom assessment questions Format and “Add checkboxes to make this [the entry of symptoms] navigation easier.” Suggestions to improve All symptom questions Written content “Create an option of ‘I don’t know.” and terminology General comment about iPad Format and “Are there instructions for those who are not computer functionality navigation users to know how to use this function?” (Referring to functionality of hovering over a definition for more information.) General comment on iPad Format and “Use ‘back’ instead of ‘previous.’” navigation Wording of General comment for instructions for Written content “I don’t want to bother my care team.” self-management suggestions call clinicians on report and terminology General comment for instructions to Written content “When should I contact my care team?” call clinicians on report and terminology General comment for instructions to Written content “What should I tell my care team?” call clinicians on report and terminology Other Patient safety Pain report suggestions Written content “Is it safe for me to take this medication?” for report Nausea and vomiting red flag safety Written content “Are we allowed to drink 2 large glasses a liquid per day? questions for safety Shouldn’t we ask the doctor first?” questions Constipation report suggestions Written content “Is it safe for me to initiate the proposed intervention?” for suggestions Resources Constipation suggestions Other concerns “Do I have suggested medications in my home?” General comment about iPad Format and “Some people will lack the technology to access the navigation system.” * CDS tool content refers to what aspect of the CDS tool that the comment sought to improve (i.e. medication vs. pain severity question) ** CDS tool component refers to what aspect of the CDS tool that the component that the comment sought to improve (i.e. written content vs. visual appeal) Ensure patient safety graphics to support concepts in the text when appropriate. To ensure that the use of SAMI-Self-Care would not We also designated a section of the display for explana- overlook a life-threatening condition, we identified a set tions or lists so that patients who needed more informa- of screening questions for each symptom that sought to tion could easily access it. Formative testing of questions direct patients with potentially dangerous conditions to through the cognitive interviewing process helped us to stop using the algorithm and seek contact with their cli- recognize that we needed to make questions as explicit nicians. To ensure that the advice provided through and detailed as possible so patients were not left to specu- SAMI-Self-Care was sound, we derived algorithm logic late about how to respond. Finally, we used established and recommendations from evidence-based clinical interface development practices for screen appearance, guidelines. When appropriate, we also inquired if a rec- layout, and text fonts to enhance readability [45]. ommended therapy had already been suggested by the patient’s clinician. If the therapy had not been suggested, we encouraged the patients to check with their clinician Promote communication with clinicians to request approval before initiating a new medication. To encourage communication with patients’ clinicians, we included explicit suggestions regarding when to con- Communicate clinical concepts effectively tact their clinicians and a script for what to say to them. In order to communicate clinical concepts effectively, we We also advised patients to notify their clinicians within conducted cognitive testing of content with patients to en- a specified time frame about interventions they may sure an adequate level of understanding, and we added have followed from SAMI-Self-Care. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 12 of 20 Table 5 Results of Usability Testing for the SAMI-Self-Care CDS Program: Clinician Perspectives Usability Testing CDS Tool Context* CDS Tool Component** Examples Themes Visual appeal General comment related to Format “Use larger fonts and colors as a way to distinguish the look and feel of the system instructions from question.” Nausea and vomiting Format “Give a visual description of what a 16-oz container might look like, e.g., a Poland spring water bottle.” Understanding of Pain severity question Written content and preference Disapproval of wording, “bearable pain.” terminology for terminology Nausea and vomiting question Written content “Clearly indicate what issue is being evaluated, e.g., [for] position change, are we asking about getting up quickly or vertigo?” Pain question Written content “Add timeframes, e.g., did taking the pain mediation offer you relief after 30 min?” Nausea and vomiting Format “Add graphics such as [a picture of] fire in the esophagus, which doesn’t need a definition.” All symptoms Written content and format “Medication lists might be overwhelming for some patients.” Pain Written content “Offer educational explanation such as risk factors regarding why the patients shouldn’t take certain medication, e.g., for ibuprofen explain why stomach protection is needed for those 65 or older.” Format and Comment about introduction Format “Select a symptom that is bothering the patients the navigation and orientation to the program most, and then come back to evaluate other symptoms.” Nausea and vomiting question Written content and algorithms. “Prioritize question order based on frequency of This related to the issue that issues experienced by the patients to reduce number chemotherapy induced nausea of questions patients have to answer and to avoid may be more common than patients having to answer questions to symptoms position-induced nausea majority might not experience.” Comment to improve the look and feel Format and navigation “Try to reduce the number of clicks needed to move of the program the system forward, e.g., they shouldn’t have to select the symptom and press next to move forward.” Provide an introduction to Format and navigation “Tell patients upfront the different symptoms or questions so patients will medications the program will ask about.” know what to expect and why General comment related to Format and navigation “Add skip patterns for those who might have used sequencing of questions the system before.” General comment about Written content “Work on lessening redundancy of the questions.” sequence of questions Wording of Symptom reports Written content “Be clear with instructions regarding communication self-management w/ clinicians.” suggestions Symptom reports for red flag questions. Written content “Clearly indicate to the patient to call now, so they do not mistakenly think the report has been automatically sent to their clinician and that someone will follow up.” Symptom reports Written content “Educate the patient on how to use the paging service.” Symptom reports Written content “Don’t put ‘during normal business hours’ because it sounds like we’re telling patients to stop bothering us.” Symptom reports Written content “List phone number of clinician on the report or a paging service for after hours.” Other Patient safety All symptoms red flag Algorithm content Identify all red flag/emergency issues. questions developed for safety Pain red flag safety questions Algorithm content “Ask about new or severe pain not just one [or the other] and, [a] ‘yes’ [response] should mean call your doctor right away.” Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 13 of 20 Table 5 Results of Usability Testing for the SAMI-Self-Care CDS Program: Clinician Perspectives (Continued) Usability Testing CDS Tool Context* CDS Tool Component** Examples Themes General comment about reports Written content Clinicians worried that they may not be informed that are generated for self-management about patient problems. All symptoms red flag questions added Algorithm content Clinicians concerned they may miss or overlook critical situations. General comment about the report Written content “Include notification to patient that they should always call provider with questions.” General comment Written content “Wouldn’t want the patient to use the program instead of getting care.” Resources General concern about use of iPad Format Concern that some patients will not have access to computers. Pain self-management for severe pain Written content about when Lack of consensus among clinicians regarding clinical Best care practices to call their clinicians best practices. Nausea and vomiting acid reflux Algorithm content Some providers recommend medication like TUMS, but GI doctors may avoid it because it creates acid. Pain and nausea and vomiting Written content wanted more Recommended some medications be added on to comprehensive lists for lists [of medications already included]. medications Pain medication question Written content “Change dosing criteria for long acting to 8–12 h.” Pain medication lists Written content “Certain medications on the list not used across the board causing worry, e.g., fentanyl or tapentadol.” * CDS tool content refers to what aspect of the CDS tool that the comment sought to improve (i.e. medication vs. pain severity question) ** CDS tool component refers to what aspect of the CDS tool that the component that the comment sought to improve (i.e. written content vs. visual appeal) Support patient activation that emerged were reorganized around the components We sought to engage patients by providing personalized of this model which included: patient safety, cultural and actionable instructions. As appropriate, we asked pa- competency, care coordination, resource availability, and tients about resource availability and their willingness to acceptance of technology. follow interventions prior to making recommendations. Barriers pertaining to cultural competency were the Through the educational display section, we provided most common and included challenges related to un- content to help patients understand why certain questions familiar terminology, lack of reference to a clinical were being asked and why specific recommendations were framework, and unusual health beliefs. Although many provided. Helpful features included suggestions for of the concerns identified by patients were shared by cli- pharmacologic and non-pharmacologic therapies. nicians, a few items were unique to patients and their caregivers, including expressing uncertainty about some Facilitate navigation and use self-management suggestions and whether the actions To expedite SAMI-Self-Care use we identified methods would be acceptable to their clinicians, expressing nega- to optimize traversal of the algorithms and minimize the tive health beliefs associated with the use of narcotic burden of data entry (see Table 6). We included a pro- medications, and not having the suggested medications gress bar for patients to know their position in the available at home. There also were concerns about com- process, and we added capabilities to change responses munication with their clinicians, availability of technol- or to pause a session. We also added display content to ogy in the community, and use of the technology. enable a patient to recognize the context of every inter- action through tabs illustrating the symptom being ad- Discussion dressed and text clarifying the context of each question. We developed a simulated algorithm-based CDS pro- Finally, we ensured that all pathways could be traversed, gram to support symptom self-management among were unique and not redundant. adults with cancer and their caregivers. Our CDS pro- gram is more complex than other previously developed Barriers that need to be addressed to promote patient- programs and provides patents with specific information, centered CDS tailored to their situation regarding when to call their Barriers to promote patient-centered CDS regarding clinician, what to tell them, and how to self-manage symptom self-management that emerged appeared to be their symptoms. In order to accomplish this task, we consistent with the Chronic Care Model so the themes have many more decisional nodes than previous patient- Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 14 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS Design Principle Design Principle Details Examples of Solutions Change in User Interface 1 Ensure Patient Safety 1a Build algorithm content based on Use of the NCCN guidelines for cancer pain management Based on published best practices, evidence-based content used established clinical guidelines to guide algorithm content [63] for developing symptom management algorithms Iterative review process of algorithm content and recommendations by multidisciplinary expert panel members 1b Identify at the beginning of a session Additional characteristics of symptoms that suggest potentially Questions added that identify severity and trigger “call now” potentially serious conditions for which dangerous or life threatening conditions identified. e.g., “in pain advice. continued use of algorithm could be algorithm, besides enquiring about new or increased pain, CDS updated for immediate exit and to contact clinician if red harmful or life threatening adding a question that asksabout cramping or squeezing in flag was triggered chest or stomach”. Distinguished nuances between pain symptoms, (e.g. new pain Any time red flag is triggered, patient provided with specific (e.g. fracture) and chronic pain) suggestions on the screen. Disclaimer needed to ensure safety (e.g. “In case of emergency, Warning placed on the welcome page of the program. call your doctor or 911 immediately. Do not use this program Bold font used as a way to capture patients’ attention. for medical emergencies.”) There should be gradation of severity indicating what issues Visual cues added to the report to help prioritize should the patient address first self-management strategies Colors (red-orange-green) and fonts used to ensure patient reviewed specific aspect of the report. e.g. call clinician now Report provided at the end can be viewed on the screen or as a printed report 1c Inquire about appropriateness of Provide guidance on why particular intervention should not be Content modified to provide reasons why a particular recommendations prior to offering advice implemented (e.g. taking ibuprofen) intervention would not be permissible (e.g. stomach ulcers) Provide language to ensure any questions are directed to care Report content updated to contact clinician if uncertainty about team at all times concerns on implementing recommendations 2 Communicate 2a Test word selection with intended Cognitive testing of terms and its interpretation Modified wording utilized in assessment and recommendations Clinical Concepts Effectively end-users to improve understanding of concepts 2b Develop explicit, detailed questions Remove ambiguity of decision points Added specificity of timeframes to questions to improve meaning (, e.g., “Did taking short acting pain medication give you relief from your pain within 30 min of taking it?”) Reference specific medications and dosages as appropriate Designed explicit decisions points to enable machine processing 2c Enhance communication with Improve system use by reducing content Added “faces” and word anchors as part of the pain scale graphics, especially for clinical concepts Created content at a 5th grade reading level Inserted images to re-enforce concepts (e.g., stop sign for emer gency, picture to show acid reflux) 2d Provide lists to enable patient to Utilization of system could be improved by equipping patients Provided lists of most common medications in defined classes in identify specific items such as medications with necessary information a designated area of the screen for lookup as needed Included generic and brand names of medications for ease of recognition 2e Provide educational information Using CDS as a way to reinforce and provide education Educational content added in final summary reports customized to promote understanding on why certain questions are being asked to their symptoms Provided rationale of why certain questions were asked and promoted understanding Improve utility by improving layout of content Used large and “heavier” font size to make text more visible Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 15 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS (Continued) Design Principle Design Principle Details Examples of Solutions Change in User Interface 2f Enhance readability with font style, font Reduced text density size, content density, selective highlighting Used a plain white background of words Provided bolding to emphasize words 3 Promote Communication 3a Provide explicit instructions for patients Urgency of establishing clinical contact based on severity Additional features added to generate report immediately on with Clinicians regarding contacting clinicians about their of the symptom needed(e.g. call right away vs. waiting 24 h) screen if patient triggered any of the emergency “red flag” concerns questions and highlighted the importance of calling clinician NOW. Post assessment report that provides guidance on what should Immediate instructions provided to the patient, on calling be done and when. clinician, onscreen of the program and not just within the report. Added explicit language on what patients should say when calling clinician. Initial reports lost the message about the importance of Report restructured to reinforce importance of contacting communicating with the clinician clinicians and keeping them informed of regimen changes. E.g. tell your doctor or nurse you are taking 200 mg of ibuprofen as needed. Clearly communicate recommendations Report modified into sections of: do now, do next and more suggestions, to help streamline and prioritize suggestions for what the patient can do and when Lack of specification of which symptoms are 3 symptoms patients can choose in current system listed at the available for assessment at beginning of the program beginning of the program. Patients advised to contact clinicians if experiencing symptoms not addressed by the system. 3b Encourage patients to notify their Reinforce the importance of notifying clinicians about any Provided instructions about what patients should specifically tell clinical care team about interventions that interventions that have been initiated within the their clinicians about interventions they have followed recommendations 4 Support Patient 4a Determine what resources are available Improve efficiency of the system and utilization by modifying Added questions to determine what interventions had already Activation to the patient question based on what patients have available to them been prescribed Inquired if a prescription was already available for a recommended medication as a way to align with current therapy of the patient’s care team 4b Identify health beliefs that may impact Modifying how content is framed Content modified conveying meaning acceptable by patients. interpretation of content and modify (e.g. pain medication vs. narcotics) content accordingly 4c Determine what patients are willing to Improving look and feel of the system that quickly provides Provided explicit, detailed instructions that include dosage do prior to making recommendations information and allows the patient to take an active role in amounts, frequencies, medication list and lifestyle suggestions their care Prioritized display in patient report to quickly and easily inform the patient on what they should do next 4d Provide explicit, detailed, actionable Inquired about what patients were willing to do prior to instructions to the extent possible recommending an intervention, e.g., use of enemas for constipation 4e Personalize content, e.g., used Create an opportunity relate to the patient and provide Changed the text to make it personable and user friendly, e.g., possessive pronouns such as “my” or “your” self-management techniques used possessive pronouns such as “my” or “your” where where appropriate appropriate 5 Facilitate Navigation and 5a Designate consistent presentation areas Make it easy for patients to find information within the site Posted medical terms with definitions in a specific area on the Use on screen for repeated display of a specific screen so end user can easily and quickly access information as type of information needed Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 16 of 20 Table 6 Design Objectives for Development of Patient-Centered CDS (Continued) Design Principle Design Principle Details Examples of Solutions Change in User Interface Avoided “pop-ups” because they felt to be interruptive and harder to navigate for a limited computer proficient user 5b Provide comprehensive set of Ensure all possible decision points are covered Guidelines and best practices used for comprehensive coverage selection options to ensure all possible selections covered for every decision node 5c Streamline data entry Improve flow and provide feedback quickly Introduced check boxes to cut down on number of questions required to determine what advice to provide and improved efficiency 5d Optimize workflow through questions Inquiring about symptom characteristics at the beginning Enabled selection of an item on a page to advance to the next of the algorithm page as appropriate Directed patients to highly specific interventions 5e Optimize workflow through questions Inquiring about symptom characteristics at the beginning Facilitated patients starting at the appropriate place in the of the algorithm algorithm by inquiring what interventions have already been attempted Introduced check boxes to reduce number of question and reduce redundancy 5f Track progress for patient Promote efficient workflow Added progress bar showing numeric value, not just graphic representing progress Included “Go Back” function to allow patient to modify earlier responses 5g Accommodate patient changes and Offered multiple ways to start over such as: “Back to Start button” pauses as well as tabs with symptom names Included “Take a Break” button to allow patient to pause the program and come back to it again 5h Provide context for all interactions so Added tabs as a way to indicate to the patient which algorithm that patient recognizes where he/she is they were in within an algorithm Provided headers to supply context for each page anchoring the patient on where they are in a given algorithm 5i Ensure completeness and uniqueness of Provided brief overview of different topics that were covered to pathways through algorithm orient patients at the beginning of a session Re-enforced context and inter-relatedness of questions by showing question and answer from the previous page Ensured that questions allow for a single non-redundant, unique pathway for all possible scenarios Ensured that every pathway led to advice 5j Create tools that will function across Assessed target patient population to determine that 85% of patients had Created CDS tool design to function on Web, smart phone, multiple platforms access or knew how to obtain access to computers or smart phones or iPad Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 17 of 20 centered CDS systems [12, 22]. Acceptability testing re- similar to Bates and colleagues, [35] our Design Objectives sults among patients were favorable, suggesting that pa- focused on streamlining the user interaction by decreasing tients found the SAMI-Self Care prototype a satisfactory the number of questions needed to inform the algorithm, approach to for symptom self-management. Providing targeting anticipated needs with advice in real-time, pro- CDS directly to patients may be a valuable tool. Future viding supplemental information as needed but without studies should explore the best modalities for providing interruption, and relying on a user-centered iterative de- access to these tools including the Web, tablet com- sign [43]. Ozkaynak and colleagues [52] noted that an im- puters, or mobile phones [46]. We anticipate that the portant difference between patient and clinician design for SAMI-Self-Care tool could be implemented on a Web- CDS is integrating technology into the user’s work flow. based platform that could be accessible across com- Designing CDS around clinician work flow is much easier puters, tablets and smartphones. A link to this content than for patient work flow as clinicians function within a could be inserted in the homepage of a patient’s oncol- health system, whereas work flow for patients spans mul- ogy treatment group or sent out directly to patients tiple health care settings and includes home, routines of through electronic mail. daily living and communication with family, clinicians and We noted that the acceptability results from clinicians the health care system [52–55]. The CDS program that for pain were less favorable than for nausea/vomiting and was evaluated in this study attempts to bridge the gap be- constipation, and were below our desired threshold of 4.0 tween care delivered in health care settings and self-care for helpfulness of the report and suggestions. This result in the home. There is a need for applications that improve required obtaining a larger sample size to gather responses outcomes across settings and patient populations [56]. Fu- about usability testing as compared to other symptoms. In ture studies are needed to assess the impact of this CDS spite of our efforts to adhere to pain guidelines, [47]some program on patient outcomes. clinicians were reluctant to support self-management be- Through this process, we identified patient barriers to cause they wanted to remain aware of their patients’ use of CDS and clinicians’ concerns about patients using symptoms and to be directly involved in management. CDS for self-management. From these barriers and con- This finding is interesting as implementation of pain cerns, we derived Design Objectives for CDS for symptom guidelines has been a challenge. Evidence suggests that self-management, which included ensuring patient safety, pain management practices have not changed over the last communicating clinical concepts effectively, promoting twenty years and many patients with cancer continue to communication with clinicians, supporting patient activa- experience severe uncontrolled pain [48, 49]. Further tion and facilitating navigation and use. These objectives study is needed to understand clinician attitudes toward may be useful to inform the design of CDS systems for pain management and barriers to encouraging patient symptom self-management of other conditions. self-care management. One concern that clinicians The barriers we identified for CDS for cancer symptom expressed is that they wanted to be aware of patients who self-management had similarities and differences to barriers had severe levels of pain. The addition of a system that associated with self-management in patients with non- alerts clinicians when patient symptoms pass a predeter- malignant conditions [57, 58]. Similar barriers included lack mined level of severity would address this concern. Clee- of knowledge, poor communication between patients and land and colleagues [13] found that automated symptom clinicians, and logistical issues in obtaining care. However, severity alerts after surgery for lung cancer reduced symp- in contrast to the patients who did not have cancer, the tom distress. In a recent study that examined clinician cancer patients did not report barriers related to physical preferences for CDS, the use of an alerting system for in- limitations, financial constraints, a need for social and emo- creased symptom severity was identified as a desirable tional support, or challenges adhering to treatment [57, 58]. component [50]. Other differences in concerns that we noted between pa- Our design objectives were specifically developed in this tients with cancer and those with non-malignant conditions study to support development of effective patient-focused were that cancer patients and their caregivers expressed CDS. To date, CDS design objectives have focused on use concerns related to ensuring patient safety and negative by clinicians. Thus, even though our CDS system was de- health beliefs about use of narcotic pain medications. Shu- signed for patients, we sought to include features known macher and colleagues [59, 60] examined pain management to be associated with effective CDS systems for clinicians processes among cancer patients and their caregivers and [35, 36, 51]. The system features identified in our study found similar concerns related to ensuring patient safety, that were similar to clinician CDS included: graphics to especially in the context of narcotic pain medications. Pa- enhance understanding of content; explicit, actionable rec- tients and their caregivers had little interaction with clini- ommendations provided at the point of decision-making; cians in the home setting and had to master complex tasks presentation of advice that cultivated trust by providing an related to taking their medications and reported that they explanation of medical logic if needed [36]. In addition, felt that they had to be the final safety check. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 18 of 20 Limitations Author contributions MEC participated in conception of the project, data collection, analyses and The sample for this study was drawn from one institu- interpretation of data, writing and revising the manuscript, final approval of tion using purposive sampling so it is not representative the manuscript and is accountable for the quality of work. JLA participated of all cancer patients. All of the patients and their care- in conception of the project, analyses and interpretation of data, writing and revising the manuscript, final approval of the manuscript and is accountable givers in this study indicated that they used the Internet for the quality of work. DLB participated in conception of the project, for seeking health information at least sometimes. This interpretation of data, writing and revising the manuscript, final approval of rate is higher than the 72% of Internet users found in the manuscript and is accountable for the quality of work. MSR participated in conception of the project, interpretation of data, writing and revising the the 2014 Pew Research survey [61]. Our rates for Inter- manuscript, final approval of the manuscript and is accountable for the net usage may be higher as the majority of our sample quality of work. IMB participated in conception of the project, interpretation had greater than a high school education. Factors associ- of data, writing and revising the manuscript, final approval of the manuscript and is accountable for the quality of work. Patient stakeholder, JP, ated with lower access of the Internet are older age, participated in conception of the project, interpretation of data, writing and lower education and lower income [62]. Another limita- revising the manuscript, final approval of the manuscript and is accountable tion is that the majority of the sample was Caucasian for the quality of work. MMN participated in data collection, analyses and interpretation of data, writing and revising the manuscript, final approval of and had higher levels of education. Further testing of the manuscript and is accountable for the quality of work. DFL participated this approach in a more diverse group of patients with in conception of the project, analyses and interpretation of data, writing and cancer is warranted. Although the use of a simulated revising the manuscript, final approval of the manuscript and is accountable for the quality of work. All authors read and approved the final manuscript. model provided a practical and economical approach to iterative development of the CDS tool, this approach did Funding not allow for real time navigation through the algo- This work was supported by The Patient Centered Outcomes Research rithms for patients. This would be the next step in the Institute Grant PI-12-001. development process. Availability of data and materials The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Conclusions The boundaries of health care are expanding and pa- Ethics approval and consent to participate This study was approved by the Dana-Farber Cancer Institute Institutional Re- tients and their caregivers often have to self-manage view board (12–300). Informed consent was obtained from all individual par- complex cancer care at home. Thus, patient-centered ticipants included in the study. decision support that meets this need is important and timely. Our system provides tailored information that in- Competing interests The authors declare that they have no competing interests. forms patients when to call their clinicians, provides a script about what to tell tem about their symptom and specific suggestions about how the self manage their Publisher’sNote Springer Nature remains neutral with regard to jurisdictional claims in symptoms at home. Patients and their caregivers rated published maps and institutional affiliations. SAMI-Self-Care as highly acceptable and found the rec- ommendations helpful. The patient-centered CDS design Author details The Phyllis F. Cantor Center, Dana-Farber Cancer Institute, 450 Brookline objectives we derived for cancer symptom self- Avenue, Boston, MA 02115, USA. Department of Psychosocial Oncology and management may be applicable for the self-management Palliative Care, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, of other conditions. MA 02115, USA. The Phyllis F. Cantor Center and the Department of Medicine, Dana-Farber Cancer Institute, 450 Brookline Ave, LW-512, Boston, MA 02115, USA. Lowe Center for Thoracic Oncology, Dana-Farber Cancer Institute, 450 Brookline Avenue, Boston, MA 02115, USA. Klesis Healthcare Abbreviations and Department of Family Medicine, Durham, NC 27705, USA. Department CDS: Clinical decision support; SAMI-Self-Care: Symptom Assessment and of Family Medicine, Duke University Medical Center, 2100 Erwin Road, Management Intervention - Self Care Durham, NC 27710, USA. Received: 4 August 2017 Accepted: 27 April 2018 Acknowledgements We acknowledged the significant contributions of the expert panel in selecting the patient-focused decision-support project and refining the con- tent and design of the decision support tools specified through this project. References In addition to the co-authors of this manuscript, members of this expert 1. Institute of Medicine Committee on Quality of Health Care in A. Crossing panel included Annmarie Uliano, Richard Boyajian, MS; Jane Brzozowski, MS; the quality chasm: a new health system for the 21st century. Washington and Drs. Aymen Elfiky, Anne Gross, Joseph Jacobson, Christopher Lathan, (DC): National Academies Press (US) Copyright 2001 by the National and Larry Shulman. We also acknowledge the contributions of the Dana- Academy of Sciences. All rights reserved; 2001. Farber Harvard Comprehensive Cancer Center Health Communication Core 2. Patient Self-Management Support Programs: An Evaluation. Final Contract of for their contributions to the design and development of the SAMI-SC Report [http://www.orau.gov/ahrq/sms_report_08.asp?p=browse_guide]. prototype. The authors also wish to thank the participants who shared their Accessed Feb 2016. valuable time and effort. We appreciate the data collection contributions of 3. McCorkle R, Ercolano E, Lazenby M, Schulman-Green D, Schilling LS, Lorig K, research coordinator Maribel Melendez and Taylor Hendel for manuscript Wagner EH. Self-management: enabling and empowering patients living preparation. with cancer as a chronic illness. CA Cancer J Clin. 2011;61(1):50–62. Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 19 of 20 4. Schulman-Green D, Bradley EH, Knobf MT, Prigerson H, DiGiovanna MP, 25. Ruland CM, Maffei RM, Borosund E, Krahn A, Andersen T, Grimsbo GH. McCorkle R. Self-management and transitions in women with advanced Evaluation of different features of an eHealth application for personalized breast cancer. J Pain Symptom Manag. 2011;42(4):517–25. illness management support: cancer patients' use and appraisal of 5. Schulman-Green D, Bradley EH, Nicholson NR, Jr., George E, Indeck A, usefulness. Int J Med Inform. 2013;82(7):593–603. McCorkle R: One step at a time: self-management and transitions among 26. Creswell JW, Plano Clark VL. Designing and conducting mixed methods women with ovarian cancer. Oncol Nurs Forum 2012, 39(4):354–360. research, 2nd edn. Thousand Oaks: Sage Publications; 2011. 6. Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management 27. Fervers B, Burgers JS, Haugh MC, Latreille J, Mlika-Cabanne N, Paquet L, Coulombe M, Poirier M, Burnand B. Adaptation of clinical guidelines: of chronic disease in primary care. JAMA. 2002;288(19):2469–75. 7. Warsi A, Wang PS, LaValley MP, Avorn J, Solomon DH. Self-management literature review and proposition for a framework and procedure. Int J Qual education programs in chronic disease: a systematic review and methodological Health Care. 2006;18(3):167–76. critique of the literature. Arch Intern Med. 2004;164(15):1641–9. 28. Amer YS, Elzalabany MM, Omar TI, Ibrahim AG, Dowidar NL. The 'Adapted 8. Lorig KR, Sobel DS, Stewart AL, Brown BW Jr, Bandura A, Ritter P, Gonzalez ADAPTE': an approach to improve utilization of the ADAPTE guideline VM, Laurent DD, Holman HR. Evidence suggesting that a chronic disease adaptation resource toolkit in the Alexandria Center for Evidence-Based self-management program can improve health status while reducing Clinical Practice Guidelines. J Eval Clin Pract. 2015;21(6):1095–106. hospitalization: a randomized trial. Med Care. 1999;37(1):5–14. 29. Fervers B, Burgers JS, Voellinger R, Brouwers M, Browman GP, Graham ID, 9. McKay HGG, Russell E, Feil EG, Boles SM, Barrera M Jr. Internet-based Harrison MB, Latreille J, Mlika-Cabane N, Paquet L, et al. Guideline diabetes self-management and support: initial outcomes from the diabetes adaptation: an approach to enhance efficiency in guideline development network project. Rehabil Psychol. 2002;47(1):31–48. and improve utilisation. BMJ Qual Saf. 2011;20(3):228–36. 30. Ruegg TA. A nurse practitioner-led urgent care center: meeting the needs 10. Mooney KH, Beck SL, Friedman RH, Farzanfar R, Wong B. Automated of the patient with cancer. Clin J Oncol Nurs. 2013;17(4):E52–7. monitoring of symptoms during ambulatory chemotherapy and oncology providers' use of the information: a randomized controlled clinical trial. 31. Cooley ME, Lobach DF, Johns E, Halpenny B, Saunders TA, Del Fiol G, Rabin Support Care Cancer. 2014;22(9):2343–50. MS, Calarese P, Berenbaum IL, Zaner K, et al. Creating computable 11. Ruland CM, Andersen T, Jeneson A, Moore S, Grimsbo GH, Borosund E, Ellison algorithms for symptom management in an outpatient thoracic oncology MC. Effects of an internet support system to assist cancer patients in reducing setting. J Pain Symptom Manag. 2013;46(6):911–24. e911 symptom distress: a randomized controlled trial. Cancer Nurs. 2013;36(1):6–17. 32. Scariot CA, Heemann A, Padovani S. Understanding the collaborative- 12. Berry DL, Hong F, Halpenny B, Partridge A, Fox E, Fann JR, Wolpin S, Lober WB, participatory design. Work (Reading, Mass). 2012;41(Suppl 1):2701–5. Bush N, Parvathaneni U, et al. The electronic self report assessment and 33. Revere D, Dixon BE, Hills R, Williams JL, Grannis SJ. Leveraging health intervention for cancer: promoting patient verbal reporting of symptom and information exchange to improve population health reporting processes: quality of life issues in a randomized controlled trial. BMC Cancer. 2014;14:513. lessons in using a collaborative-participatory design process. EGEMS (Wash, DC). 2014;2(3):1082. 13. Cleeland CS, Wang XS, Shi Q, Mendoza TR, Wright SL, Berry MD, Malveaux D, 34. Collins D. Pretesting survey instruments: an overview of cognitive methods. Shah PK, Gning I, Hofstetter WL, et al. Automated symptom alerts reduce Qual Life Res Int J Qual Life Asp Treat Care Rehab. 2003;12(3):229–38. postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. J Clin Oncol Off J Am Soc Clin Oncol. 2011;29(8):994–1000. 35. Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, 14. Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management Khorasani R, Tanasijevic M, Middleton B. Ten commandments for effective approaches for people with chronic conditions: a review. Patient Educ clinical decision support: making the practice of evidence-based medicine a Couns. 2002;48(2):177–87. reality. J Am Med Inform Assoc. 2003;10(6):523–30. 15. Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ. 36. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice Effectiveness of web-based interventions on patient empowerment: a using clinical decision support systems: a systematic review of trials to systematic review and meta-analysis. J Med Internet Res. 2010;12(2):e23. identify features critical to success. BMJ. 2005;330(7494):765. 37. Crabtree BF, Yanoshik MK, Miller WL, O'Connor PJ. Successful focus groups: 16. Gustafson DH, Hawkins R, McTavish F, Pingree S, Chen WC, Volrathongchai K, advancing the state of the art. Thousand Oaks: SAGE Publications, Inc.; 1993. Stengle W, Stewart JA, Serlin RC. Internet-based interactive support for Cancer patients: are integrated systems better? J Commun. 2008;58(2):238–57. 38. Virzi R. Refining the test phase of usability evaluation: how many subjects is 17. Marie N, Luckett T, Davidson PM, Lovell M, Lal S. Optimal patient enough? Hum Factors. 1992;34(4):457–68. education for cancer pain: a systematic review and theory-based meta- 39. Tariman JD, Berry DL, Halpenny B, Wolpin S, Schepp K. Validation and analysis. Support Care Cancer. 2013;21(12):3529–37. testing of the acceptability E-scale for web-based patient-reported 18. Yun YH, Lee KS, Kim YW, Park SY, Lee ES, Noh DY, Kim S, Oh JH, Jung SY, outcomes in cancer care. Appl Nurs Res. 2011;24(1):53–8. Chung KW et al: Web-based tailored education program for disease-free 40. Berry DL, Halpenny B, Wolpin S, Davison BJ, Ellis WJ, Lober WB, McReynolds J, cancer survivors with cancer-related fatigue: a randomized controlled trial. J Wulff J. Development and evaluation of the personal patient profile-prostate Clin Oncol Off J Am Soc Clin Oncol 2012, 30(12):1296–1303. (P3P), a web-based decision support system for men newly diagnosed with localized prostate cancer. J Med Internet Res. 2010;12(4):e67. 19. Lewis FM, Brandt PA, Cochrane BB, Griffith KA, Grant M, Haase JE, Houldin AD, Post-White J, Zahlis EH, Shands ME. The enhancing connections 41. Mullen KH, Berry DL, Zierler BK. Computerized symptom and quality-of-life program: a six-state randomized clinical trial of a cancer parenting program. assessment for patients with cancer part II: acceptability and usability. Oncol Nurs Forum. 2004;31(5):E84–9. J Consult Clin Psychol. 2015;83(1):12–23. 20. Ruland CM, Jeneson A, Andersen T, Andersen R, Slaughter L, Bente Schjodt 42. Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008; O, Moore SM. Designing tailored internet support to assist cancer patients 62(1):107–15. in illness management. AMIA Annu Symp Proc. 2007;2007:635–9. 43. Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care 21. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A model in the new millennium: thus far, the evidence on the chronic care roadmap for national action on clinical decision support. J Am Med Inform model is encouraging, but we need better tools to help practices improve Assoc. 2007;14(2):141–5. their systems. Health affairs (Project Hope). 2009;28(1):75–85. 22. Head BA, Keeney C, Studts JL, Khayat M, Bumpous J, Pfeifer M. Feasibility 44. Lobach DF, Johns EB, Halpenny B, Saunders TA, Brzozowski J, Del Fiol G, and acceptance of a telehealth intervention to promote symptom Berry DL, Braun IM, Finn K, Wolfe J et al: Increasing complexity in rule-based management during treatment for head and neck Cancer. J Support Oncol. clinical decision support: the symptom assessment and management 2011;9(1):e1–e11. intervention. JMIR Med Inform 2016, 4(4):e36. 45. Johnson CM, Johnson TR, Zhang J. A user-centered framework for 23. Weaver A, Young AM, Rowntree J, Townsend N, Pearson S, Smith J, Gibson O, redesigning health care interfaces. J Biomed Inform. 2005;38(1):75–87. Cobern W, Larsen M, Tarassenko L. Application of mobile phone technology for managing chemotherapy-associated side-effects. Ann Oncol. 2007;18(11): 46. Mirkovic J, Kaufman DR, Ruland CM. Supporting cancer patients in illness 1887–92. management: usability evaluation of a mobile app. JMIR mHealth and 24. Berry DL, Hong F, Halpenny B, Partridge AH, Fann JR, Wolpin S, Lober WB, uHealth. 2014;2(3):e33. doi:https://doi.org/10.2196/mhealth.3359. Bush NE, Parvathaneni U, Back AL, et al. Electronic self-report assessment for 47. Gordon DB, Dahl JL, Miaskowski C, McCarberg B, Todd KH, Paice JA, Lipman cancer and self-care support: results of a multicenter randomized trial. J Clin AG, Bookbinder M, Sanders SH, Turk DC, et al. American pain society Oncol Off J Am Soc Clin Oncol. 2014;32(3):199–205. recommendations for improving the quality of acute and cancer pain Cooley et al. BMC Medical Informatics and Decision Making (2018) 18:31 Page 20 of 20 management: American pain society quality of care task force. Arch Intern Med. 2005;165(14):1574–80. 48. Isaac T, Stuver SO, Davis RB, Block S, Weeks JC, Berry DL, Weingart SN. Incidence of severe pain in newly diagnosed ambulatory patients with stage IV cancer. Pain Res Manag. 2012, 17;(5):347–52. 49. Zhao F, Chang VT, Cleeland C, Cleary JF, Mitchell EP, Wagner LI, Fisch MJ. Determinants of pain severity changes in ambulatory patients with cancer: an analysis from eastern cooperative oncology group trial E2Z02. J Clin Oncol. 2014;32(4):312–9. 50. Berry DL, Nayak MM, Abrahm JL, Braun I, Rabin MS, Cooley ME. Clinician perspectives on symptom and quality of life experiences of patients during cancer therapies: implications for eHealth. Psycho-Oncology. 2017; 51. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012; 45(6):1202–16. 52. Ozkaynak M, Brennan PF, Hanauer DA, Johnson S, Aarts J, Zheng K, Haque SN. Patient-centered care requires a patient-oriented workflow model. J Am Med Inform Assoc. 2013;20(e1):e14–6. 53. Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK. Promise of and potential for patient-facing technologies to enable meaningful use. Am J Prev Med. 2011;40(5 Suppl 2):S162–72. 54. Clauser SB, Wagner EH, Aiello Bowles EJ, Tuzzio L, Greene SM. Improving modern cancer care through information technology. Am J Prev Med. 2011; 40(5 Suppl 2):S198–207. 55. Valdez RS, Holden RJ, Novak LL, Veinot TC. Transforming consumer health informatics through a patient work framework: connecting patients to context. J Am Med Inform Assoc. 2015;22(1):2–10. 56. McCoy AB, Wright A, Eysenbach G, Malin BA, Patterson ES, Xu H, Sittig DF. State of the art in clinical informatics: evidence and examples. Yearb Med inform. 2013;8(1):13–9. 57. Bayliss EA, Steiner JF, Fernald DH, Crane LA, Main DS. Descriptions of barriers to self-care by persons with comorbid chronic diseases. Ann Fam Med. 2003;1(1):15–21. 58. Fort MP, Alvarado-Molina N, Pena L, Mendoza Montano C, Murrillo S, Martinez H. Barriers and facilitating factors for disease self-management: a qualitative analysis of perceptions of patients receiving care for type 2 diabetes and/or hypertension in San Jose, Costa Rica and Tuxtla Gutierrez, Mexico. BMC Fam Pract. 2013;14:131. 59. Schumacher KL, Plano Clark VL, West CM, Dodd MJ, Rabow MW, Miaskowski C. Pain medication management processes used by oncology outpatients and family caregivers part II: home and lifestyle contexts. J Pain Symptom Manag. 2014;48(5):784–96. 60. Schumacher KL, Plano Clark VL, West CM, Dodd MJ, Rabow MW, Miaskowski C. Pain medication management processes used by oncology outpatients and family caregivers part I: health systems contexts. J Pain Symptom Manag. 2014;48(5):770–83. 61. Fox S, Duggan M. Health online 2013. Pew Research Center. Internet, Science, and Tech. Internet. 15 Jan 2013. (http://www.pewinternet.org/2013/01/15/ health-online-2013/) 62. Smith A, PRCI. Science and technology: technology adoption by lower income populations. San Diego: American Public Human Services Association, IT Solution Management; 2013. 63. NCCN. Clinical practice guidelines in oncology. Adult Cancer Pain. Version 2. 2014 [http://williams.medicine.wisc.edu/pain.pdf

Journal

BMC Medical Informatics and Decision Making – Springer Journals

Published: May 29, 2018

There are no references for this article.