Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Effect of Targeted Behavioral Science Messages on COVID-19 Vaccination Registration Among Employees of a Large Health System

Effect of Targeted Behavioral Science Messages on COVID-19 Vaccination Registration Among... Research Letter | Infectious Diseases Effect of Targeted Behavioral Science Messages on COVID-19 Vaccination Registration Among Employees of a Large Health System A Randomized Trial Henri C. Santos, PhD; Amir Goren, PhD; Christopher F. Chabris, PhD; Michelle N. Meyer, JD, PhD Introduction Supplemental content Author affiliations and article information are The first opportunities to field test interventions to increase COVID-19 vaccination were among listed at the end of this article. health care workers (HCWs), who were among the first to be offered COVID-19 vaccines. After 1 large Pennsylvania health system sent 36 vaccine-related mass emails to employees over 5 weeks (eAppendix in Supplement 1), 9723 of 23 700 HCWs (41%) had still not scheduled their vaccination. We sought to determine whether individually addressed emails designed with behaviorally informed 1-5 features could increase vaccination registration compared with a delayed control group. Methods This project followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline. The Geisinger institutional review board determined that this health care operations project did not constitute human participants research and that a follow-up research analysis was exempt from review or the requirement for informed consent under 45 CFR §46.104(d)(4)(iii). The trial protocol is available in Supplement 2. In this randomized trial, we assigned 9723 employees (eFigure in Supplement 1) who had not scheduled a COVID-19 vaccination to a delayed control condition (3241 [33%] randomized; 3179 [33%] received intervention) or to receive 1 of 2 individually addressed emails with 3 components. Both emails explained that Pennsylvania would soon expand vaccine eligibility beyond HCWs, reducing employees’ access to appointments, and encouraged them to schedule an appointment. The 6482 employees in these intervention groups were assigned to receive an email that framed the decision to be vaccinated either by noting that many US residents and fellow employees had chosen to be vaccinated, ie, social norms (3241 [33%] assigned; 3198 [33%] received intervention) or by favorably juxtaposing the vaccine’s risks with those of COVID-19, ie, reframing risks (3241 [33%] assigned; 3190 [33%] received intervention). Both emails asked employees to make an active choice to receive a vaccine (hyperlinked to a scheduling portal) or not (hyperlinked to a survey soliciting their primary reason for declining). Employees in the delayed condition were randomly assigned to receive 1 of these emails (social norms: 1589 [50%]; reframing risks: 1589 [50%]) 3 days later. The primary outcome was registration on the vaccination scheduling portal during the 3 days after the first emails were sent. Random assignment (with the randomizr package) and logistic regression analyses were conducted using R version 4.0.2 (R Project for Statistical Computing). For all analyses, odds ratios (ORs) from logistic regressions were calculated, along with asymptotic 95% CIs; 2-tailed P < .05 was used to determine statistical significance. Detailed methods appear in the eAppendix in Supplement 1. Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 1/4 JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System Results The overall employee population of 23 700 HCWs comprised 17 362 (73%) women and 21 168 (89%) White employees, with a mean age of 43 years. Of the 9723 targeted employees, 9457 (97%) had valid email addresses. Both emails (ie, social norms and reframing risks) led to more registrations in the first 3 days than the delayed condition (delayed control group: percentage of participants registering: 3.17%; 95% CI, 2.62%-3.85%; social norms: percentage of participants registering, 6.47%; 95% CI, 5.67%-7.38%; OR, 2.11; 95% CI, 1.65-2.69; P < .001; reframing risks: percentage of participants registering, 6.90%; 95% CI, 6.07%-7.83%; OR, 2.26; 95% CI, 1.77-2.87; P < .001) (Figure 1). There was no significant difference in registrations between the 2 email conditions (OR, 1.07; 95% CI, 0.88-1.30; P = .50). Among the 1229 HCWs who declined to register and then Figure 1. Vaccine Appointment Registrations by Email Condition A Registrations during first 3 d of study B Cumulative registrations during study period Email phase A: Social norms 300 A2: Social norms B: Reframing risks B2: Reframing risks No contact Panel A shows registrations during the first 3 days of 4 the study (January 15, 2021, 4:55 PM, to January 18, 2021, 4:40 PM). Error bars represent asymptotic 95% CIs. Panel B shows cumulative registrations over time. Email 2 and corrected email 2 were only sent to participants in A2 and B2. The lines for A2 and B2 represent registrations made by the delayed control group after they were sent the social norms and reframing risks emails, respectively. Data collection ended on January 19, 2020, at 3:19 PM, when the Delayed Social Reframing provided link no longer forwarded employees to the Jan 16 Jan 17 Jan 18 Jan 19 Jan 20 control norms risks employee registration portal. Figure 2. Main Reasons for Declining to Schedule Vaccination Concerns about unknown risks Concerns about pregnancy or breastfeeding Other Had COVID-19, vaccine unnecessary Depends when offered; want to wait and see Concerns about the vaccine’s ingredients (eg, mRNA) Allergies that could react to the vaccine Known side effects (eg, headache, fatigue) Preexisting medical condition Don’t trust rushed FDA process Multiple reasons Not at high risk for serious COVID-19 disease Concerns about efficacy Already vaccinated Religious reasons Overall, 239 of 1229 respondents (19%) who declined Not at high risk of COVID-19 infection to receive the vaccine provided free-text reasons Work out of state; will receive when available here rather than choose 1 of the reasons listed in the survey. Privacy concerns about Geisinger/state tracking Most of these were recategorized into existing reasons Others should be prioritized or 1 of 4 new categories: multiple reasons, already Depends which vaccine vaccinated, religious reasons, and others should be 0 200 400 600 prioritized. The remaining respondents were left in the No. of respondents other category. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 2/4 Vaccine appointment registrations, % Cumulative registrations, No. Email 1, 4:55 PM EST Email 2, 4:40 PM EST Corrected email 2, 12:49 PM EST JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System completed the survey, the most common reasons were unknown vaccine risks (430 [35%]) and pregnancy-related concerns (165 [13%]) (Figure 2). Discussion During the 3-day study period, an individual email nudge caused more than twice as many HCWs to register for a COVID-19 vaccination compared with HCWs in the control condition, with no significant difference between the 2 emails. A limitation of this trial is that due to the imminent closure of employee-only vaccination clinics, we could only delay the intervention in the control group by 3 days. Moreover, by choosing to compare 2 behaviorally informed emails, we are unable to exclude the possibility that a plain reminder might have had the same effect. Furthermore, we could not measure actual vaccination, as appointment slots were unexpectedly unavailable for many who registered for one. Given the large volume of previous COVID-19 vaccine promotion to HCWs, it may seem counterintuitive that a single additional reminder could increase vaccination by late adopters. However, competing demands on attention, behavioral inertia, and unwieldy processes that make it hard to follow through on intentions likely conspire to make a single, timely, targeted reminder 2-5 helpful. The emails’ behavioral features—active choice, appeal to authority, and emphases on scarcity, social norms, and risk recalibration (eAppendix in Supplement 1)—may have contributed to their effect. The 3.17% absolute increase in vaccination appointments we observed is larger than many real-world nudges and may be greater among recipients who are less hesitant about receiving the vaccine. Sending targeted emails, patient portal messages, or text messages designed with behavioral science is inexpensive, scalable, and easily implemented and could be an effective way to encourage vaccination by HCWs and the general public. ARTICLE INFORMATION Accepted for Publication: May 24, 2021. Published: July 28, 2021. doi:10.1001/jamanetworkopen.2021.18702 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Santos HC et al. JAMA Network Open. Corresponding Author: Michelle N. Meyer, JD, PhD, Center for Translational Bioethics and Health Care Policy, Geisinger Health System, 100 N Academy Ave, Danville, PA 17822 (michellenmeyer@gmail.com). Author Affiliations: Behavioral Insights Team, Steele Institute for Health Innovation, Geisinger Health System, Danville, Pennsylvania (Santos, Goren, Chabris, Meyer); Autism and Developmental Medicine Institute, Geisinger Health System, Lewisburg, Pennsylvania (Chabris); Center for Translational Bioethics and Health Care Policy, Geisinger Health System, Danville, Pennsylvania (Meyer). Author Contributions: Dr Santos had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Chabris and Meyer contributed equally. Concept and design: All authors. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Santos, Chabris, Meyer. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Santos, Goren. Obtained funding: Meyer. Administrative, technical, or material support: Chabris. Supervision: Goren, Chabris, Meyer. Conflict of Interest Disclosures: None reported. Trial Registration: ClinicalTrials.gov Identifier: NCT04728594 Data Sharing Statement: See Supplement 3. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 3/4 JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System Additional Contributions: We thank Marguerite Macpherson, BS, Elizabeth S. Humphrey, BA, and Peter Rowe, BA, for administering the emails; Greg Strevig, AAS, for help in obtaining data for analysis; and Daniel Rosica, BA, Tamara Gjorgjieva, BS, and Maheen Shermohammed, PhD, for research assistance. All contributors are affiliated with Geisinger Health System, and none received compensation for this work other than their usual employment salary. Additional Information: The data, materials, and reproducible code used in this study will be deposited to OSF: https://osf.io/qg5m6/. REFERENCES 1. Milkman KL, Patel MS, Gandhi L, et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc Natl Acad SciUSA. 2021;118(20):e2101165118. doi:10.1073/ pnas.2101165118 2. Moehring A, Collis A, Garimella K, et al. Surfacing norms to increase vaccine acceptance. psyArXiv. Preprint updated March 19, 2021. doi:10.31234/osf.io/srv6t 3. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211(4481): 453-458. doi:10.1126/science.7455683 4. Patel MS, Volpp KG, Small DS, et al. Using active choice within the electronic health record to increase influenza vaccination rates. J Gen Intern Med. 2017;32(7):790-795. doi:10.1007/s11606-017-4046-6 5. Bakr O, Afsar-Manesh N, Raja N, et al. Application of behavioral economics principles improves participation in mailed outreach for colorectal cancer screening. Clin Transl Gastroenterol. 2020;11(1):e00115. doi:10.14309/ctg. 6. DellaVigna S, Linos E. RCTs to scale: comprehensive evidence from two nudge units. NBER Working Paper. July 28, 2020. Accessed June 22, 2021. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3661086 SUPPLEMENT 1. eAppendix. Supplementary Methods eFigure. Study Flowchart eReferences. SUPPLEMENT 2. Trial Protocol SUPPLEMENT 3. Data Sharing Statement JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 4/4 1 Sending Emails Designed with Behavioral Science to 2 Increase COVID-19 Vaccination (NCT04728594) 4 Study Protocol with Statistical Analysis Plan 6 May 21, 2021 7 2 8 Study Protocol 10 Purpose 12 The purpose of the study was to evaluate, prospectively, the potential impact of different 13 email message conditions (social norms, reframing risk) on registration for COVID-19 14 vaccination by Geisinger Health System employees who have not yet made an appointment. 16 Introduction 18 Laboratory studies of interventions to increase COVID-19 vaccination intentions and 20 science can contribute to reaching population immunity. The first opportunities to field test such 21 interventions in the COVID-19 context are with healthcare workers (HCWs), who are among the 22 first to be offered COVID-19 vaccines and are important ambassadors for COVID-19 vaccine 23 acceptance in the general population. While most HCWs employed by a large Pennsylvania 24 health system reported intentions to get vaccinated, many were hesitant due to concerns about 25 vaccine side effects and unknown risks and wanted to wait and see how others fared with 26 vaccination. Tha -related 27 mass messages to all employees over five weeks (supplement). After this effort, around 60% had 28 scheduled their vaccinations. We targeted the remainder, testing two direct emails designed with 29 behaviorally-informed features to promote vaccination against a delayed control group. 31 Methods 33 Sample 35 Eligible employees were those without a COVID-19 appointment on record. Sample size 36 was determined by the number of eligible employees (if our study had not been conducted, all 37 would have received one email from the health system promoting vaccination). 39 Experimental and control conditions 41 All emails were designed to be a personal appeal from a medical expert and authority 42 figure in the health system. The first paragraph of the emails emphasized the upcoming scarcity 43 of the vaccines, and the options encouraged employees to make an active choice of whether to 44 get the vaccine or not. Those who clicked a link to get the vaccine were automatically sent to a 45 scheduling portal. Those who did not choose to get the vaccine were automatically sent to an 46 online survey. 48 The online survey presented 16 reasons for COVID-19 vaccine hesitancy. After 49 respondents selected their main reason for hesitancy, we presented explanations that aimed to 51 FAQ on the vaccines. After reading this explanation, respondents were given another chance to 52 register for an appointment. The online survey was anonymous, so we could not link responses 53 to experimental condition or actual registration for the vaccine. 3 55 Social norms. In the social norms condition, employees received an email from the 56 -oriented subject 57 -19 vaccine plan," and contained the following content: 59 Hi [first name] , 61 - 63 large group of people. As a result, we expect vaccine appointment availability for 64 employees to soon become very limited, so I strongly encourage you to schedule your 65 appointment as soon as possible. 67 More than 11 million Americans have received a COVID-19 vaccine. This includes more 68 than 14,000 of your Geisinger colleagues including more than 80% of providers like 69 me and more are already scheduled. 71 Please choose from one of the following options: 73 - Yes, I want a COVID-19 vaccine. 74 - -19. 76 Stay safe, 78 Dr. Stan Martin, Division Chief, Infectious Diseases 80 Reframing Risk. In the reframing risk condition, employees received an email from the 81 -of-fact subject line: 84 Hi [first name] , 86 - 87 is a 88 large group of people. As a result, we expect vaccine appointment availability for 89 employees to soon become very limited, so I strongly encourage you to schedule your 90 appointment as soon as possible. 92 I understand that you may be concerned about side effects of the vaccines. Mild side 93 effects, like headache, soreness and low-grade fever are not uncommon; they show that 94 the vaccine is starting to work. According to the CDC, so far across the country, serious 95 reactions to the vaccine have been rare about 1 in every 100,000 people have had one. 96 By contrast, COVID-19 can cause severe complications or have serious known and 97 unknown long-term effects, even among people who had mild symptoms. 99 Please choose from one of the following options: 4 101 - Yes, I want a COVID-19 vaccine. 102 - -19. 104 Stay safe, 106 Dr. Stan Martin, Division Chief, Infectious Diseases 108 Delayed control. Participants in the delayed control group did not receive the emails 109 until January 18, 2021. A data pull of the primary outcome was planned for January 18, 2021, 110 but the data was not able to be reliably gathered at that time. Without any reliable data on the 111 performance of the two groups , we decided to evenly 112 randomize the delayed control group to both email versions. The emails sent to the delayed 114 was revised to inform employees about the end of the employee-specific distribution of vaccines: 116 Tomorrow is the final day you can easily receive a COVID-19 vaccine as an employee. 118 offering COVID-19 vaccines to all tho 119 groups, a large group of people. As a result, we expect vaccine appointment availability 120 for employees to soon become very limited, so I strongly encourage you to schedule your 121 appointment as soon as possible. 123 The following day a correction to the emails sent to the delayed control group was sent; it 125 of the email in red and boldface (the correction did not change any information about the 126 distribution plan or upcoming deadline): 128 CORRECTION 129 approved by the state to receive vaccines beginning Wednesday. Phase 1B group 130 approval has not happened yet. 133 Outcome measures 135 The primary outcome measure was registration for a COVID-19 vaccination made 136 through the employee scheduling portal. The secondary outcomes were whether the email was 137 opened and whether the link in it was clicked. All measures were binary and measured by the 138 time the delayed group was emailed (12:49 EST, January 18, 2021) and again by the time the 139 link no longer sent respondents to the online registration portal (13:19 EST, January 19, 2021). 140 The full study period, including primary and exploratory analysis of the outcomes four days after 141 the intervention, was from 16:55 EST, January 15, 2021 to 13:19 EST, January 19, 2021. 143 Registration during the first three days was the ultimate outcome of interest, since it 144 reflected the behavioral choice that could lead to vaccination, and by limiting analysis to this 145 period we could measure the effect of the two email interventions relative to the control group. 5 147 In our ClinicalTrials.gov record, we preregistered that the study would examine data after 148 two days. This was extended to about three days, which was the time when the delayed control 149 group was emailed. This altered timing was dictated by when the Geisinger Marketing 150 department was ready to send emails to the delayed control group. Data for email opens and link 151 clicks, as well as registrations, opens, and link clicks were recorded and were uploaded on the 152 Open Science Framework. 154 Statistical Analysis Plan 156 The researchers hypothesized that more people receiving either of the emails designed 157 with behavioral science will register for COVID-19 vaccinations than those the delayed-contact 158 control group. 160 Binary logistic generalized linear models (GLMs) were used to analyze registration, 161 email open, and email link click rates as a function of experimental condition. The data was 162 analyzed with logistic regression models with the control group as the reference group, to 163 compare the two email conditions versus the control group. This set of analyses was only 164 conducted for scheduling a vaccination appointment as opposed to email engagement outcomes 165 (e.g., number of emails opened), which was not applicable for the control group. A second set of 166 logistic regression models predicting scheduling a vaccination appointment and email 167 engagement were run, comparing the two email conditions against each other. This comparison 168 was exploratory since we had no reason to predict one message would be superior to the other. 170 Odds ratios (ORs) were calculated, along with asymptotic 95% confidence intervals 171 (CIs); two-tailed p-values < 0.05 were used to determine statistical significance. Raw 172 percentages with asymptotic 95% CIs were also presented in graphs. All analyses were 173 conducted in R. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JAMA Network Open American Medical Association

Effect of Targeted Behavioral Science Messages on COVID-19 Vaccination Registration Among Employees of a Large Health System

Download PDF
21 pages

Loading next page...
 
/lp/american-medical-association/effect-of-targeted-behavioral-science-messages-on-covid-19-vaccination-6kfxXv5GNF

References (14)

Publisher
American Medical Association
Copyright
Copyright 2021 Santos HC et al. JAMA Network Open.
eISSN
2574-3805
DOI
10.1001/jamanetworkopen.2021.18702
Publisher site
See Article on Publisher Site

Abstract

Research Letter | Infectious Diseases Effect of Targeted Behavioral Science Messages on COVID-19 Vaccination Registration Among Employees of a Large Health System A Randomized Trial Henri C. Santos, PhD; Amir Goren, PhD; Christopher F. Chabris, PhD; Michelle N. Meyer, JD, PhD Introduction Supplemental content Author affiliations and article information are The first opportunities to field test interventions to increase COVID-19 vaccination were among listed at the end of this article. health care workers (HCWs), who were among the first to be offered COVID-19 vaccines. After 1 large Pennsylvania health system sent 36 vaccine-related mass emails to employees over 5 weeks (eAppendix in Supplement 1), 9723 of 23 700 HCWs (41%) had still not scheduled their vaccination. We sought to determine whether individually addressed emails designed with behaviorally informed 1-5 features could increase vaccination registration compared with a delayed control group. Methods This project followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline. The Geisinger institutional review board determined that this health care operations project did not constitute human participants research and that a follow-up research analysis was exempt from review or the requirement for informed consent under 45 CFR §46.104(d)(4)(iii). The trial protocol is available in Supplement 2. In this randomized trial, we assigned 9723 employees (eFigure in Supplement 1) who had not scheduled a COVID-19 vaccination to a delayed control condition (3241 [33%] randomized; 3179 [33%] received intervention) or to receive 1 of 2 individually addressed emails with 3 components. Both emails explained that Pennsylvania would soon expand vaccine eligibility beyond HCWs, reducing employees’ access to appointments, and encouraged them to schedule an appointment. The 6482 employees in these intervention groups were assigned to receive an email that framed the decision to be vaccinated either by noting that many US residents and fellow employees had chosen to be vaccinated, ie, social norms (3241 [33%] assigned; 3198 [33%] received intervention) or by favorably juxtaposing the vaccine’s risks with those of COVID-19, ie, reframing risks (3241 [33%] assigned; 3190 [33%] received intervention). Both emails asked employees to make an active choice to receive a vaccine (hyperlinked to a scheduling portal) or not (hyperlinked to a survey soliciting their primary reason for declining). Employees in the delayed condition were randomly assigned to receive 1 of these emails (social norms: 1589 [50%]; reframing risks: 1589 [50%]) 3 days later. The primary outcome was registration on the vaccination scheduling portal during the 3 days after the first emails were sent. Random assignment (with the randomizr package) and logistic regression analyses were conducted using R version 4.0.2 (R Project for Statistical Computing). For all analyses, odds ratios (ORs) from logistic regressions were calculated, along with asymptotic 95% CIs; 2-tailed P < .05 was used to determine statistical significance. Detailed methods appear in the eAppendix in Supplement 1. Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 1/4 JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System Results The overall employee population of 23 700 HCWs comprised 17 362 (73%) women and 21 168 (89%) White employees, with a mean age of 43 years. Of the 9723 targeted employees, 9457 (97%) had valid email addresses. Both emails (ie, social norms and reframing risks) led to more registrations in the first 3 days than the delayed condition (delayed control group: percentage of participants registering: 3.17%; 95% CI, 2.62%-3.85%; social norms: percentage of participants registering, 6.47%; 95% CI, 5.67%-7.38%; OR, 2.11; 95% CI, 1.65-2.69; P < .001; reframing risks: percentage of participants registering, 6.90%; 95% CI, 6.07%-7.83%; OR, 2.26; 95% CI, 1.77-2.87; P < .001) (Figure 1). There was no significant difference in registrations between the 2 email conditions (OR, 1.07; 95% CI, 0.88-1.30; P = .50). Among the 1229 HCWs who declined to register and then Figure 1. Vaccine Appointment Registrations by Email Condition A Registrations during first 3 d of study B Cumulative registrations during study period Email phase A: Social norms 300 A2: Social norms B: Reframing risks B2: Reframing risks No contact Panel A shows registrations during the first 3 days of 4 the study (January 15, 2021, 4:55 PM, to January 18, 2021, 4:40 PM). Error bars represent asymptotic 95% CIs. Panel B shows cumulative registrations over time. Email 2 and corrected email 2 were only sent to participants in A2 and B2. The lines for A2 and B2 represent registrations made by the delayed control group after they were sent the social norms and reframing risks emails, respectively. Data collection ended on January 19, 2020, at 3:19 PM, when the Delayed Social Reframing provided link no longer forwarded employees to the Jan 16 Jan 17 Jan 18 Jan 19 Jan 20 control norms risks employee registration portal. Figure 2. Main Reasons for Declining to Schedule Vaccination Concerns about unknown risks Concerns about pregnancy or breastfeeding Other Had COVID-19, vaccine unnecessary Depends when offered; want to wait and see Concerns about the vaccine’s ingredients (eg, mRNA) Allergies that could react to the vaccine Known side effects (eg, headache, fatigue) Preexisting medical condition Don’t trust rushed FDA process Multiple reasons Not at high risk for serious COVID-19 disease Concerns about efficacy Already vaccinated Religious reasons Overall, 239 of 1229 respondents (19%) who declined Not at high risk of COVID-19 infection to receive the vaccine provided free-text reasons Work out of state; will receive when available here rather than choose 1 of the reasons listed in the survey. Privacy concerns about Geisinger/state tracking Most of these were recategorized into existing reasons Others should be prioritized or 1 of 4 new categories: multiple reasons, already Depends which vaccine vaccinated, religious reasons, and others should be 0 200 400 600 prioritized. The remaining respondents were left in the No. of respondents other category. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 2/4 Vaccine appointment registrations, % Cumulative registrations, No. Email 1, 4:55 PM EST Email 2, 4:40 PM EST Corrected email 2, 12:49 PM EST JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System completed the survey, the most common reasons were unknown vaccine risks (430 [35%]) and pregnancy-related concerns (165 [13%]) (Figure 2). Discussion During the 3-day study period, an individual email nudge caused more than twice as many HCWs to register for a COVID-19 vaccination compared with HCWs in the control condition, with no significant difference between the 2 emails. A limitation of this trial is that due to the imminent closure of employee-only vaccination clinics, we could only delay the intervention in the control group by 3 days. Moreover, by choosing to compare 2 behaviorally informed emails, we are unable to exclude the possibility that a plain reminder might have had the same effect. Furthermore, we could not measure actual vaccination, as appointment slots were unexpectedly unavailable for many who registered for one. Given the large volume of previous COVID-19 vaccine promotion to HCWs, it may seem counterintuitive that a single additional reminder could increase vaccination by late adopters. However, competing demands on attention, behavioral inertia, and unwieldy processes that make it hard to follow through on intentions likely conspire to make a single, timely, targeted reminder 2-5 helpful. The emails’ behavioral features—active choice, appeal to authority, and emphases on scarcity, social norms, and risk recalibration (eAppendix in Supplement 1)—may have contributed to their effect. The 3.17% absolute increase in vaccination appointments we observed is larger than many real-world nudges and may be greater among recipients who are less hesitant about receiving the vaccine. Sending targeted emails, patient portal messages, or text messages designed with behavioral science is inexpensive, scalable, and easily implemented and could be an effective way to encourage vaccination by HCWs and the general public. ARTICLE INFORMATION Accepted for Publication: May 24, 2021. Published: July 28, 2021. doi:10.1001/jamanetworkopen.2021.18702 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Santos HC et al. JAMA Network Open. Corresponding Author: Michelle N. Meyer, JD, PhD, Center for Translational Bioethics and Health Care Policy, Geisinger Health System, 100 N Academy Ave, Danville, PA 17822 (michellenmeyer@gmail.com). Author Affiliations: Behavioral Insights Team, Steele Institute for Health Innovation, Geisinger Health System, Danville, Pennsylvania (Santos, Goren, Chabris, Meyer); Autism and Developmental Medicine Institute, Geisinger Health System, Lewisburg, Pennsylvania (Chabris); Center for Translational Bioethics and Health Care Policy, Geisinger Health System, Danville, Pennsylvania (Meyer). Author Contributions: Dr Santos had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Chabris and Meyer contributed equally. Concept and design: All authors. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Santos, Chabris, Meyer. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Santos, Goren. Obtained funding: Meyer. Administrative, technical, or material support: Chabris. Supervision: Goren, Chabris, Meyer. Conflict of Interest Disclosures: None reported. Trial Registration: ClinicalTrials.gov Identifier: NCT04728594 Data Sharing Statement: See Supplement 3. JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 3/4 JAMA Network Open | Infectious Diseases Effect of Targeted Messages on COVID-19 Vaccination Among Employees of a Large Health System Additional Contributions: We thank Marguerite Macpherson, BS, Elizabeth S. Humphrey, BA, and Peter Rowe, BA, for administering the emails; Greg Strevig, AAS, for help in obtaining data for analysis; and Daniel Rosica, BA, Tamara Gjorgjieva, BS, and Maheen Shermohammed, PhD, for research assistance. All contributors are affiliated with Geisinger Health System, and none received compensation for this work other than their usual employment salary. Additional Information: The data, materials, and reproducible code used in this study will be deposited to OSF: https://osf.io/qg5m6/. REFERENCES 1. Milkman KL, Patel MS, Gandhi L, et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc Natl Acad SciUSA. 2021;118(20):e2101165118. doi:10.1073/ pnas.2101165118 2. Moehring A, Collis A, Garimella K, et al. Surfacing norms to increase vaccine acceptance. psyArXiv. Preprint updated March 19, 2021. doi:10.31234/osf.io/srv6t 3. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211(4481): 453-458. doi:10.1126/science.7455683 4. Patel MS, Volpp KG, Small DS, et al. Using active choice within the electronic health record to increase influenza vaccination rates. J Gen Intern Med. 2017;32(7):790-795. doi:10.1007/s11606-017-4046-6 5. Bakr O, Afsar-Manesh N, Raja N, et al. Application of behavioral economics principles improves participation in mailed outreach for colorectal cancer screening. Clin Transl Gastroenterol. 2020;11(1):e00115. doi:10.14309/ctg. 6. DellaVigna S, Linos E. RCTs to scale: comprehensive evidence from two nudge units. NBER Working Paper. July 28, 2020. Accessed June 22, 2021. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3661086 SUPPLEMENT 1. eAppendix. Supplementary Methods eFigure. Study Flowchart eReferences. SUPPLEMENT 2. Trial Protocol SUPPLEMENT 3. Data Sharing Statement JAMA Network Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 (Reprinted) July 28, 2021 4/4 1 Sending Emails Designed with Behavioral Science to 2 Increase COVID-19 Vaccination (NCT04728594) 4 Study Protocol with Statistical Analysis Plan 6 May 21, 2021 7 2 8 Study Protocol 10 Purpose 12 The purpose of the study was to evaluate, prospectively, the potential impact of different 13 email message conditions (social norms, reframing risk) on registration for COVID-19 14 vaccination by Geisinger Health System employees who have not yet made an appointment. 16 Introduction 18 Laboratory studies of interventions to increase COVID-19 vaccination intentions and 20 science can contribute to reaching population immunity. The first opportunities to field test such 21 interventions in the COVID-19 context are with healthcare workers (HCWs), who are among the 22 first to be offered COVID-19 vaccines and are important ambassadors for COVID-19 vaccine 23 acceptance in the general population. While most HCWs employed by a large Pennsylvania 24 health system reported intentions to get vaccinated, many were hesitant due to concerns about 25 vaccine side effects and unknown risks and wanted to wait and see how others fared with 26 vaccination. Tha -related 27 mass messages to all employees over five weeks (supplement). After this effort, around 60% had 28 scheduled their vaccinations. We targeted the remainder, testing two direct emails designed with 29 behaviorally-informed features to promote vaccination against a delayed control group. 31 Methods 33 Sample 35 Eligible employees were those without a COVID-19 appointment on record. Sample size 36 was determined by the number of eligible employees (if our study had not been conducted, all 37 would have received one email from the health system promoting vaccination). 39 Experimental and control conditions 41 All emails were designed to be a personal appeal from a medical expert and authority 42 figure in the health system. The first paragraph of the emails emphasized the upcoming scarcity 43 of the vaccines, and the options encouraged employees to make an active choice of whether to 44 get the vaccine or not. Those who clicked a link to get the vaccine were automatically sent to a 45 scheduling portal. Those who did not choose to get the vaccine were automatically sent to an 46 online survey. 48 The online survey presented 16 reasons for COVID-19 vaccine hesitancy. After 49 respondents selected their main reason for hesitancy, we presented explanations that aimed to 51 FAQ on the vaccines. After reading this explanation, respondents were given another chance to 52 register for an appointment. The online survey was anonymous, so we could not link responses 53 to experimental condition or actual registration for the vaccine. 3 55 Social norms. In the social norms condition, employees received an email from the 56 -oriented subject 57 -19 vaccine plan," and contained the following content: 59 Hi [first name] , 61 - 63 large group of people. As a result, we expect vaccine appointment availability for 64 employees to soon become very limited, so I strongly encourage you to schedule your 65 appointment as soon as possible. 67 More than 11 million Americans have received a COVID-19 vaccine. This includes more 68 than 14,000 of your Geisinger colleagues including more than 80% of providers like 69 me and more are already scheduled. 71 Please choose from one of the following options: 73 - Yes, I want a COVID-19 vaccine. 74 - -19. 76 Stay safe, 78 Dr. Stan Martin, Division Chief, Infectious Diseases 80 Reframing Risk. In the reframing risk condition, employees received an email from the 81 -of-fact subject line: 84 Hi [first name] , 86 - 87 is a 88 large group of people. As a result, we expect vaccine appointment availability for 89 employees to soon become very limited, so I strongly encourage you to schedule your 90 appointment as soon as possible. 92 I understand that you may be concerned about side effects of the vaccines. Mild side 93 effects, like headache, soreness and low-grade fever are not uncommon; they show that 94 the vaccine is starting to work. According to the CDC, so far across the country, serious 95 reactions to the vaccine have been rare about 1 in every 100,000 people have had one. 96 By contrast, COVID-19 can cause severe complications or have serious known and 97 unknown long-term effects, even among people who had mild symptoms. 99 Please choose from one of the following options: 4 101 - Yes, I want a COVID-19 vaccine. 102 - -19. 104 Stay safe, 106 Dr. Stan Martin, Division Chief, Infectious Diseases 108 Delayed control. Participants in the delayed control group did not receive the emails 109 until January 18, 2021. A data pull of the primary outcome was planned for January 18, 2021, 110 but the data was not able to be reliably gathered at that time. Without any reliable data on the 111 performance of the two groups , we decided to evenly 112 randomize the delayed control group to both email versions. The emails sent to the delayed 114 was revised to inform employees about the end of the employee-specific distribution of vaccines: 116 Tomorrow is the final day you can easily receive a COVID-19 vaccine as an employee. 118 offering COVID-19 vaccines to all tho 119 groups, a large group of people. As a result, we expect vaccine appointment availability 120 for employees to soon become very limited, so I strongly encourage you to schedule your 121 appointment as soon as possible. 123 The following day a correction to the emails sent to the delayed control group was sent; it 125 of the email in red and boldface (the correction did not change any information about the 126 distribution plan or upcoming deadline): 128 CORRECTION 129 approved by the state to receive vaccines beginning Wednesday. Phase 1B group 130 approval has not happened yet. 133 Outcome measures 135 The primary outcome measure was registration for a COVID-19 vaccination made 136 through the employee scheduling portal. The secondary outcomes were whether the email was 137 opened and whether the link in it was clicked. All measures were binary and measured by the 138 time the delayed group was emailed (12:49 EST, January 18, 2021) and again by the time the 139 link no longer sent respondents to the online registration portal (13:19 EST, January 19, 2021). 140 The full study period, including primary and exploratory analysis of the outcomes four days after 141 the intervention, was from 16:55 EST, January 15, 2021 to 13:19 EST, January 19, 2021. 143 Registration during the first three days was the ultimate outcome of interest, since it 144 reflected the behavioral choice that could lead to vaccination, and by limiting analysis to this 145 period we could measure the effect of the two email interventions relative to the control group. 5 147 In our ClinicalTrials.gov record, we preregistered that the study would examine data after 148 two days. This was extended to about three days, which was the time when the delayed control 149 group was emailed. This altered timing was dictated by when the Geisinger Marketing 150 department was ready to send emails to the delayed control group. Data for email opens and link 151 clicks, as well as registrations, opens, and link clicks were recorded and were uploaded on the 152 Open Science Framework. 154 Statistical Analysis Plan 156 The researchers hypothesized that more people receiving either of the emails designed 157 with behavioral science will register for COVID-19 vaccinations than those the delayed-contact 158 control group. 160 Binary logistic generalized linear models (GLMs) were used to analyze registration, 161 email open, and email link click rates as a function of experimental condition. The data was 162 analyzed with logistic regression models with the control group as the reference group, to 163 compare the two email conditions versus the control group. This set of analyses was only 164 conducted for scheduling a vaccination appointment as opposed to email engagement outcomes 165 (e.g., number of emails opened), which was not applicable for the control group. A second set of 166 logistic regression models predicting scheduling a vaccination appointment and email 167 engagement were run, comparing the two email conditions against each other. This comparison 168 was exploratory since we had no reason to predict one message would be superior to the other. 170 Odds ratios (ORs) were calculated, along with asymptotic 95% confidence intervals 171 (CIs); two-tailed p-values < 0.05 were used to determine statistical significance. Raw 172 percentages with asymptotic 95% CIs were also presented in graphs. All analyses were 173 conducted in R.

Journal

JAMA Network Open – American Medical Association

Published: Jul 28, 2021

There are no references for this article.