Randomized Trial of a Training Program to Improve Home Visitor Communication around Sensitive Topics

Randomized Trial of a Training Program to Improve Home Visitor Communication around Sensitive Topics Introduction Strong communication skills are necessary to engage families, perform accurate assessments, and motivate behavior change around sensitive issues encountered in home visiting. Methods A two-arm, cluster-randomized trial evaluated the impact of a trans-model communications training course for home visitors. Fourteen home visiting programs in Maryland were assigned to a training intervention (n = 7 programs; 30 visitors) or wait-list control group (n = 7 programs; 34 visitors). Independent observers assessed training fidelity. Visitor’s attitudes, knowledge, and confidence were assessed through surveys. Their skills were assessed through coding of video- recorded visits with standardized mothers. Data were collected at baseline, within 2 weeks post-training, and at 2 months post-training. Regression models accounted for clustering within programs and controlled for characteristics on which study groups differed at baseline. Results Independent observers rated the training highly on fidelity and acceptability. Home visitors rated it as useful, con- sistent with their model, and worth the effort. Immediately following the training, the training group scored higher than the control group on a range of indicators in all domains—knowledge, attitudes, confidence, and skills in using motivational communication techniques. At 2 months post-training, impacts on knowledge and attitudes persisted; impacts on confidence and observed skill were attenuated. Discussion The training course showed favorable immediate impacts on knowledge, attitudes, confidence, and skills, and long-term impacts on home visitor knowledge and attitudes. The findings underscore the need for ongoing reinforcement of skills following training. Keywords Home visiting · Training evaluation · Communication · Fidelity · Motivational interviewing Significance Introduction To our knowledge, this study is the first to use standardized Workforce development is a high priority in evidence-based mothers to assess observed changes in home visitor skill fol- home visiting. Building a stable, competent workforce is lowing a communication skills training course. The findings one of the top ten priorities on the national home visiting provide empirical support for the acceptability and effec- research agenda (Home Visiting Research Network 2013) tiveness of a trans-model approach to teaching motivational and one of HRSA’s four focal areas for the federal Mater- communication skills for use by home visitors when talking nal, Infant, and Early Childhood Home Visiting (MIECHV) with families about sensitive issues. Innovation Awards (Health Resources and Services Admin- istration 2016). Staff selection, training, and coaching are key elements of effective implementation systems to assure that staff are competent to carry out their roles effectively (Fixsen et al. 2005). Results from a recent meta-analytic * Allison West review showed that the quality of home visitor training and awest25@jhu.edu supervision predicted program outcomes (Casillas et  al. Department of Population, Family and Reproductive 2016). Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA Vol:.(1234567890) 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S71 One important area of workforce development concerns training course on home visitors’ knowledge, attitudes, home visitors’ ability to communicate effectively with high confidence, and communication skills around sensitive risk families around sensitive issues. Strong communication issues. We developed an innovative observational method, skills are essential to engage and earn the trust of families, the standardized mother procedure, to assess home visitor perform accurate assessments, and motivate behavior change skill. (Frankel 2001). Most evidence-based home visiting pro- grams seek to enroll families with multiple, complex needs in areas such as mental health, substance abuse, domestic violence, and poor parenting (Michalopoulous et al. 2015). Methods Evidence suggests that home visitors find it difficult to dis- cuss these topics with families (Jones Harden et al. 2010; Participants and Procedures Monteiro 2016; Tandon et al. 2005) and desire enhanced training and support to address sensitive issues effectively Maryland home visiting programs were eligible for study (Gill et al. 2007; Tandon et al. 2008). participation if they used an evidence-based model and Home visitor competence in addressing sensitive issues served pregnant women and children under the age of requires training and ongoing support to promote the use of three. Programs receiving MIECHV funding were given skills in practice. Results from a survey conducted through priority for study enrollment. Seven programs were the Home Visiting Applied Research Collaborative showed excluded because they had participated in one of the two that home visiting programs expect home visitors to have pilot versions of the training program. Two additional strong communication skills and that they expect them to programs were excluded because they had recently par- acquire these skills through training and other professional ticipated in a similar training program. Home visitors at development activities after hire (Home Visiting Applied participating programs were eligible if they had active Research Collaborative 2017). All evidence-based home caseloads and could travel to the training location. Super- visiting models require pre-service and ongoing training visors of eligible home visitors were also invited to par- (Sama-Miller et al. 2017). Most training is model-specific ticipate to enhance their capacity to reinforce skills and (Home Visiting Research Network 2013), although models support training transfer. and local implementing agencies often support or encour- A total of 19 programs were approached for study par- age home visitors to obtain additional training, certifica- ticipation between June and August 2016 (Fig. 1). The study tions, or endorsements (e.g., Child Development Associate was first introduced at meetings of local home visiting pro- or Infant Mental Health endorsement). In addition, several grams, after which the Maryland Department of Health states have developed core competencies for home visitors, emailed programs additional study details. The study coordi- and MIECHV Innovation Awards have been given to states nator then scheduled web-based study information sessions developing training to support cross-model competencies with staff at each site. Three programs did not respond to (Health Resources and Services Administration 2017). recruitment contacts. One program declined participation In 2016, the Maryland MIECHV program partnered with due to program leadership transitions. Information sessions the University of Maryland, Baltimore County (UMBC) to were held with 15 programs. One program declined par- develop and implement a communication skills certificate ticipation after the informational session. Healthy Families training course for home visitors, and with Johns Hopkins America (HFA; n = 12) and Early Head Start home-based University to conduct an independent evaluation of the (EHS; n = 2) programs were randomly assigned to the train- course. Course developers engaged home visiting models ing intervention (n = 7 programs; 29 visitors) or the wait-list and local programs as partners in developing the training, control group (n = 7 programs; 34 visitors). Programs were which was aligned with principles of adult learning (Trivette randomized by site to acknowledge the shared characteristics et  al. 2009), training transfer (Burke and Hutchins 2007; and experiences among home visitors within programs and Grossman and Salas 2011), stages of change (DiClemente to reduce the possibility of contamination effects. 2005), and motivational interviewing (Miller and Rollnick Data were collected at three points: pre-training baseline, 2012). The course’s main objective was to build home visitor within 2 weeks of training completion (follow-up 1), and knowledge, attitudes, confidence, and skills to communicate at 2-months post-training (follow-up 2; Fig. 2). Research with families around sensitive topics. teams traveled to each program to obtain informed consent This manuscript reports findings from a two-arm clus- and collect data at each time point. Participants were remu- ter-randomized trial in which 14 home visiting programs nerated for completing study activities. Study procedures were assigned to either the certificate training course or a were approved by the Institutional Review Boards of the wait-list control condition. The purpose of the study was Johns Hopkins Bloomberg School of Public Health and the to evaluate the fidelity, acceptability, and impacts of the Maryland State Department of Health. 1 3 S72 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Fig. 1 Consort diagram ≤ 2 Weeks 2 Months Fig. 2 Study design Post-Training Post-Training BaselineTraining Follow-up 1 Follow-up 2 Intervention groupXO O O 1 2 2 IC, then O R Control groupO O 2 2 NOTE: IC = Informed consent; O = Visitor baseline survey and mock visits; R = Site randomization; X = Intervention group training; O = Assessment of trainer/training fidelity; O = Assessment of HV knowledge, attitudes, confidence, and skills. 1 2 was to support participants’ development of core motiva- Training Intervention tional communication competencies in each topic area. The first two training days were held back-to-back and focused on Training developers used input from home visitors and supervisors to design the course. They solicited input basic communication and listening skills, stages of change, and motivational interviewing techniques. The last five train- through focus groups and individual interviews. In addi- tion, trainers solicited feedback regularly from a stakeholder ing days were held on a biweekly schedule. Participants who completed all modules received a certificate from UMBC’s advisory board comprised of state partners, home visiting program staff, and content matter experts. Prior to the ran- Professional Studies program. domized trial, the course was tested and refined based on results from pilot pre- and post-tests and participant feed- back on usefulness and acceptability. The final course consisted of 6 modules delivered in 7 days over 12 weeks (Table 1). The objective of each module 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S73 Table 1 Communication training course overview Module Length Description of topic area 1. Communication 2 days • Communication skills for difficult conversations with families • Stages of change and motivational interviewing 2. Healthy relationships 1 day • Identifying signs of family distress, including domestic violence • Sensitive conversations around family conflict, safety planning 3. Parenting and child development 1 day • Child development • Promoting positive parenting 4. Mental health 1 day • Assessing mental health, supporting access to services, promoting coping • Techniques for staff to promote their own mental health 5. Substance use 1 day • Addressing substance misuse • Screening and communication to promote behavior change and inform families about treatment and referral options 6. Cultural sensitivity 1 day • Working with diverse populations time points). A sample item was, “Sharing concerns about Measures a possible parenting risk is a sign of care and respect for the family.” Three additional items assessed attitudes toward dis- Fidelity and Acceptability of Training cussing concerns about parenting behaviors (α = 0.47–0.54 across the three time points). A sample item was, “Keep- A study team member observed each training session to ing silent when observing harsh parenting behavior sends a assess two dimensions of training fidelity: quality of deliv - signal that the behavior is acceptable.” Items were summed ery and participant responsiveness (James Bell Associates to create scale scores, with higher scores reflecting more 2009). The observer rated trainers on ten aspects of quality positive attitudes. on a Likert scale (1 = low quality to 5 = high quality), such as Knowledge and confidence were assessed at the two fol- pace, rapport with learners, and responsiveness to trainees’ low-up points. Items were developed to align with course questions and concerns (adapted from Healthy Teen Net- objectives and content and were reviewed by the trainers for work & RTI International 2017). The observer rated each content validity. The 36 knowledge items were presented on trainee on two dimensions of participant responsiveness: a 6-point Likert scale (1 = strongly disagree to 6 = strongly understanding and participation (1 = little understanding/low agree) and converted to dichotomous variables by recoding participation to 5 = good understanding/high participa- the disagree (1–3) and agree (4–6) response values as either tion). To assess acceptability, trainees completed a four- correct or incorrect (scored as 1 or 0, respectively). Dichoto- item Likert scale at follow-up 2 to rate their perception that mous values were added to generate a score (0–36) at each the program taught them skills that were helpful in their time point. To reduce testing effects, 17 new True/False work, they could use easily in their work, were consistent knowledge items were added at follow-up 2; items were with what their model teaches, and that the training was scored as correct (1) or incorrect (0). Thirteen confidence worth the effort (1 = strongly disagree to 4 = strongly agree). items were measured on a 7-point Likert scale (0 = very Items were summed to create on overall training accept- strongly disagree to 6 = very strongly agree); response val- ability score. ues were added to generate a score (0–78) at each time point (α = 0.95 at follow-up 1 and 0.96 at follow-up 2). For ease of Home Visitor Characteristics, Attitudes, Knowledge, interpretation, knowledge and confidence scores were con- and Confidence verted to scales ranging from 0 to 100, with higher scores reflecting greater knowledge and confidence, respectively. Home visitor characteristics, attitudes, knowledge, and confidence were assessed using self-administered surveys. Home Visitor Skill Home visitor age, educational attainment, race, ethnicity, years of experience as a home visitor, and caseload size Communication skill was assessed at all three time points were assessed at baseline. Attitudes toward discussing par- using video-recorded “mock visits” with trained actresses enting risks and concerns about parenting behaviors were serving as standardized mothers. The research team cre- assessed at all three time points. Three Likert-type items ated six scenarios depicting the following sensitive issues: assessed home visitors’ attitudes toward discussing concerns maternal depression, maternal smoking, maternal alcohol about risks for poor parenting with parents (1 = strongly use, domestic violence, parenting/spanking, and maternal disagree to 4 = strongly agree; α = 0.68–0.76 across three 1 3 S74 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 anxiety. Each scenario was designed to generate a 20–30 min assessments. Ten percent of videos were blind-coded by all conversation. Two mock visits were recorded with every coders to establish reliability. Intraclass Correlation Coef- visitor at each time point. Scenarios were paired to achieve ficients (ICCs) for 16 of 18 individual items were above variation in content, stage of change, and maternal reflective 0.60, suggesting good or excellent agreement across cod- capacity (e.g., capacity to explore and link thoughts, feel- ers (Cicchetti 1994). ICCs for avoiding labels and stereo- ings, and behavior). types and softening sustain talk were lower (0.50 and 0.13, Home visitors’ use of communication skills in mock visits respectively). The low ICC for softening sustain talk may be was assessed in two ways. First, the standardized mothers explained, in part, by low levels of sustain talk. Coders were provided global ratings of the home visitors’ skills imme- blind to home visitors’ group assignments. diately after each mock visit at follow-ups 1 and 2 by indi- cating their level of agreement with eleven statements on Analytic Approach a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree). Items were developed by the researchers or were Central tendency and variability in baseline characteristics, adapted from instruments that assessed perceptions of the training fidelity and acceptability, and outcomes were exam- working alliance (Horvath and Greenberg 1994). Sample ined using descriptive statistics. Minimal missing survey items included, “I sensed that the visitor was impatient or data (< 5%) were handled using ipsative mean imputation. frustrated with me,” and “The visitor encouraged me to There were no missing observational data. Characteristics express my thoughts and feelings.” Items were summed to of intervention and control group home visitors were com- create a total score (α = 0.89). Standardized mothers were pared for baseline equivalence using Chi square and t-tests. blind to group assignments and their ratings were not shared Variables with group differences at baseline were included with the home visitors. as covariates in all models. To assess the impact of the train- Second, trained research assistants coded the video- ing on outcomes, we used a conservative approach to mul- recorded mock visits using the Motivational Interviewing tiple regression that adjusted for clustering of home visitors Treatment Integrity (MITI) Scale version 4.1 (Moyers et al. within the 14 home visiting programs. Models estimating 2015). They coded the frequencies of ten specific behaviors attitudes and observed skills included baseline scores as (Give Information, Persuade, Persuade with Permission, covariates (knowledge and confidence were not assessed at Questions, Simple Reflection, Complex Reflection, Affirm, baseline). Seek Collaboration, Emphasize Autonomy, and Confront). The Questions code was adapted by making a distinction between Open- and Closed-ended Questions using conven- Results tions established in MITI 3.1 (Moyers et al. 2010). Four global scores (Cultivating Change Talk, Softening Sustain Baseline Characteristics Talk, Partnership, & Empathy) were assigned using a 5-point Likert scale (1 = low to 5 = high). A technical global com- The sample was diverse in demographic and work-related posite score was calculated by averaging cultivating change characteristics (Table 2). Home visitors randomized to the talk and softening sustain talk scores. A relational global control group were similar to treatment group participants in composite score was calculated by averaging Partnership race and ethnicity, educational attainment, years as a home and Empathy scores. Composite scores were calculated for visitor, attitudes toward talking with parents about parent- percentage of complex reflections, total instances of MI ing risks and behaviors, and observed communication skill. adherent behaviors (Affirm +Seek Collaboration + Empha- Participants randomized to the control group were slightly size Autonomy), and total instances of MI non-adherent older and had smaller caseloads than home visitors in the behaviors (Persuade + Confront). Finally, three new scores treatment group. were assigned to assess specific strategies that were empha- sized in the training. These included two new global scores, Elicit-Provide-Elicit and Avoiding Labels and Stereotypes, Fidelity and Acceptability and one new composite score that assessed the use of Open- ended questions, Affirmations, Reflections, and Summary All but one trainee attended every module; one trainee Statements (OARS). MITI developers and trainers reviewed missed the mental health module. Across all modules, inde- and approved adapted and new items for face and construct pendent observer ratings were very high for trainer fidelity validity. (Mean = 4.8, SD = 0.2), trainees’ levels of understanding Coders included two undergraduate and three graduate (Mean = 4.4; SD = 0.5) and trainees’ levels of participation students. Each received 27 h of formal training and par- (Mean = 4.0; SD = 0.7). At follow-up 2, trainees gave favora- ticipated in weekly and bi-weekly reliability meetings and ble ratings for acceptability (Mean = 3.6, SD = 0.5). 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S75 Table 2 Baseline characteristics Baseline characteristics Control (n = 34) Treatment (n = 29) p of home visitors by treatment group (N = 63) Age (mean, range) 40.2 (24–62) 34.9 (25–53) .04 Race and ethnicity .96  Black/African American 40% 41%  White, non-Hispanic 30% 26%  Hispanic/Latina 27% 30% Educational attainment .83  High school/GED 1 (3%) 0 (0%)  Some college/associates degree 11 (33%) 9 (31%)  Bachelor’s degree 17 (52%) 18 (62%)  Master’s degree 4 (12%) 2 (7%) Number of years as home visitor 6.3 (0.5–30) 3.3 (0.5–17) .06 Number of families in caseload 12.4 (4–19) 15.2 (0–24) .05 Associations Between Knowledge, Attitudes, Program Impacts Confidence and Skill Attitudes, Knowledge and Confidence Few consistent patterns of relationships among knowledge, attitudes, confidence, and observed skill were observed At follow-up 1, treatment group home visitors demonstrated between follow-up 1 and follow-up 2. Knowledge was more favorable attitudes than control group home visitors positively associated with more favorable attitudes toward toward talking with parents about parenting risks but not talking about parenting risks and parenting behaviors at toward talking with parents about their parenting behaviors both time points. Knowledge was also associated with one (Table 3). At follow-up 1, the treatment group also exhibited observed behavior at both time points: Elicit-Provide- higher levels of knowledge and confidence compared to the Plicit. Neither confidence nor attitudes showed any con- control group. Effects sizes ranged from medium to large sistent pattern of association with observed skills across (Cohen 1988). At follow-up 2, impacts on knowledge and time. attitudes endured, but the effects on confidence were attenu- ated and were no longer statistically significant. Table 3 Immediate and long-term training effects on attitudes, knowledge and confidence Outcome Control adjusted Treatment adjusted Coefficient Cohen’s d p mean (SD) mean (SD) Baseline (N = 53)  Attitudes: talking about parenting risks 8.3 (2.7) 7.7 (3.1) − 1.04 0.21 .07  Attitudes: talking about parenting behaviors 8.8 (2.4) 8.8 (2.0) − 0.26 0 .63 Follow-up 1 (N = 53)  Attitudes: talking about parenting risks 8.0 (2.9) 10.4 (1.5) 2.40 1.04 .01  Attitudes: talking about parenting behaviors 9.4 (1.9) 9.6 (1.9) 0.58 0.03 .22  Knowledge 76.1 (7.1) 85.1 (7.1) 7.94 1.26 .003  Confidence 74.5 (15.7) 83.5 (14.1) 8.98 0.60 .04 Follow-up 2 (N = 51)  Attitudes: talking about parenting risks 8.6 (2.6) 9.5 (2.3) 1.28 0.38 .05  Attitudes: talking about parenting behaviors 9.3 (1.9) 9.4 (2.0) 0.20 0.06 .60  Knowledge 76.1 (9.1) 84.5 (8.5) 8.32 0.96 <.001  Knowledge supplement 74.6 (14.4) 82.8 (11.1) 8.77 0.64 .02  Confidence 76.0 (17.1) 80.3 (15.4) 4.52 0.26 .37 Scale scores for knowledge and confidence had possible range of 0–100. All models controlled for age and caseload size and were adjusted for clustering at the site level. Coefficients represent treatment group; control group is reference group. Models estimating training effects on atti- tudes controlled for baseline scores on attitude scales 1 3 S76 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Observed Skills although the effect of training on avoidance of MI non- adherent strategies endured. At follow-up 1, standardized mothers gave more favora- ble ratings of their interactions with training group home visitors compared to control group visitors (p = 0.04). At Discussion follow-up 2, this difference was no longer statistically significant. At follow-up 1, the training group demon- This cluster randomized trial found that a six-module course strated favorable scores relative to the control group on had favorable and consistent immediate impacts on home the adapted MITI (Moyers et al. 2015) for five of the eight visitors’ communication knowledge, attitudes, confidence, observed communication skills. Effect sizes ranged from and observed skills. The training produced the most endur- moderate to large (Table  4). The training showed larg- ing observed effect on reducing home visitors’ use of strat- est impacts on relational global scores, use of the elicit- egies that are incompatible with theories of motivational provide-elicit strategy, and avoidance of MI non-adherent interviewing such as persuading or confronting parents strategies. By follow-up 2, most impacts were attenuated, who may not yet be open to behavior change. The findings Table 4 Immediate and long- Outcome Control Treatment Coefficient Cohen’s d p term effects on observed skills adjusted mean adjusted mean (SD) (SD) Baseline (N = 53)  Total MI adherent 4.7 (2.7) 3.9 (2.4) − 0.64 − 0.32 .21  Total MI non-adherent 7.6 (4.9) 6.7 (4.3) − 0.65 − 0.20 .62  Technical global 3.2 (0.5) 3.3 (0.5) 0.06 0.11 .65  Relational global 3.1 (0.6) 3.0 (0.7) − 0.20 − 0.22 .34  Percent complex reflections 0.4 (0.2) 0.5 (0.2) − 0.01 0.13 .80  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.7 (0.5) 2.5 (0.7) − 0.24 0.28 .09   Open-ended questions, affirma- 17.2 (8.3) 17.8 (5.3) − 0.65 0.08 .70 tions, reflections, summaries (OARS)   Avoiding labels and stereotypes 4.0 (0.6) 4.1 (0.8) 0.05 0.14 .78 Follow-up 1 (N = 53)  Total MI adherent 2.8 (2.3) 3.5 (2.3) 0.43 0.30 .38  Total MI non-adherent 7.5 (6.8) 3.2 (3.2) − 3.59 − 0.80 .02  Technical global 3.4 (0.5) 3.6 (0.3) 0.21 0.56 .02  Relational global 3.3 (0.4) 3.9 (0.5) 0.51 1.24 <.001  Percent complex reflections 0.4 (0.2) 0.6 (0.2) 0.11 0.50 <.001  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.7 (0.6) 3.3 (0.7) 0.52 0.96 .002   OARS 18.8 (7.1) 18.7 (7.2) 0.74 − 0.01 .75   Avoiding labels and stereotypes 4.1 (0.5) 4.4 (0.5) 0.34 0.56 .08 Follow-up 2 (N = 51)  Total MI adherent 3.0 (2.0) 3.1 (1.5) 0.09 0.05 .83  Total MI non-adherent 5.1 (4.4) 2.9 (2.3) − 1.80 − 0.62 .04  Technical global 3.5 (0.4) 3.7 (0.3) 0.20 0.59 .20  Relational global 3.5 (0.6) 3.8 (0.5) 0.21 0.51 .19  Percent complex reflections 0.4 (0.2) 0.5 (0.2) 0.05 0.32 .47  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.9 (0.7) 3.3 (0.6) 0.24 0.56 .29   OARS 14.8 (6.2) 16.1 (6.2) 0.56 0.21 .82   Avoiding labels and stereotypes 4.3 (0.5) 4.5 (0.3) 0.14 0.32 .40 Coefficients represent treatment group; control group is reference group. All models controlled for baseline score, age, and caseload size and were adjusted for clustering at the site level 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S77 suggest that home visitors with varying levels of education how communication strategies might be tailored when fami- and experience could learn and apply motivational commu- lies present specific challenges, such as maternal depression nication skills in simulated visits. or substance abuse. More research is needed to understand To our knowledge, this study is the first to use mock vis- multi-level factors that influence training transfer and work - its and ratings by standardized mothers to assess observed force performance, including supervision and coaching. To changes in skill following a communication skills training begin to address this, we are coding audio recordings of course for home visitors. Although more time intensive and supervision sessions to examine the extent to which skills costly, video-recorded observations are more objective than taught in the training were reinforced in supervision. More self-report measures of skill. In addition, mock visits offer research is also needed to examine links between specific two key advantages over recorded observations of real visits, interpersonal skills and family outcomes. particularly in research. First, mock visits are less intrusive We recommend that programs and researchers increase and reduce concerns regarding family privacy and confiden- the use of observational measures of home visiting and tiality. Second, mock visits allow for standardized presenta- consider mock visits as a tool by which to observe home tion of the stimulus; thus, each home visitor encounters the visitor behaviors. Prior research has shown marked vari- same mother with the same presenting issues. This study ability in communication skills in actual visits (Korfmacher demonstrated that mock visits are a feasible and effective et al. 2018). We know there is keen interest by programs in way to assess variability in home visitor communication observational measures (Duggan and O’Neill 2016). Our skills in response to scenarios depicting sensitive issues, experience suggests that these instruments have tremendous and to assess change in skills over time. potential as tools for use in training, supervision, and assess- The training’s impacts on confidence and observed skills ment. Use of these methods and instruments in research and diminished over time and did not remain consistent across practice could be tested using the practice-based research outcomes. The findings underscore the importance of ongo- network of the Home Visiting Applied Research Collabora- ing supervision, coaching, and other forms of ongoing rein- tive (2017). As the US continues to invest in the scale up of forcement to facilitate the transfer of skills to practice (Burke evidence-based home visiting, we must assure that workers and Hutchins 2007; DeRoten et al. 2013; Schwalbe et al. are competent to communicate effectively about sensitive 2014). Closer examination of the data showed that long- issues in the lives of the families served. term effects may have varied depending on the nature of Acknowledgements The Maryland Maternal, Infant, and Early Child- the scenario. Thus, attenuated effects on skills may reflect hood Home Visiting Evaluation is supported by the Department of variability in home visitors’ ability to apply skills across dif- Health and Human Services, Health Resources and Services Admin- ferent topic areas. Further research with larger samples will istration grant D89MC28267 to the Maryland Department of Health. be needed to understand more fully individual differences in We gratefully acknowledge the training developers at the University of Maryland, Baltimore County who designed and implemented the how home visitors respond to different scenarios. training program and were committed to facilitating the independent Study limitations included a small sample size which may evaluation and dissemination of results. We also thank the Maryland have reduced the ability to detect small effects. A post-hoc home visiting staff who participated in evaluation activities. power analysis using Gpower (Faul et al. 2007) indicated Open Access This article is distributed under the terms of the Crea- that our sample of 51 participants at follow-up 2 yielded tive Commons Attribution 4.0 International License (http://creat iveco sufficient power to detect a medium to large effect at the mmons.or g/licenses/b y/4.0/), which permits unrestricted use, distribu- recommended 0.80 level using simple multiple regression tion, and reproduction in any medium, provided you give appropriate (Cohen 1988). However, we used a conservative analytic credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. approach that adjusted for clustering within programs and thus reduced statistical power. In addition, although care was taken to balance content and difficulty level of the sce- narios across time points, mock visits may have varied in References perceived difficulty across assessments. Finally, self-reports of attitudes and confidence may have been prone to social Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An inte- desirability bias. grative literature review. Human Resource Development Review, 6(3), 263–296. These findings provide empirical support for the accept- Casillas, K. L., Fauchier, A., Derkash, B. T., & Garrido, E. F. (2016). ability and effectiveness of a trans-model approach to teach- Implementation of evidence-based home visiting programs aimed ing motivational communication skills for use by home at reducing child maltreatment: A meta-analytic review. Child visitors when talking with families about sensitive issues. Abuse & Neglect, 53, 64–80. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for Future research could extend this work to examine how evaluating normed and standardized assessment instruments in home visitors’ communication with families varies depend- psychology. Psychological Assessment, 6(4), 284. ing on the nature of the topic being discussed, and to test 1 3 S78 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Cohen, J. (1988). Statistical power analysis for the behavioral sciences home_visit ing_resea rch_agend a_2013_10_29_final .pdfAc cesse (2nd ed). Hillsdale, NJ: Lawrence Earlbaum Associates. d May 2, 2018. De Roten, Y., Zimmermann, G., Ortega, D., & Despland, J. N. (2013). Horvath, A. O., & Greenberg, L. S. (1989). Development and valida- Meta-analysis of the effects of MI training on clinicians’ behavior. tion of the Working Alliance Inventory. Journal of Counseling Journal of Substance Abuse Treatment, 45(2), 155–162. Psychology, 36(2), 223–233. DiClemente, C. C. (2005). Conceptual models and applied research: James Bell Associates. (2009). Measuring implementation fidelity. The ongoing contribution of the transtheoretical model. Journal (Evaluation brief). http://www.jbassoc .com/Report sPubl icatio ns/ of Addictions Nursing, 16(1–2), 5–12.Evaluation %20Br ief%20-%20Mea sur ing%20Im plemen t ation%20 Duggan, A., & O’Neill, K. (2016). Communication as a core element Fid elity _Octob %E2%80%A6.pdf. Accessed 2 May 2018. of home visiting: HARC’s assessment of the state of the art inno- Korfmacher, J., Filene, J., O’Neill, K., & Duggan, A. (2018). Assessing vation to advance the field. Arlington, VA: National Summit on visitor’s communication style through observation to promote pre- Quality in Home Visiting Programs. cision home visiting. Washington, DC: National Quality in Home Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power Visiting Summit. 3: A flexible statistical power analysis program for the social, Michalopoulos, C., Lee, H., Duggan, A., Lundquist, E., Tso, A., behavioral, and biomedical sciences. Behavior Research Methods, Crowne, S., Burrell, L., Somers, J., Filene, J., & Knox, V. (2015). 39(2), 175–191. The mother and infant home visiting program evaluation: Early Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, findings on the MIECHV program. OPRE Report 2015-11. Wash- F. (2005). Implementation research: A synthesis of the literature. ington, DC: OPRE, ACF, U.S. Department of Health and Human Tampa, FL: University of South Florida, Louis de la Parte Florida Services. Mental Health Institute, National Implementation Research Net- Miller, W. R., & Rollnick, S. (2012). Motivational interviewing: Help- work. (FMHI Publication No. 231). ing people change (3rd edn.). New York: Guilford Press. Frankel, R. M. (2001). Cracking the code: Theory and method in Monteiro, M. G. (2016). The four challenges of home visitation pro- clinical communication analysis. Health Communication, 13(1), grams: Alcohol and substance abuse, intrafamilial violence, and 101–110. mental disorders. Home visitation programs (pp.  9–11). New Gill, S., Greenberg, M. T., Moon, C., & Margraf, P. (2007). Home visi- York: Springer. tor competence, burnout, support, and client engagement. Journal Moyers, T. B., Manuel, J. K., & Ernst, D. (2015). Motivational Inter- of Human Behavior in the Social Environment, 15(1), 23–44. viewing Treatment Integrity Coding Manual 4.2.1. Unpublished Grossman, R., & Salas, E. (2011). The transfer of training: What really manual. matters. International Journal of Training and Development, Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. 15(2), 103–120. (2010). Revised Global Scales: Motivational Interviewing Treat- Harden, B. J., Denmark, N., & Saul, D. (2010). Understanding the ment Integrity Coding Manual 3.1.1. Unpublished manual. needs of staff in Head Start programs: The characteristics, per - Sama-Miller, E., Akers, L., Mraz-Esposito, A., Zukiewicz, M., Avellar, ceptions, and experiences of home visitors. Children and Youth S., Paulsell, D., & Grosso, D., P (2017). Home visiting evidence of Services Review, 32(3), 371–379. effectiveness review: Executive summary. Administration for Chil- Health Resources and Services Administration. (2016). Maternal, dren and Families: Office of Planning, Research and Evaluation. infant and early childhood home visiting program—Innovation Schwalbe, C. S., Oh, H. Y., & Zweben, A. (2014). Sustaining motiva- awards [Funding opportunity announcement]. https ://mchb.hrsa. tional interviewing: A meta-analysis of training studies. Addic- gov/fundingopp or tunities /def ault.aspx?id=68f16 89a-6096-4db6- tion, 109(8), 1287–1294. ab76-bc41e b1dc4 1e. Retrieved 2 May 2018. Tandon, S. D., Mercer, C. D., Saylor, E. L., & Duggan, A. K. (2008). Health Resources and Services Administration. (2017). FY 2017 Home Paraprofessional home visitors’ perspectives on addressing poor visiting innovation awards. https://mc hb.hrsa.gov/home-visiting/ mental health, substance abuse, and domestic violence: A qualita- innov ation award s. Accessed 2 May 2018. tive study. Early Childhood Research Quarterly, 23(3), 419–428. Healthy Teen Network & RTI International. (2017). ACT for youth Tandon, S. D., Parillo, K. M., Jenkins, C., & Duggan, A. K. (2005). center of excellence: Facilitator observation form. http://www. Formative evaluation of home visitors’ role in addressing poor actfo ryout h.net/resou rces/capp/coe-facil itato r-obser vatio n.doc. mental health, domestic violence, and substance abuse among Accessed 17 Sept 2017. low-income pregnant and parenting women. Maternal and Child Home Visiting Applied Research Collaborative. (2017). Home visiting Health Journal, 9(3), 273–283. applied research collaborative. http://www.hvrn.org/index .html. Trivette, C. M., Dunst, C. J., Hamby, D. W., & O’Herin, C. E. (2009). Accessed 17 Sept 2017. Characteristics and consequences of adult learning methods and Home Visiting Research Network. (2013). Home visiting research strategies. Winterberry Research, 2(2), 1–33. agenda. http://www.hvresear ch.or g/wp-content/uploa ds/2018/01/ 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Maternal and Child Health Journal Springer Journals

Randomized Trial of a Training Program to Improve Home Visitor Communication around Sensitive Topics

Free
9 pages

Loading next page...
 
/lp/springer_journal/randomized-trial-of-a-training-program-to-improve-home-visitor-kcOslsI80D
Publisher
Springer US
Copyright
Copyright © 2018 by The Author(s)
Subject
Medicine & Public Health; Public Health; Sociology, general; Population Economics; Pediatrics; Gynecology; Maternal and Child Health
ISSN
1092-7875
eISSN
1573-6628
D.O.I.
10.1007/s10995-018-2531-0
Publisher site
See Article on Publisher Site

Abstract

Introduction Strong communication skills are necessary to engage families, perform accurate assessments, and motivate behavior change around sensitive issues encountered in home visiting. Methods A two-arm, cluster-randomized trial evaluated the impact of a trans-model communications training course for home visitors. Fourteen home visiting programs in Maryland were assigned to a training intervention (n = 7 programs; 30 visitors) or wait-list control group (n = 7 programs; 34 visitors). Independent observers assessed training fidelity. Visitor’s attitudes, knowledge, and confidence were assessed through surveys. Their skills were assessed through coding of video- recorded visits with standardized mothers. Data were collected at baseline, within 2 weeks post-training, and at 2 months post-training. Regression models accounted for clustering within programs and controlled for characteristics on which study groups differed at baseline. Results Independent observers rated the training highly on fidelity and acceptability. Home visitors rated it as useful, con- sistent with their model, and worth the effort. Immediately following the training, the training group scored higher than the control group on a range of indicators in all domains—knowledge, attitudes, confidence, and skills in using motivational communication techniques. At 2 months post-training, impacts on knowledge and attitudes persisted; impacts on confidence and observed skill were attenuated. Discussion The training course showed favorable immediate impacts on knowledge, attitudes, confidence, and skills, and long-term impacts on home visitor knowledge and attitudes. The findings underscore the need for ongoing reinforcement of skills following training. Keywords Home visiting · Training evaluation · Communication · Fidelity · Motivational interviewing Significance Introduction To our knowledge, this study is the first to use standardized Workforce development is a high priority in evidence-based mothers to assess observed changes in home visitor skill fol- home visiting. Building a stable, competent workforce is lowing a communication skills training course. The findings one of the top ten priorities on the national home visiting provide empirical support for the acceptability and effec- research agenda (Home Visiting Research Network 2013) tiveness of a trans-model approach to teaching motivational and one of HRSA’s four focal areas for the federal Mater- communication skills for use by home visitors when talking nal, Infant, and Early Childhood Home Visiting (MIECHV) with families about sensitive issues. Innovation Awards (Health Resources and Services Admin- istration 2016). Staff selection, training, and coaching are key elements of effective implementation systems to assure that staff are competent to carry out their roles effectively (Fixsen et al. 2005). Results from a recent meta-analytic * Allison West review showed that the quality of home visitor training and awest25@jhu.edu supervision predicted program outcomes (Casillas et  al. Department of Population, Family and Reproductive 2016). Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA Vol:.(1234567890) 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S71 One important area of workforce development concerns training course on home visitors’ knowledge, attitudes, home visitors’ ability to communicate effectively with high confidence, and communication skills around sensitive risk families around sensitive issues. Strong communication issues. We developed an innovative observational method, skills are essential to engage and earn the trust of families, the standardized mother procedure, to assess home visitor perform accurate assessments, and motivate behavior change skill. (Frankel 2001). Most evidence-based home visiting pro- grams seek to enroll families with multiple, complex needs in areas such as mental health, substance abuse, domestic violence, and poor parenting (Michalopoulous et al. 2015). Methods Evidence suggests that home visitors find it difficult to dis- cuss these topics with families (Jones Harden et al. 2010; Participants and Procedures Monteiro 2016; Tandon et al. 2005) and desire enhanced training and support to address sensitive issues effectively Maryland home visiting programs were eligible for study (Gill et al. 2007; Tandon et al. 2008). participation if they used an evidence-based model and Home visitor competence in addressing sensitive issues served pregnant women and children under the age of requires training and ongoing support to promote the use of three. Programs receiving MIECHV funding were given skills in practice. Results from a survey conducted through priority for study enrollment. Seven programs were the Home Visiting Applied Research Collaborative showed excluded because they had participated in one of the two that home visiting programs expect home visitors to have pilot versions of the training program. Two additional strong communication skills and that they expect them to programs were excluded because they had recently par- acquire these skills through training and other professional ticipated in a similar training program. Home visitors at development activities after hire (Home Visiting Applied participating programs were eligible if they had active Research Collaborative 2017). All evidence-based home caseloads and could travel to the training location. Super- visiting models require pre-service and ongoing training visors of eligible home visitors were also invited to par- (Sama-Miller et al. 2017). Most training is model-specific ticipate to enhance their capacity to reinforce skills and (Home Visiting Research Network 2013), although models support training transfer. and local implementing agencies often support or encour- A total of 19 programs were approached for study par- age home visitors to obtain additional training, certifica- ticipation between June and August 2016 (Fig. 1). The study tions, or endorsements (e.g., Child Development Associate was first introduced at meetings of local home visiting pro- or Infant Mental Health endorsement). In addition, several grams, after which the Maryland Department of Health states have developed core competencies for home visitors, emailed programs additional study details. The study coordi- and MIECHV Innovation Awards have been given to states nator then scheduled web-based study information sessions developing training to support cross-model competencies with staff at each site. Three programs did not respond to (Health Resources and Services Administration 2017). recruitment contacts. One program declined participation In 2016, the Maryland MIECHV program partnered with due to program leadership transitions. Information sessions the University of Maryland, Baltimore County (UMBC) to were held with 15 programs. One program declined par- develop and implement a communication skills certificate ticipation after the informational session. Healthy Families training course for home visitors, and with Johns Hopkins America (HFA; n = 12) and Early Head Start home-based University to conduct an independent evaluation of the (EHS; n = 2) programs were randomly assigned to the train- course. Course developers engaged home visiting models ing intervention (n = 7 programs; 29 visitors) or the wait-list and local programs as partners in developing the training, control group (n = 7 programs; 34 visitors). Programs were which was aligned with principles of adult learning (Trivette randomized by site to acknowledge the shared characteristics et  al. 2009), training transfer (Burke and Hutchins 2007; and experiences among home visitors within programs and Grossman and Salas 2011), stages of change (DiClemente to reduce the possibility of contamination effects. 2005), and motivational interviewing (Miller and Rollnick Data were collected at three points: pre-training baseline, 2012). The course’s main objective was to build home visitor within 2 weeks of training completion (follow-up 1), and knowledge, attitudes, confidence, and skills to communicate at 2-months post-training (follow-up 2; Fig. 2). Research with families around sensitive topics. teams traveled to each program to obtain informed consent This manuscript reports findings from a two-arm clus- and collect data at each time point. Participants were remu- ter-randomized trial in which 14 home visiting programs nerated for completing study activities. Study procedures were assigned to either the certificate training course or a were approved by the Institutional Review Boards of the wait-list control condition. The purpose of the study was Johns Hopkins Bloomberg School of Public Health and the to evaluate the fidelity, acceptability, and impacts of the Maryland State Department of Health. 1 3 S72 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Fig. 1 Consort diagram ≤ 2 Weeks 2 Months Fig. 2 Study design Post-Training Post-Training BaselineTraining Follow-up 1 Follow-up 2 Intervention groupXO O O 1 2 2 IC, then O R Control groupO O 2 2 NOTE: IC = Informed consent; O = Visitor baseline survey and mock visits; R = Site randomization; X = Intervention group training; O = Assessment of trainer/training fidelity; O = Assessment of HV knowledge, attitudes, confidence, and skills. 1 2 was to support participants’ development of core motiva- Training Intervention tional communication competencies in each topic area. The first two training days were held back-to-back and focused on Training developers used input from home visitors and supervisors to design the course. They solicited input basic communication and listening skills, stages of change, and motivational interviewing techniques. The last five train- through focus groups and individual interviews. In addi- tion, trainers solicited feedback regularly from a stakeholder ing days were held on a biweekly schedule. Participants who completed all modules received a certificate from UMBC’s advisory board comprised of state partners, home visiting program staff, and content matter experts. Prior to the ran- Professional Studies program. domized trial, the course was tested and refined based on results from pilot pre- and post-tests and participant feed- back on usefulness and acceptability. The final course consisted of 6 modules delivered in 7 days over 12 weeks (Table 1). The objective of each module 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S73 Table 1 Communication training course overview Module Length Description of topic area 1. Communication 2 days • Communication skills for difficult conversations with families • Stages of change and motivational interviewing 2. Healthy relationships 1 day • Identifying signs of family distress, including domestic violence • Sensitive conversations around family conflict, safety planning 3. Parenting and child development 1 day • Child development • Promoting positive parenting 4. Mental health 1 day • Assessing mental health, supporting access to services, promoting coping • Techniques for staff to promote their own mental health 5. Substance use 1 day • Addressing substance misuse • Screening and communication to promote behavior change and inform families about treatment and referral options 6. Cultural sensitivity 1 day • Working with diverse populations time points). A sample item was, “Sharing concerns about Measures a possible parenting risk is a sign of care and respect for the family.” Three additional items assessed attitudes toward dis- Fidelity and Acceptability of Training cussing concerns about parenting behaviors (α = 0.47–0.54 across the three time points). A sample item was, “Keep- A study team member observed each training session to ing silent when observing harsh parenting behavior sends a assess two dimensions of training fidelity: quality of deliv - signal that the behavior is acceptable.” Items were summed ery and participant responsiveness (James Bell Associates to create scale scores, with higher scores reflecting more 2009). The observer rated trainers on ten aspects of quality positive attitudes. on a Likert scale (1 = low quality to 5 = high quality), such as Knowledge and confidence were assessed at the two fol- pace, rapport with learners, and responsiveness to trainees’ low-up points. Items were developed to align with course questions and concerns (adapted from Healthy Teen Net- objectives and content and were reviewed by the trainers for work & RTI International 2017). The observer rated each content validity. The 36 knowledge items were presented on trainee on two dimensions of participant responsiveness: a 6-point Likert scale (1 = strongly disagree to 6 = strongly understanding and participation (1 = little understanding/low agree) and converted to dichotomous variables by recoding participation to 5 = good understanding/high participa- the disagree (1–3) and agree (4–6) response values as either tion). To assess acceptability, trainees completed a four- correct or incorrect (scored as 1 or 0, respectively). Dichoto- item Likert scale at follow-up 2 to rate their perception that mous values were added to generate a score (0–36) at each the program taught them skills that were helpful in their time point. To reduce testing effects, 17 new True/False work, they could use easily in their work, were consistent knowledge items were added at follow-up 2; items were with what their model teaches, and that the training was scored as correct (1) or incorrect (0). Thirteen confidence worth the effort (1 = strongly disagree to 4 = strongly agree). items were measured on a 7-point Likert scale (0 = very Items were summed to create on overall training accept- strongly disagree to 6 = very strongly agree); response val- ability score. ues were added to generate a score (0–78) at each time point (α = 0.95 at follow-up 1 and 0.96 at follow-up 2). For ease of Home Visitor Characteristics, Attitudes, Knowledge, interpretation, knowledge and confidence scores were con- and Confidence verted to scales ranging from 0 to 100, with higher scores reflecting greater knowledge and confidence, respectively. Home visitor characteristics, attitudes, knowledge, and confidence were assessed using self-administered surveys. Home Visitor Skill Home visitor age, educational attainment, race, ethnicity, years of experience as a home visitor, and caseload size Communication skill was assessed at all three time points were assessed at baseline. Attitudes toward discussing par- using video-recorded “mock visits” with trained actresses enting risks and concerns about parenting behaviors were serving as standardized mothers. The research team cre- assessed at all three time points. Three Likert-type items ated six scenarios depicting the following sensitive issues: assessed home visitors’ attitudes toward discussing concerns maternal depression, maternal smoking, maternal alcohol about risks for poor parenting with parents (1 = strongly use, domestic violence, parenting/spanking, and maternal disagree to 4 = strongly agree; α = 0.68–0.76 across three 1 3 S74 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 anxiety. Each scenario was designed to generate a 20–30 min assessments. Ten percent of videos were blind-coded by all conversation. Two mock visits were recorded with every coders to establish reliability. Intraclass Correlation Coef- visitor at each time point. Scenarios were paired to achieve ficients (ICCs) for 16 of 18 individual items were above variation in content, stage of change, and maternal reflective 0.60, suggesting good or excellent agreement across cod- capacity (e.g., capacity to explore and link thoughts, feel- ers (Cicchetti 1994). ICCs for avoiding labels and stereo- ings, and behavior). types and softening sustain talk were lower (0.50 and 0.13, Home visitors’ use of communication skills in mock visits respectively). The low ICC for softening sustain talk may be was assessed in two ways. First, the standardized mothers explained, in part, by low levels of sustain talk. Coders were provided global ratings of the home visitors’ skills imme- blind to home visitors’ group assignments. diately after each mock visit at follow-ups 1 and 2 by indi- cating their level of agreement with eleven statements on Analytic Approach a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree). Items were developed by the researchers or were Central tendency and variability in baseline characteristics, adapted from instruments that assessed perceptions of the training fidelity and acceptability, and outcomes were exam- working alliance (Horvath and Greenberg 1994). Sample ined using descriptive statistics. Minimal missing survey items included, “I sensed that the visitor was impatient or data (< 5%) were handled using ipsative mean imputation. frustrated with me,” and “The visitor encouraged me to There were no missing observational data. Characteristics express my thoughts and feelings.” Items were summed to of intervention and control group home visitors were com- create a total score (α = 0.89). Standardized mothers were pared for baseline equivalence using Chi square and t-tests. blind to group assignments and their ratings were not shared Variables with group differences at baseline were included with the home visitors. as covariates in all models. To assess the impact of the train- Second, trained research assistants coded the video- ing on outcomes, we used a conservative approach to mul- recorded mock visits using the Motivational Interviewing tiple regression that adjusted for clustering of home visitors Treatment Integrity (MITI) Scale version 4.1 (Moyers et al. within the 14 home visiting programs. Models estimating 2015). They coded the frequencies of ten specific behaviors attitudes and observed skills included baseline scores as (Give Information, Persuade, Persuade with Permission, covariates (knowledge and confidence were not assessed at Questions, Simple Reflection, Complex Reflection, Affirm, baseline). Seek Collaboration, Emphasize Autonomy, and Confront). The Questions code was adapted by making a distinction between Open- and Closed-ended Questions using conven- Results tions established in MITI 3.1 (Moyers et al. 2010). Four global scores (Cultivating Change Talk, Softening Sustain Baseline Characteristics Talk, Partnership, & Empathy) were assigned using a 5-point Likert scale (1 = low to 5 = high). A technical global com- The sample was diverse in demographic and work-related posite score was calculated by averaging cultivating change characteristics (Table 2). Home visitors randomized to the talk and softening sustain talk scores. A relational global control group were similar to treatment group participants in composite score was calculated by averaging Partnership race and ethnicity, educational attainment, years as a home and Empathy scores. Composite scores were calculated for visitor, attitudes toward talking with parents about parent- percentage of complex reflections, total instances of MI ing risks and behaviors, and observed communication skill. adherent behaviors (Affirm +Seek Collaboration + Empha- Participants randomized to the control group were slightly size Autonomy), and total instances of MI non-adherent older and had smaller caseloads than home visitors in the behaviors (Persuade + Confront). Finally, three new scores treatment group. were assigned to assess specific strategies that were empha- sized in the training. These included two new global scores, Elicit-Provide-Elicit and Avoiding Labels and Stereotypes, Fidelity and Acceptability and one new composite score that assessed the use of Open- ended questions, Affirmations, Reflections, and Summary All but one trainee attended every module; one trainee Statements (OARS). MITI developers and trainers reviewed missed the mental health module. Across all modules, inde- and approved adapted and new items for face and construct pendent observer ratings were very high for trainer fidelity validity. (Mean = 4.8, SD = 0.2), trainees’ levels of understanding Coders included two undergraduate and three graduate (Mean = 4.4; SD = 0.5) and trainees’ levels of participation students. Each received 27 h of formal training and par- (Mean = 4.0; SD = 0.7). At follow-up 2, trainees gave favora- ticipated in weekly and bi-weekly reliability meetings and ble ratings for acceptability (Mean = 3.6, SD = 0.5). 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S75 Table 2 Baseline characteristics Baseline characteristics Control (n = 34) Treatment (n = 29) p of home visitors by treatment group (N = 63) Age (mean, range) 40.2 (24–62) 34.9 (25–53) .04 Race and ethnicity .96  Black/African American 40% 41%  White, non-Hispanic 30% 26%  Hispanic/Latina 27% 30% Educational attainment .83  High school/GED 1 (3%) 0 (0%)  Some college/associates degree 11 (33%) 9 (31%)  Bachelor’s degree 17 (52%) 18 (62%)  Master’s degree 4 (12%) 2 (7%) Number of years as home visitor 6.3 (0.5–30) 3.3 (0.5–17) .06 Number of families in caseload 12.4 (4–19) 15.2 (0–24) .05 Associations Between Knowledge, Attitudes, Program Impacts Confidence and Skill Attitudes, Knowledge and Confidence Few consistent patterns of relationships among knowledge, attitudes, confidence, and observed skill were observed At follow-up 1, treatment group home visitors demonstrated between follow-up 1 and follow-up 2. Knowledge was more favorable attitudes than control group home visitors positively associated with more favorable attitudes toward toward talking with parents about parenting risks but not talking about parenting risks and parenting behaviors at toward talking with parents about their parenting behaviors both time points. Knowledge was also associated with one (Table 3). At follow-up 1, the treatment group also exhibited observed behavior at both time points: Elicit-Provide- higher levels of knowledge and confidence compared to the Plicit. Neither confidence nor attitudes showed any con- control group. Effects sizes ranged from medium to large sistent pattern of association with observed skills across (Cohen 1988). At follow-up 2, impacts on knowledge and time. attitudes endured, but the effects on confidence were attenu- ated and were no longer statistically significant. Table 3 Immediate and long-term training effects on attitudes, knowledge and confidence Outcome Control adjusted Treatment adjusted Coefficient Cohen’s d p mean (SD) mean (SD) Baseline (N = 53)  Attitudes: talking about parenting risks 8.3 (2.7) 7.7 (3.1) − 1.04 0.21 .07  Attitudes: talking about parenting behaviors 8.8 (2.4) 8.8 (2.0) − 0.26 0 .63 Follow-up 1 (N = 53)  Attitudes: talking about parenting risks 8.0 (2.9) 10.4 (1.5) 2.40 1.04 .01  Attitudes: talking about parenting behaviors 9.4 (1.9) 9.6 (1.9) 0.58 0.03 .22  Knowledge 76.1 (7.1) 85.1 (7.1) 7.94 1.26 .003  Confidence 74.5 (15.7) 83.5 (14.1) 8.98 0.60 .04 Follow-up 2 (N = 51)  Attitudes: talking about parenting risks 8.6 (2.6) 9.5 (2.3) 1.28 0.38 .05  Attitudes: talking about parenting behaviors 9.3 (1.9) 9.4 (2.0) 0.20 0.06 .60  Knowledge 76.1 (9.1) 84.5 (8.5) 8.32 0.96 <.001  Knowledge supplement 74.6 (14.4) 82.8 (11.1) 8.77 0.64 .02  Confidence 76.0 (17.1) 80.3 (15.4) 4.52 0.26 .37 Scale scores for knowledge and confidence had possible range of 0–100. All models controlled for age and caseload size and were adjusted for clustering at the site level. Coefficients represent treatment group; control group is reference group. Models estimating training effects on atti- tudes controlled for baseline scores on attitude scales 1 3 S76 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Observed Skills although the effect of training on avoidance of MI non- adherent strategies endured. At follow-up 1, standardized mothers gave more favora- ble ratings of their interactions with training group home visitors compared to control group visitors (p = 0.04). At Discussion follow-up 2, this difference was no longer statistically significant. At follow-up 1, the training group demon- This cluster randomized trial found that a six-module course strated favorable scores relative to the control group on had favorable and consistent immediate impacts on home the adapted MITI (Moyers et al. 2015) for five of the eight visitors’ communication knowledge, attitudes, confidence, observed communication skills. Effect sizes ranged from and observed skills. The training produced the most endur- moderate to large (Table  4). The training showed larg- ing observed effect on reducing home visitors’ use of strat- est impacts on relational global scores, use of the elicit- egies that are incompatible with theories of motivational provide-elicit strategy, and avoidance of MI non-adherent interviewing such as persuading or confronting parents strategies. By follow-up 2, most impacts were attenuated, who may not yet be open to behavior change. The findings Table 4 Immediate and long- Outcome Control Treatment Coefficient Cohen’s d p term effects on observed skills adjusted mean adjusted mean (SD) (SD) Baseline (N = 53)  Total MI adherent 4.7 (2.7) 3.9 (2.4) − 0.64 − 0.32 .21  Total MI non-adherent 7.6 (4.9) 6.7 (4.3) − 0.65 − 0.20 .62  Technical global 3.2 (0.5) 3.3 (0.5) 0.06 0.11 .65  Relational global 3.1 (0.6) 3.0 (0.7) − 0.20 − 0.22 .34  Percent complex reflections 0.4 (0.2) 0.5 (0.2) − 0.01 0.13 .80  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.7 (0.5) 2.5 (0.7) − 0.24 0.28 .09   Open-ended questions, affirma- 17.2 (8.3) 17.8 (5.3) − 0.65 0.08 .70 tions, reflections, summaries (OARS)   Avoiding labels and stereotypes 4.0 (0.6) 4.1 (0.8) 0.05 0.14 .78 Follow-up 1 (N = 53)  Total MI adherent 2.8 (2.3) 3.5 (2.3) 0.43 0.30 .38  Total MI non-adherent 7.5 (6.8) 3.2 (3.2) − 3.59 − 0.80 .02  Technical global 3.4 (0.5) 3.6 (0.3) 0.21 0.56 .02  Relational global 3.3 (0.4) 3.9 (0.5) 0.51 1.24 <.001  Percent complex reflections 0.4 (0.2) 0.6 (0.2) 0.11 0.50 <.001  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.7 (0.6) 3.3 (0.7) 0.52 0.96 .002   OARS 18.8 (7.1) 18.7 (7.2) 0.74 − 0.01 .75   Avoiding labels and stereotypes 4.1 (0.5) 4.4 (0.5) 0.34 0.56 .08 Follow-up 2 (N = 51)  Total MI adherent 3.0 (2.0) 3.1 (1.5) 0.09 0.05 .83  Total MI non-adherent 5.1 (4.4) 2.9 (2.3) − 1.80 − 0.62 .04  Technical global 3.5 (0.4) 3.7 (0.3) 0.20 0.59 .20  Relational global 3.5 (0.6) 3.8 (0.5) 0.21 0.51 .19  Percent complex reflections 0.4 (0.2) 0.5 (0.2) 0.05 0.32 .47  Training-specific MITI adaptations   Using elicit-provide-elicit strategy 2.9 (0.7) 3.3 (0.6) 0.24 0.56 .29   OARS 14.8 (6.2) 16.1 (6.2) 0.56 0.21 .82   Avoiding labels and stereotypes 4.3 (0.5) 4.5 (0.3) 0.14 0.32 .40 Coefficients represent treatment group; control group is reference group. All models controlled for baseline score, age, and caseload size and were adjusted for clustering at the site level 1 3 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 S77 suggest that home visitors with varying levels of education how communication strategies might be tailored when fami- and experience could learn and apply motivational commu- lies present specific challenges, such as maternal depression nication skills in simulated visits. or substance abuse. More research is needed to understand To our knowledge, this study is the first to use mock vis- multi-level factors that influence training transfer and work - its and ratings by standardized mothers to assess observed force performance, including supervision and coaching. To changes in skill following a communication skills training begin to address this, we are coding audio recordings of course for home visitors. Although more time intensive and supervision sessions to examine the extent to which skills costly, video-recorded observations are more objective than taught in the training were reinforced in supervision. More self-report measures of skill. In addition, mock visits offer research is also needed to examine links between specific two key advantages over recorded observations of real visits, interpersonal skills and family outcomes. particularly in research. First, mock visits are less intrusive We recommend that programs and researchers increase and reduce concerns regarding family privacy and confiden- the use of observational measures of home visiting and tiality. Second, mock visits allow for standardized presenta- consider mock visits as a tool by which to observe home tion of the stimulus; thus, each home visitor encounters the visitor behaviors. Prior research has shown marked vari- same mother with the same presenting issues. This study ability in communication skills in actual visits (Korfmacher demonstrated that mock visits are a feasible and effective et al. 2018). We know there is keen interest by programs in way to assess variability in home visitor communication observational measures (Duggan and O’Neill 2016). Our skills in response to scenarios depicting sensitive issues, experience suggests that these instruments have tremendous and to assess change in skills over time. potential as tools for use in training, supervision, and assess- The training’s impacts on confidence and observed skills ment. Use of these methods and instruments in research and diminished over time and did not remain consistent across practice could be tested using the practice-based research outcomes. The findings underscore the importance of ongo- network of the Home Visiting Applied Research Collabora- ing supervision, coaching, and other forms of ongoing rein- tive (2017). As the US continues to invest in the scale up of forcement to facilitate the transfer of skills to practice (Burke evidence-based home visiting, we must assure that workers and Hutchins 2007; DeRoten et al. 2013; Schwalbe et al. are competent to communicate effectively about sensitive 2014). Closer examination of the data showed that long- issues in the lives of the families served. term effects may have varied depending on the nature of Acknowledgements The Maryland Maternal, Infant, and Early Child- the scenario. Thus, attenuated effects on skills may reflect hood Home Visiting Evaluation is supported by the Department of variability in home visitors’ ability to apply skills across dif- Health and Human Services, Health Resources and Services Admin- ferent topic areas. Further research with larger samples will istration grant D89MC28267 to the Maryland Department of Health. be needed to understand more fully individual differences in We gratefully acknowledge the training developers at the University of Maryland, Baltimore County who designed and implemented the how home visitors respond to different scenarios. training program and were committed to facilitating the independent Study limitations included a small sample size which may evaluation and dissemination of results. We also thank the Maryland have reduced the ability to detect small effects. A post-hoc home visiting staff who participated in evaluation activities. power analysis using Gpower (Faul et al. 2007) indicated Open Access This article is distributed under the terms of the Crea- that our sample of 51 participants at follow-up 2 yielded tive Commons Attribution 4.0 International License (http://creat iveco sufficient power to detect a medium to large effect at the mmons.or g/licenses/b y/4.0/), which permits unrestricted use, distribu- recommended 0.80 level using simple multiple regression tion, and reproduction in any medium, provided you give appropriate (Cohen 1988). However, we used a conservative analytic credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. approach that adjusted for clustering within programs and thus reduced statistical power. In addition, although care was taken to balance content and difficulty level of the sce- narios across time points, mock visits may have varied in References perceived difficulty across assessments. Finally, self-reports of attitudes and confidence may have been prone to social Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An inte- desirability bias. grative literature review. Human Resource Development Review, 6(3), 263–296. These findings provide empirical support for the accept- Casillas, K. L., Fauchier, A., Derkash, B. T., & Garrido, E. F. (2016). ability and effectiveness of a trans-model approach to teach- Implementation of evidence-based home visiting programs aimed ing motivational communication skills for use by home at reducing child maltreatment: A meta-analytic review. Child visitors when talking with families about sensitive issues. Abuse & Neglect, 53, 64–80. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for Future research could extend this work to examine how evaluating normed and standardized assessment instruments in home visitors’ communication with families varies depend- psychology. Psychological Assessment, 6(4), 284. ing on the nature of the topic being discussed, and to test 1 3 S78 Maternal and Child Health Journal (2018) 22 (Suppl 1):S70–S78 Cohen, J. (1988). Statistical power analysis for the behavioral sciences home_visit ing_resea rch_agend a_2013_10_29_final .pdfAc cesse (2nd ed). Hillsdale, NJ: Lawrence Earlbaum Associates. d May 2, 2018. De Roten, Y., Zimmermann, G., Ortega, D., & Despland, J. N. (2013). Horvath, A. O., & Greenberg, L. S. (1989). Development and valida- Meta-analysis of the effects of MI training on clinicians’ behavior. tion of the Working Alliance Inventory. Journal of Counseling Journal of Substance Abuse Treatment, 45(2), 155–162. Psychology, 36(2), 223–233. DiClemente, C. C. (2005). Conceptual models and applied research: James Bell Associates. (2009). Measuring implementation fidelity. The ongoing contribution of the transtheoretical model. Journal (Evaluation brief). http://www.jbassoc .com/Report sPubl icatio ns/ of Addictions Nursing, 16(1–2), 5–12.Evaluation %20Br ief%20-%20Mea sur ing%20Im plemen t ation%20 Duggan, A., & O’Neill, K. (2016). Communication as a core element Fid elity _Octob %E2%80%A6.pdf. Accessed 2 May 2018. of home visiting: HARC’s assessment of the state of the art inno- Korfmacher, J., Filene, J., O’Neill, K., & Duggan, A. (2018). Assessing vation to advance the field. Arlington, VA: National Summit on visitor’s communication style through observation to promote pre- Quality in Home Visiting Programs. cision home visiting. Washington, DC: National Quality in Home Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power Visiting Summit. 3: A flexible statistical power analysis program for the social, Michalopoulos, C., Lee, H., Duggan, A., Lundquist, E., Tso, A., behavioral, and biomedical sciences. Behavior Research Methods, Crowne, S., Burrell, L., Somers, J., Filene, J., & Knox, V. (2015). 39(2), 175–191. The mother and infant home visiting program evaluation: Early Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, findings on the MIECHV program. OPRE Report 2015-11. Wash- F. (2005). Implementation research: A synthesis of the literature. ington, DC: OPRE, ACF, U.S. Department of Health and Human Tampa, FL: University of South Florida, Louis de la Parte Florida Services. Mental Health Institute, National Implementation Research Net- Miller, W. R., & Rollnick, S. (2012). Motivational interviewing: Help- work. (FMHI Publication No. 231). ing people change (3rd edn.). New York: Guilford Press. Frankel, R. M. (2001). Cracking the code: Theory and method in Monteiro, M. G. (2016). The four challenges of home visitation pro- clinical communication analysis. Health Communication, 13(1), grams: Alcohol and substance abuse, intrafamilial violence, and 101–110. mental disorders. Home visitation programs (pp.  9–11). New Gill, S., Greenberg, M. T., Moon, C., & Margraf, P. (2007). Home visi- York: Springer. tor competence, burnout, support, and client engagement. Journal Moyers, T. B., Manuel, J. K., & Ernst, D. (2015). Motivational Inter- of Human Behavior in the Social Environment, 15(1), 23–44. viewing Treatment Integrity Coding Manual 4.2.1. Unpublished Grossman, R., & Salas, E. (2011). The transfer of training: What really manual. matters. International Journal of Training and Development, Moyers, T. B., Martin, T., Manuel, J. K., Miller, W. R., & Ernst, D. 15(2), 103–120. (2010). Revised Global Scales: Motivational Interviewing Treat- Harden, B. J., Denmark, N., & Saul, D. (2010). Understanding the ment Integrity Coding Manual 3.1.1. Unpublished manual. needs of staff in Head Start programs: The characteristics, per - Sama-Miller, E., Akers, L., Mraz-Esposito, A., Zukiewicz, M., Avellar, ceptions, and experiences of home visitors. Children and Youth S., Paulsell, D., & Grosso, D., P (2017). Home visiting evidence of Services Review, 32(3), 371–379. effectiveness review: Executive summary. Administration for Chil- Health Resources and Services Administration. (2016). Maternal, dren and Families: Office of Planning, Research and Evaluation. infant and early childhood home visiting program—Innovation Schwalbe, C. S., Oh, H. Y., & Zweben, A. (2014). Sustaining motiva- awards [Funding opportunity announcement]. https ://mchb.hrsa. tional interviewing: A meta-analysis of training studies. Addic- gov/fundingopp or tunities /def ault.aspx?id=68f16 89a-6096-4db6- tion, 109(8), 1287–1294. ab76-bc41e b1dc4 1e. Retrieved 2 May 2018. Tandon, S. D., Mercer, C. D., Saylor, E. L., & Duggan, A. K. (2008). Health Resources and Services Administration. (2017). FY 2017 Home Paraprofessional home visitors’ perspectives on addressing poor visiting innovation awards. https://mc hb.hrsa.gov/home-visiting/ mental health, substance abuse, and domestic violence: A qualita- innov ation award s. Accessed 2 May 2018. tive study. Early Childhood Research Quarterly, 23(3), 419–428. Healthy Teen Network & RTI International. (2017). ACT for youth Tandon, S. D., Parillo, K. M., Jenkins, C., & Duggan, A. K. (2005). center of excellence: Facilitator observation form. http://www. Formative evaluation of home visitors’ role in addressing poor actfo ryout h.net/resou rces/capp/coe-facil itato r-obser vatio n.doc. mental health, domestic violence, and substance abuse among Accessed 17 Sept 2017. low-income pregnant and parenting women. Maternal and Child Home Visiting Applied Research Collaborative. (2017). Home visiting Health Journal, 9(3), 273–283. applied research collaborative. http://www.hvrn.org/index .html. Trivette, C. M., Dunst, C. J., Hamby, D. W., & O’Herin, C. E. (2009). Accessed 17 Sept 2017. Characteristics and consequences of adult learning methods and Home Visiting Research Network. (2013). Home visiting research strategies. Winterberry Research, 2(2), 1–33. agenda. http://www.hvresear ch.or g/wp-content/uploa ds/2018/01/ 1 3

Journal

Maternal and Child Health JournalSpringer Journals

Published: May 31, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off