Innovations in primary care behavioral health: a pilot study across the U.S. Air Force

Innovations in primary care behavioral health: a pilot study across the U.S. Air Force Abstract Integrated primary care services have grown in popularity in recent years and demonstrated significant benefits to the patient experience, patient health, and health care operations. However, broader systems-level factors for health care organizations, such as utilization, access, and cost, have been understudied. The current study reviews the results of quality improvement project conducted by the U.S. Air Force, which has practiced integrated primary care behavioral health for over 20 years. This study focuses on exploring how shifting the access point for behavioral from specialty mental health clinics to primary care, along with the use of technicians in patient care, can improve a range of health outcomes. Retrospective data analysis was conducted on an internal Air Force quality improvement project implemented at three military treatment facilities from October 2014 to September 2015. Positive preliminary support for these innovations was seen in the form of expanded patient populations, decreased time to first appointment, increased patient encounters, and decreased purchased community care compared with non-participating sites. Incorporation of behavioral health technicians further increased number of patient encounters while maintaining high levels of patient satisfaction across diverse clinical settings; in fact, patients preferred appointments with both technicians and behavioral health providers, compared with appointments with behavioral health providers only. These findings encourage further systematic review of systems-level factors in primary care behavioral health and adoption of the use of provider extenders in primary care behavioral health clinics. Implications Practice: Primary care behavioral health (PCBH) programs should consider utilizing technicians to extend the reach of behavioral health consultant. Policy: Health care systems should consider a stepped care approach to mental health care using PCBH and specialty clinics. Research: Larger scale, controlled studies should evaluate alternative delivery models and use of provider extenders based on pilot findings. As the need for mental health care in the USA continues to rise, there has become increased interest in innovative service delivery models [1–3]. In particular, the field of integrated care has rapidly grown, with a wide variety of particular care models being developed and evaluated [3]. These integrated care models have shown significant benefits for improving patient access [4], patient satisfaction [5], and patient outcomes [6–8]—all critical components embedded within the Institute for Healthcare Improvement’s (IHI) “Triple Aim”—improved population health, experience of care, and reduced health care costs [9]. One particular model of care in this literature is referred to as the primary care behavioral health (PCBH) model. Although the field is continuing to refine and build consensus, PCBH is defined by several key components [10]. PCBH emphasizes a team-based approach to the health care using a biopsychosocial model of health. The PCBH model incorporates a behavioral health consultant (BHC) into the primary care team to serve as a generalist and strives to intervene on the day they are referred (via use of the “warm handoff”) [11]. The goal is to incorporate biopsychosocial assessment and intervention into everyday care, accomplished via focused, time-limited visits (≤4 total, each lasting 15–40 min) and coordinated follow-up with other primary care team members [10, 12]. Although limited research to date directly compares various models of integrated care, PCBH has been associated with strong outcomes under the Triple Aim [3]. For example, PCBH has been found to reduce functional impairment [6] and successfully treat post-traumatic stress disorder [7], insomnia [8], and suicidal ideation [13]. The U.S. Air Force (USAF) has utilized PCBH for over 20 years, expanding from one to over 70 military treatment facilities (MTFs) across the world [14]. The USAF has invested considerable resources in training and developing BHCs—clinical psychologists and social workers—evaluating both patient experience and health outcomes in this model [5, 6]. The military health system (MHS) provides care to over 9.4 million beneficiaries, across all ages and demographic groups, including active duty and retired military members and their children and partners [15]. Thus, innovations within this diverse and complex health care system may have critical parallels to civilian health care. Air Force PCBH The USAF was one of the first larger health care organizations to practice PCBH and continues to emphasize many of its hallmark features [16]. BHCs are directly located in primary care clinics, providing the aforementioned focused, timely encounters through a consultative fashion to a primary care provider (PCP). This involves warm-handoffs (directly meeting with a patient following his or her PCP visit) and routine feedback to the PCP via daily huddles, one-on-one feedback, and use of a shared electronic health record. The BHC and PCP consult on a shared treatment plan, and the BHC is expected to use the biopsychosocial model to work with individuals experiencing a wide range of conditions (e.g., anxiety, depression, chronic pain, sleep, weight management) [17]. Patients who need a higher level of care are subsequently referred to specialty mental health treatment. The USAF PCBH model has been associated with high levels of patient satisfaction [5], patient– provider rapport [18], and improved patient functioning [6]. It has relied on a structured, competency-based phased training approach for new BHCs, involving a combination of experiential learning, didactic instruction, and on-the-job observation and mentoring [19]. However, challenges to this model remain, paralleling that of the civilian health care system. Underutilization of behavioral health care can limit the active engagement of BHCs in patient outcomes. The reasons for underutilization are multifaceted and may include practice behaviors of the BHC [20], patient stigma [21], and PCP misconceptions [22], among others. In an effort to address this challenge of underutilization, the Air Force Medical Service (AFMS) recently piloted several innovations in its long-standing PCBH program. METHOD Data in the present study are based on a 1-year quality improvement (QI) initiative approved by the Wilford Hall Institutional Review Board (IRB) for internal program evaluation and conducted from October 2014 to September 2015 [23]. This current study was a retrospective data analysis of this program evaluation project and was reviewed and approved by the Uniformed Services University IRB. Participating sites Solicitation for MTF participation in the pilot study was sought in June and July 2014. Six MTFs confirmed interest in participating in the study. Each MTF was ranked based on empanelment size, current manning of mental health personnel, mental health clinic access to care rates, and ability of clinics to support this project while maximizing diversity of sites to promote generalizability. The three MTFs selected as pilot sites were chosen by AFMS program managers to provide variety in geographical location and empanelment size to allow for generalizability across the MHS. Procedure Overall program management The QI initiative involved two main adjustments to current program standard operating procedures. One, providers and behavioral health technicians (BHTs) originally assigned to provide full-time specialty mental health services were reallocated to primary care to supplement the existing BHCs. Two, mental health clinic staff were trained to triage and book patients an initial PCBH appointment unless certain criteria were met, described in Fig. 1 (with dotted lines indicating process change). The current military insurance provider, TRICARE, allows all beneficiaries to self-refer to a community TRICARE network provider without a referral. This policy was not changed, although primary care and administrative staff were encouraged to book appointments directly into PCBH for patients self-referring for mental health–related care. In addition, no changes were made to the USAF PCBH program’s clinical and operational policies (e.g., regular consultations with a patient’s PCP; referring appropriate cases to specialty mental health; brief, solution-focused visits). Finally, while access procedures were put in place designed to encourage this stepped care approach, patient preference was respected and if a patient insisted on being seen in the specialty mental health clinic, this request was granted. It should be emphasized that this QI program did not involve any change to total personnel manning or funding—simply a reallocation of their clinical time as described above (“zero-sum”). Any personnel changes at these clinics were determined by higher headquarters consistent with the broader Air Force mission and without consideration of this pilot program. Fig 1 View largeDownload slide | Patient flow diagram for quality improvement project. BHC behavioral health consultant; PCBH primary care behavioral health. Fig 1 View largeDownload slide | Patient flow diagram for quality improvement project. BHC behavioral health consultant; PCBH primary care behavioral health. BHT utilization BHT roles were not specified by overall program management. Furthermore, as previously mentioned, technicians were reassigned from the specialty mental health clinic to primary care to assist with the anticipated increase in services. These BHTs were enlisted personnel who receive extensive training in psychological assessment and intervention as part of their technical school education (approximately 3–4 months). This includes rigorous evaluation and on-the-job training over the first several years of employment. Although there are no specific educational requirements beyond a high school diploma or equivalent, many of these technicians obtain associate’s, bachelor’s, or other professional degrees, ultimately becoming certified alcohol and drug abuse counselors and providing a full range of psychological care under supervision in a deployed setting [24]. The technicians employed in this study had already passed the certification required by the Air Force for involvement in patient care, practiced only under the scope of a licensed provider, and received pay commiserate with their enlisted rank. For this project, they received additional competency-based phased training designed to parallel USAF training for BHCs [18]. At one facility, the technician was trained to be directly involved in full-time patient care and operated in this capacity approximately 50% of the utilization year. Due to successful feedback and results, that technician trained technicians at other sites. This included providing a patient with informed consent, reviewing screening measures, and conducting a focused biopsychosocial, functional assessment (essentially, the first 15–20 min of a patient encounter in PCBH). Afterward, the technician would brief the BHC on the case, who would then meet with the patient for treatment planning for the remaining 5–10 min. Such operations were designed to parallel the traditional use of medical technicians in primary care while remaining consistent with the PCBH model of brief, solution-focused appointments [25]. An advantage of this model included flexibility (i.e., an atypical deviation in length of visit), which did not delay access for subsequent appointments and availability of PCBH personnel for warm-handoffs, maintaining the population health focus of PCBH [11, 12]. Although other behavioral health models of care have used provider extenders for telephonic care management and structured screening [26], technicians in this model participated in the face-to-face functional assessment of the patient’s concerns. Strategic messaging Before the pilot study began, the medical facilities launched a comprehensive strategic messaging campaign to alert the population and military leadership about the changes to accessing mental health care. Mental health and PCBH staff briefed commanders and their personnel about these changes, created base newspaper articles, and informed patients about PCBH services via online secure messaging. Examples are provided in the Supplementary Material. Measures Process metrics The following data were collected for PCBH and mental health clinics for both participating (n = 3) and nonparticipating (n = 69) MTFs: number of patient encounters per PCBH and mental health clinics, number of unique patients per PCBH and mental health clinics, access to care (days it took to be seen for initial appointment), and the amount of money reimbursed by TRICARE for outpatient mental health therapy spent in the local community. These process metrics were made available to the project manager overseeing this QI study for the fiscal year prior to implementation of the pilot program and the fiscal year during implementation. Data were provided on a quarterly basis as part of ongoing and standard AFMS program evaluation practices for behavioral health care operations. Patient satisfaction The Anonymous Patient Satisfaction Survey for Internal Behavioral Health Consultant Services has been used for program evaluation within the USAF for many years and found to show good internal consistency and concurrent validity [5, 22]. It has been revised and edited to include additional aspects of patient satisfaction and increase the range of response options due to noted concerns about ceiling effects [22]. Five items measured patients’ satisfaction with the BHC on various clinical skills (e.g., effort to listen, perceived knowledge of concern, likelihood of recommending services) and two additional questions measured the patient’s overall satisfaction and willingness to recommend services. Questions were measured on a seven-point Likert scale with responses ranging from 0 = extremely dissatisfied to 6 = extremely satisfied, 0 = definitely would not to 6 = definitely would, and 0 = extremely poor to 6 = extremely good. The current version was further revised for this study to allow patients to indicate whether or not a technician was involved in the encounter, and if so, to evaluate their satisfaction with the technician’s assessment of the presenting concern and effort to listen. Internal consistency in the current sample was α = .95. This survey was collected as part of standard clinical operations throughout the process improvement initiative. At each site, the Family Health Clinic front desk staff were instructed to give the surveys to each patient who had a PCBH appointment to reduce social desirability (i.e., avoiding the provision of feedback directly to their BHC). The completed surveys, excluding patient identifying information, were returned to the PCBH administrative staff after a sufficient number of surveys had been collected to prevent identification of specific respondents. These responses were then entered by administrative staff and sent to the process improvement project manager. These procedures are similar to previous naturalistic collection of patient satisfaction within the MHS [22]. This revised survey was only used during implementation of the technician training program from July to September 2015. Data analytic plan The use of retrospective data limited certain statistical analysis. Due to the significantly unequal cell sizes in comparing participating with nonparticipating sites, analysis of variance (ANOVA) for process metrics was not recommended [27]. In addition, the limited degrees of freedom available for time-series designs precluded formal testing of pre- and post-changes [28]. Thus, the limited descriptive data available are presented for these process metrics. Because each site differed in empanelment size and because total number of encounters for three participating sites versus 69 nonparticipating sites involved significantly different frames of reference, changes from pre- to post-implementation were converted to aggregated percentage change scores. In regard to patient satisfaction data, more robust statistical analyses could be performed. Unfortunately, due to the small number of participating sites, comparisons across sites could not be statistically tested. All data were analyzed for missingness and normality. Independent samples t-tests were conducted to examine differences in patient satisfaction when a technician was involved in care versus when one was not. A Bonferroni correction was applied given multiple planned analysis (p < .05/8 = .006). Hierarchical linear regression was employed to understand factors of patient satisfaction with both their BHC and technician that influenced their overall satisfaction and willingness to recommend services. RESULTS Basic information on staff and empanelment size for selected sites is listed in Table 1. As mentioned previously, additional staff was not recruited for this participating site; instead, sites managed their personnel to support additional resources in primary care consistent with implementation guidance described above. It should also be noted that overall, staffing across all three locations decreased over 20%, with only one individual site reporting an increase in staffing across both PCBH and specialty mental health clinics—and of less than 0.4 full-time equivalent providers (6% increase). As a fidelity check, all sites reported adherence to program implementation guidance as feasible, showing an increase in PCBH providers, with the exception of one site where there was a decrease in providers. However, this decrease was at a considerably smaller rate (5%) than seen in the specialty mental health clinic at the same location (47%), representing a prioritization of maintaining manning in PCBH. Table 1 | Pilot site characteristicsa Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) aPercentage change from pre- to post-intervention listed in FY2015 (Post) columns. bPer 2010 U.S. Census (population per square mile). View Large Table 1 | Pilot site characteristicsa Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) aPercentage change from pre- to post-intervention listed in FY2015 (Post) columns. bPer 2010 U.S. Census (population per square mile). View Large Descriptive results for key process metrics are shown in Table 2. As shown, results were positive but limited by the exploratory nature of available data analysis and percentage change was used due to unequal facility sizes. Participating sites saw an increase in both total patient encounters and total patients served during the implementation year. Furthermore, they had a much greater increase by percentage volume compared with non-participating sites. Pilot sites also saw a decrease in no-show rates for both PCBH and specialty mental health clinics, and this decrease in rates again exceeded those in nonparticipating sites. In regard to access to care, specialty mental health clinics at participating sites saw a slight decrease in access to care (longer wait times, equivalent to less than 10% of a day) while nonparticipating sites saw shorter wait times. However, the opposite effect was observed for PCBH clinics. Participating sites, while already increasing patient volume, experienced improved access to care and nonparticipating PCBH sites saw longer wait times. In addition, where nonparticipating sites experienced a 16% increase in community purchased care costs (the amount of money that TRICARE spends on behavioral health care outside the MTF) between pre- and post-implementation fiscal years, pilot sites saw a 4% decrease in purchased care costs during this same time period. This 20% difference in community health care costs accounts for a potential savings of nearly $7 million across the entire AFMS per year [23]. In regard to technician involvement, the initial site in which a technician was involved in direct patient care had a 61% increase in average daily patient visits when comparing the quarter proceeding technician training (4.73) with the quarter following technician training (7.6). Table 2 | Key process metrics for PCBH enhancement pilot Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 PCBH primary care behavioral health. aAverage percentage across pilot sites. Percentage change rounded to nearest hundredth reported as whole number. View Large Table 2 | Key process metrics for PCBH enhancement pilot Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 PCBH primary care behavioral health. aAverage percentage across pilot sites. Percentage change rounded to nearest hundredth reported as whole number. View Large In regard to patient satisfaction, during the 3-month collection period, there were approximately 2,078 encounters and 329 patients returned completed surveys. Although this response rate is low (16%), it is on par with other primary care settings [22, 29]. Patient satisfaction was analyzed comparing encounters with a technician compared with those without a technician aggregated across all three sites due to sample size within specific sites. Three cases were missing data on key study variables; after eliminating these cases, data were missing in only 3% of cases and data were considered missing completely at random (n = 326; Little’s MCAR test all ns for key study variables); thus, expectation maximization imputation methods were utilized to handle missing data consistent with prudent statistical techniques [30]. Of the 326 patient encounters, patients indicated technician involvement in 77 (24%) encounters. As shown in Table 3, patients who worked with a BHT rated their willingness to recommend services as greater than those patients who did not meet with a BHT. Table 3 | Patient satisfaction in technician-involved versus nontechnician-involved encounters Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 BHC behavioral health consultant. aDegrees of freedom adjusted due to heterogeneity of variances. *p < .05 using Bonferroni correction for planned analysis. View Large Table 3 | Patient satisfaction in technician-involved versus nontechnician-involved encounters Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 BHC behavioral health consultant. aDegrees of freedom adjusted due to heterogeneity of variances. *p < .05 using Bonferroni correction for planned analysis. View Large Hierarchical linear regression was used to understand the impact of specific aspects of patient satisfaction (e.g., amount of time spent with the BHC, provider or technician’s effort to listen) on overall patient satisfaction for those encounters with a technician. All aspects of patient satisfaction for the provider were entered on an initial step and two items assessing the technician’s role (their effort to listen and assessment of the problem) were entered on a second step. Results are shown in Table 4. Inclusion of the BHT items accounted for an additional 3% of the variance in patient satisfaction, F change (2, 70) = 7.76, p = .001; of note, their effort to listen was significantly associated with patient satisfaction, β = .36, p < .001. Table 4 | Hierarchical linear regression predicting patient satisfaction from provider and technician factors R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** BHC behavioral health consultant. *p < .01, **p < .001. View Large Table 4 | Hierarchical linear regression predicting patient satisfaction from provider and technician factors R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** BHC behavioral health consultant. *p < .01, **p < .001. View Large DISCUSSION The results of this preliminary pilot are promising and have implications for civilian health care organizations. Findings reinforce the benefits of PCBH, with a focus on understudied practice-level factors [31–33]. Specifically, findings demonstrate the ability of PCBH to reduce specialty mental health burden, improving population reach and access. This appears to have the added benefit of increasing patient compliance through improved appointment attendance. These benefits were seen not only in PCBH, but also in specialty mental health clinics at participating sites, suggesting advantages of a stepped care approach that improved communication, coordination, and reach of behavioral health services within health care operations. Improved coordination between primary care and specialty mental health may also have contributed to a decrease in no-show rates and promote patient engagement. Reduced no-shows and improved access, combined with increased population penetration, were also accompanied by large cost savings by reducing the use of outside community providers with higher per-encounter reimbursement costs. In this regard, the findings provide initial, but potentially compelling evidence for cost-saving models which may influence the insurance market landscape in health care to encourage further systematic research in this understudied area of systems-level factors. Sites incorporating PCBH into their patient flow for specialty mental health referrals saved money during a time when costs continued to increase at other facilities and while experiencing greater demand for services. Cost savings were even stronger in the final two fiscal quarters of the implementation project, suggesting that as clinic populations become more familiar with new procedures and increase their service uptake, costs may decline even further. The positive findings associated with technicians are also promising preliminary results. Although there may be some apprehension about a patient having to meet with yet another individual to describe their presenting concern, findings were not consistent with this being seen as a barrier. Given the parallel process to typical primary care operations, patients may not have been acutely aware of the difference. This was likely reinforced by training the BHC to provide a biopsychosocial conceptualization to the patient when entering the room, reaffirming that the technician had fully briefed the provider on the case, allowing the patient to validate that report, and then move immediately into intervention. In this way, the patient would not need to “tell their story” yet again while still feeling heard. In fact, patient satisfaction was improved by a technician’s effort to listen—and this survey was completed at the end of the entire encounter; thus, it is likely that the ability of the BHC to convey the technicians’ report reinforced a feeling of team-based care. Results further indicated that the quality of care with both a technician and BHC is perceived as similar or perhaps preferred to care with that same BHC only, while providing potential benefits that may indirectly lead to improved utilization. In particular, incorporating technicians in patient care appears to provide opportunities to increase provider efficiency and patient access—resulting in at least three extra appointments daily in the current study—while maintaining the experience of care. Several possibilities may explain these findings, although these links have not been empirically tested. The first is that by having this model whereby someone from the PCBH team is always available, even if an appointment has been scheduled, allows for both the warm-handoffs and the scheduled follow-ups to occur without decreasing either patient or PCP access, which can be a barrier to referral [22]. Second, this model may increase comfort with PCBH as “routine” for patients and PCPs, as we know that individuals bring certain schemas to various situations and that these influence their perceptions and expectations in these environments [34]. Finally, research suggests barriers to BHC service utilization are higher among nonprovider members of the primary care team [22]. Having a technician available may also facilitate more interactions between team members by promoting relationships between team members with similar levels of training and responsibility (e.g., BHT to medical technician, BHCs to PCPs). Future research should directly test these potential explanations. This preliminary study is not without limitations. The sample size and time points for which data were collected limit the ability to utilize more advanced statistical analysis. Furthermore, the low response rate for patient satisfaction may limit our ability to evaluate the representativeness of the sample, despite its consistency with response rates in previous studies in similar settings. It also introduces the potential for selection and nonresponse biases in the sample. It is important to note that in an effort to preserve “real-world” practices, few additional constraints were placed on pilot sites. Although this improves generalizability, it also significantly increases heterogeneity of the intervention sample. Thus, it is quite possible that a number of other variables (e.g., fidelity to model, staff training and expertise, population demographics) may have contributed to the positive findings. Without direct measures of these confounds, the ability to draw causal inferences is more limited. Despite these constraints, we feel these results offer important encouragement for further research in the field of PCBH at the systemic level. It is important to emphasize that these changes were incorporated in a system where there are clearly defined competency-based expectations for PCBH providers and clinics. These expectations are easily translatable to new facilities, as the Department of Defense has expanded many of these clinics only in the past few years [18]. As discussed above, these MTFs have considerable variability in size, staffing, and resources and are in locations with significant diversity in patient empanelment, environment, and presenting concerns. The MHS provides care not only for service members, but also for family members and retirees, whose diversity mirrors the civilian population. Thus, the USAF’s PCBH training and implementation program provides a “roadmap” for both civilian and military health care settings to allow for further testing and evaluation. In addition, this study utilized a pre- and post-test intervention design with control group comparison for analyzing results. Although statistical analysis is limited by the small intervention sample size and limited time points, these preliminary findings are reinforced by prudent methodological design. Although these facilities are heterogeneous and were not selected entirely at random, it is important to highlight that results were compared against the population of 69 other MTFs who also vary in fidelity to the model on measures such as staff training and expertise, population demographics, and a host of other potential variables of interest. Thus, these positive findings across this naturalistic variability speak to the potential generalizability of these findings. Finally, these benefits were achieved at “zero cost” to the existing PCBH and specialty mental health clinics. No additional personnel were recruited or hired across all three sites; manning decreased while it was reallocated. No additional compensation was provided to these sites to support these changes. Thus, there is little net cost to implementing these changes relative to the significant potential benefit demonstrated by these preliminary positive findings, which may seek to encourage further systematic research in this area. Future research should seek to address the limitation of the small intervention sample by further expanding this QI initiative based on these promising preliminary findings. Furthermore, efforts should be made to incorporate these changes into new PCBH programs and to parallel civilian-integrated health care settings. This will not only improve generalizability, but many of the same observed benefits in this setting are directly translatable outside of a military setting. Focusing on these practice-level decisions has the potential to shape the opportunities for success in the area of integrated behavioral health care. SUPPLEMENTARY MATERIAL Supplementary material is available at Translational Behavioral Medicine online. Acknowledgments: This project would not have been possible with the clinical team responsible for executing the quality improvement project at local facilities. Listed in alphabetical order: Capt (Dr.) Adam Dell, Capt (Dr.) Abby Fields, SrA Romia Jones, Mr. Michael Jordan, SSgt Ebony Santiago, SrA Samantha Webber, and SSgt Natalie Yarbrough. We also extend our sincere thanks to Dr. Jisuk Park who assisted with preliminary review of the study findings. There was no funding associated with this project. This article was authored by employees of the U.S. government. Any views expressed herein are those of the authors and do not necessarily represent the views of the U.S. government or the Department of Defense. Compliance with Ethical Standards Primary Data: This manuscript is not under consideration at another journal and has not been previously published. Portions of these data have been presented at various conferences, including the Society of Behavioral Medicine. As above, we are employees of the U.S. Department of Defense, but we have control of the primary data presented for review if desired. Conflict of Interest: The authors report no conflicts of interest. Ethical Approval: This study has been reviewed by the Wilford Hall Institutional Review Board and the Uniformed Services University Institutional Review Board and was conducted in compliance with ethical standards for human subjects’ research. Due to the nature of this study, informed consent and animal welfare statements do not apply. References 1. Burkey MD , Kaye DL , Frosch E . Training in integrated mental health-primary care models: A national survey of child psychiatry program directors . Acad Psychiatry . 2014 ; 38 ( 4 ): 485 – 488 . Google Scholar CrossRef Search ADS PubMed 2. Gomez D , Bridges AJ , Andrews AR et al. Delivering parent management training in an integrated primary care setting: description and preliminary outcome data . Cogn Behav Pract . 2014 ; 21 : 296 – 309 . Google Scholar CrossRef Search ADS 3. Vogel ME , Kanzler KE , Aikens JE , Goodie JL . Integration of behavioral health and primary care: Current knowledge and future directions . J Behav Med . 2017 ; 40 ( 1 ): 69 – 84 . Google Scholar CrossRef Search ADS PubMed 4. Ayalon L , Areán PA , Linkins K , Lynch M , Estes CL . Integration of mental health services into primary care overcomes ethnic disparities in access to mental health services between black and white elderly . Am J Geriatr Psychiatry . 2007 ; 15 ( 10 ): 906 – 912 . Google Scholar CrossRef Search ADS PubMed 5. Runyan CN , Fonseca VP , Meyer JG , Oordt MS , Talcott GW . A novel approach for mental health disease management: The Air Force Medical Service’s interdisciplinary model . Dis Manag . 2003 ; 6 ( 3 ): 179 – 188 . Google Scholar CrossRef Search ADS PubMed 6. Bryan CJ , Corso ML , Corso KA , Morrow CE , Kanzler KE , Ray-Sannerud B . Severity of mental health impairment and trajectories of improvement in an integrated primary care clinic . J Consult Clin Psychol . 2012 ; 80 ( 3 ): 396 – 403 . Google Scholar CrossRef Search ADS PubMed 7. Cigrang JA , Rauch SA , Mintz J et al. ; STRONG STAR Consortium . Treatment of active duty military with PTSD in primary care: A follow-up report . J Anxiety Disord . 2015 ; 36 : 110 – 114 . Google Scholar CrossRef Search ADS PubMed 8. Goodie JL , Isler WC , Hunter C , Peterson AL . Using behavioral health consultants to treat insomnia in primary care: a clinical case series . J Clin Psychol . 2009 ; 65 ( 3 ): 294 – 304 . Google Scholar CrossRef Search ADS PubMed 9. Berwick DM , Nolan TW , Whittington J . The triple aim: Care, health, and cost . Health Aff (Millwood) . 2008 ; 27 ( 3 ): 759 – 769 . Google Scholar CrossRef Search ADS PubMed 10. Mauksch L , Peek CJ , Fogarty CT . Seeking a wider lens for scientific rigor in emerging fields: The case of the primary care behavioral health model . Fam Syst Health . 2017 ; 35 ( 3 ): 251 – 256 . Google Scholar CrossRef Search ADS PubMed 11. Horevitz E , Organista KC , Arean PA . Depression treatment uptake in integrated primary care: How a “Warm Handoff” and other factors affect decision making by Latinos . Psychiatr Serv . 2015 ; 66 ( 8 ): 824 – 830 . Google Scholar CrossRef Search ADS PubMed 12. Reiter JT , Dobmeyer AC , Hunter CL . The primary care behavioral health (PCBH) model: An overview and operation definition . J Clin Psychol Med Settings . 2018 . doi: 10.1007/s10880-017-9531-x . [Epub ahead of print]. 13. Bryan CJ , Corso KA , Corso ML , Kanzler KE , Ray-Sannerud B , Morrow CE . Therapeutic alliance and change in suicidal ideation during treatment in integrated primary care settings . Arch Suicide Res . 2012 ; 16 ( 4 ): 316 – 323 . Google Scholar CrossRef Search ADS PubMed 14. Hunter CL , Goodie JL . Behavioral health in the Department of Defense Patient-Centered Medical Home: History, finance, policy, work force development, and evaluation . Transl Behav Med . 2012 ; 2 ( 3 ): 355 – 363 . Google Scholar CrossRef Search ADS PubMed 15. Defense Health Agency . Evaluation of the TRICARE Program: Fiscal Year 2017 Report to Congress . Falls Church, VA : Defense Health Agency ; 2017 . 16. Funderburk JS , Dobmeyer AC , Hunter CL , Walsh CO , Maisto SA . Provider practices in the primary care behavioral health (PCBH) model: An initial examination in the Veterans Health Administration and United States Air Force . Fam Syst Health . 2013 ; 31 ( 4 ): 341 – 353 . Google Scholar CrossRef Search ADS PubMed 17. Air Force Medical Operations Agency . Primary Care Behavioral Health Services: Behavioral Health Optimization Program Field Manual . Joint Base San Antonio – Lackland : Air Force Medical Service ; 2014 . 18. Corso KA , Bryan CJ , Corso ML et al. Therapeutic alliance and treatment outcome in the primary care behavioral health model . Fam Syst Health . 2012 ; 30 ( 2 ): 87 – 100 . Google Scholar CrossRef Search ADS PubMed 19. Dobmeyer AC , Hunter CL , Corso ML et al. Primary care behavioral health provider training: Systematic development and implementation in a large medical system . J Clin Psychol Med Settings . 2016 ; 23 ( 3 ): 207 – 224 . Google Scholar CrossRef Search ADS PubMed 20. Robinson PJ , Strosahl KD . Behavioral health consultation and primary care: Lessons learned . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 58 – 71 . Google Scholar CrossRef Search ADS PubMed 21. Shim R , Rust G . Primary care, behavioral health, and public health: Partners in reducing mental health stigma . Am J Public Health . 2013 ; 103 ( 5 ): 774 – 776 . Google Scholar CrossRef Search ADS PubMed 22. Landoll RR , Nielsen MK , Waggoner KK . US air force behavioral health optimization program: Team members’ satisfaction and barriers to care . Fam Pract . 2017 ; 34 ( 1 ): 71 – 76 . Google Scholar CrossRef Search ADS PubMed 23. Nielsen MK. Revolutionizing Mental Health Care Delivery in the United States Air Force by Shifting the Access Point to Primary Care . Maxwell AFB, AL : Air Command and Staff College ; 2016 . 24. Department of the Air Force . AFSC 4C0X1 Mental Health Service Specialty: Career Field Education and Training Plan . Washington, DC : Headquarters Air Force ; 2015 . 25. Kellermann AL , Saultz JW , Mehrotra A , Jones SS , Dalal S . Primary care technicians: A solution to the primary care workforce gap . Health Aff (Millwood) . 2013 ; 32 ( 11 ): 1893 – 1898 . Google Scholar CrossRef Search ADS PubMed 26. Oslin DW , Ross J , Sayers S , Murphy J , Kane V , Katz IR . Screening, assessment, and management of depression in VA primary care clinics. The Behavioral Health Laboratory . J Gen Intern Med . 2006 ; 21 ( 1 ): 46 – 50 . Google Scholar CrossRef Search ADS PubMed 27. Wang Y , Rodriguez de Gil PR , Chen Y et al. Comparing the performance of approaches for testing the homogeneity of variance assumption in one-factor ANOVA models . Educ Psychol Meas . 2017 ; 77 : 305 – 329 . Google Scholar CrossRef Search ADS 28. Lopez Bernal J , Cummins S , Gasparrini A . Interrupted time series regression for the evaluation of public health interventions: A tutorial . Int J Epidemiol . 2016 ; 46 : 348 – 355 . 29. Panattoni L , Stone A , Chung S , Tai-Seale M . Patients report better satisfaction with part-time primary care physicians, despite less continuity of care and access . J Gen Intern Med . 2015 ; 30 ( 3 ): 327 – 333 . Google Scholar CrossRef Search ADS PubMed 30. Kline R. Principles and Practice of Structural Equation Modeling . New York, NY : Guilford ; 2005 . 31. Cummings NA , O’Donohue WT , Cummings JL . The financial dimension of integrated behavioral/primary care . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 31 – 39 . Google Scholar CrossRef Search ADS PubMed 32. Miller BF , Mendenhall TJ , Malik AD . Integrated primary care: An inclusive three-world view through process metrics and empirical discrimination . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 21 – 30 . Google Scholar CrossRef Search ADS PubMed 33. Muse AR , Lamson AL , Didericksen KW , Hodgson JL . A systematic review of evaluation research in integrated behavioral health care: Operational and financial characteristics . Fam Syst Health . 2017 ; 35 ( 2 ): 136 – 154 . Google Scholar CrossRef Search ADS PubMed 34. Zacks JM , Speer NK , Swallow KM , Braver TS , Reynolds JR . Event perception: A mind-brain perspective . Psychol Bull . 2007 ; 133 ( 2 ): 273 – 293 . Google Scholar CrossRef Search ADS PubMed Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Translational Behavioral Medicine Oxford University Press

Innovations in primary care behavioral health: a pilot study across the U.S. Air Force

Loading next page...
 
/lp/ou_press/innovations-in-primary-care-behavioral-health-a-pilot-study-across-the-GNc9dkf86n
Copyright
Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2018.
ISSN
1869-6716
eISSN
1613-9860
D.O.I.
10.1093/tbm/iby046
Publisher site
See Article on Publisher Site

Abstract

Abstract Integrated primary care services have grown in popularity in recent years and demonstrated significant benefits to the patient experience, patient health, and health care operations. However, broader systems-level factors for health care organizations, such as utilization, access, and cost, have been understudied. The current study reviews the results of quality improvement project conducted by the U.S. Air Force, which has practiced integrated primary care behavioral health for over 20 years. This study focuses on exploring how shifting the access point for behavioral from specialty mental health clinics to primary care, along with the use of technicians in patient care, can improve a range of health outcomes. Retrospective data analysis was conducted on an internal Air Force quality improvement project implemented at three military treatment facilities from October 2014 to September 2015. Positive preliminary support for these innovations was seen in the form of expanded patient populations, decreased time to first appointment, increased patient encounters, and decreased purchased community care compared with non-participating sites. Incorporation of behavioral health technicians further increased number of patient encounters while maintaining high levels of patient satisfaction across diverse clinical settings; in fact, patients preferred appointments with both technicians and behavioral health providers, compared with appointments with behavioral health providers only. These findings encourage further systematic review of systems-level factors in primary care behavioral health and adoption of the use of provider extenders in primary care behavioral health clinics. Implications Practice: Primary care behavioral health (PCBH) programs should consider utilizing technicians to extend the reach of behavioral health consultant. Policy: Health care systems should consider a stepped care approach to mental health care using PCBH and specialty clinics. Research: Larger scale, controlled studies should evaluate alternative delivery models and use of provider extenders based on pilot findings. As the need for mental health care in the USA continues to rise, there has become increased interest in innovative service delivery models [1–3]. In particular, the field of integrated care has rapidly grown, with a wide variety of particular care models being developed and evaluated [3]. These integrated care models have shown significant benefits for improving patient access [4], patient satisfaction [5], and patient outcomes [6–8]—all critical components embedded within the Institute for Healthcare Improvement’s (IHI) “Triple Aim”—improved population health, experience of care, and reduced health care costs [9]. One particular model of care in this literature is referred to as the primary care behavioral health (PCBH) model. Although the field is continuing to refine and build consensus, PCBH is defined by several key components [10]. PCBH emphasizes a team-based approach to the health care using a biopsychosocial model of health. The PCBH model incorporates a behavioral health consultant (BHC) into the primary care team to serve as a generalist and strives to intervene on the day they are referred (via use of the “warm handoff”) [11]. The goal is to incorporate biopsychosocial assessment and intervention into everyday care, accomplished via focused, time-limited visits (≤4 total, each lasting 15–40 min) and coordinated follow-up with other primary care team members [10, 12]. Although limited research to date directly compares various models of integrated care, PCBH has been associated with strong outcomes under the Triple Aim [3]. For example, PCBH has been found to reduce functional impairment [6] and successfully treat post-traumatic stress disorder [7], insomnia [8], and suicidal ideation [13]. The U.S. Air Force (USAF) has utilized PCBH for over 20 years, expanding from one to over 70 military treatment facilities (MTFs) across the world [14]. The USAF has invested considerable resources in training and developing BHCs—clinical psychologists and social workers—evaluating both patient experience and health outcomes in this model [5, 6]. The military health system (MHS) provides care to over 9.4 million beneficiaries, across all ages and demographic groups, including active duty and retired military members and their children and partners [15]. Thus, innovations within this diverse and complex health care system may have critical parallels to civilian health care. Air Force PCBH The USAF was one of the first larger health care organizations to practice PCBH and continues to emphasize many of its hallmark features [16]. BHCs are directly located in primary care clinics, providing the aforementioned focused, timely encounters through a consultative fashion to a primary care provider (PCP). This involves warm-handoffs (directly meeting with a patient following his or her PCP visit) and routine feedback to the PCP via daily huddles, one-on-one feedback, and use of a shared electronic health record. The BHC and PCP consult on a shared treatment plan, and the BHC is expected to use the biopsychosocial model to work with individuals experiencing a wide range of conditions (e.g., anxiety, depression, chronic pain, sleep, weight management) [17]. Patients who need a higher level of care are subsequently referred to specialty mental health treatment. The USAF PCBH model has been associated with high levels of patient satisfaction [5], patient– provider rapport [18], and improved patient functioning [6]. It has relied on a structured, competency-based phased training approach for new BHCs, involving a combination of experiential learning, didactic instruction, and on-the-job observation and mentoring [19]. However, challenges to this model remain, paralleling that of the civilian health care system. Underutilization of behavioral health care can limit the active engagement of BHCs in patient outcomes. The reasons for underutilization are multifaceted and may include practice behaviors of the BHC [20], patient stigma [21], and PCP misconceptions [22], among others. In an effort to address this challenge of underutilization, the Air Force Medical Service (AFMS) recently piloted several innovations in its long-standing PCBH program. METHOD Data in the present study are based on a 1-year quality improvement (QI) initiative approved by the Wilford Hall Institutional Review Board (IRB) for internal program evaluation and conducted from October 2014 to September 2015 [23]. This current study was a retrospective data analysis of this program evaluation project and was reviewed and approved by the Uniformed Services University IRB. Participating sites Solicitation for MTF participation in the pilot study was sought in June and July 2014. Six MTFs confirmed interest in participating in the study. Each MTF was ranked based on empanelment size, current manning of mental health personnel, mental health clinic access to care rates, and ability of clinics to support this project while maximizing diversity of sites to promote generalizability. The three MTFs selected as pilot sites were chosen by AFMS program managers to provide variety in geographical location and empanelment size to allow for generalizability across the MHS. Procedure Overall program management The QI initiative involved two main adjustments to current program standard operating procedures. One, providers and behavioral health technicians (BHTs) originally assigned to provide full-time specialty mental health services were reallocated to primary care to supplement the existing BHCs. Two, mental health clinic staff were trained to triage and book patients an initial PCBH appointment unless certain criteria were met, described in Fig. 1 (with dotted lines indicating process change). The current military insurance provider, TRICARE, allows all beneficiaries to self-refer to a community TRICARE network provider without a referral. This policy was not changed, although primary care and administrative staff were encouraged to book appointments directly into PCBH for patients self-referring for mental health–related care. In addition, no changes were made to the USAF PCBH program’s clinical and operational policies (e.g., regular consultations with a patient’s PCP; referring appropriate cases to specialty mental health; brief, solution-focused visits). Finally, while access procedures were put in place designed to encourage this stepped care approach, patient preference was respected and if a patient insisted on being seen in the specialty mental health clinic, this request was granted. It should be emphasized that this QI program did not involve any change to total personnel manning or funding—simply a reallocation of their clinical time as described above (“zero-sum”). Any personnel changes at these clinics were determined by higher headquarters consistent with the broader Air Force mission and without consideration of this pilot program. Fig 1 View largeDownload slide | Patient flow diagram for quality improvement project. BHC behavioral health consultant; PCBH primary care behavioral health. Fig 1 View largeDownload slide | Patient flow diagram for quality improvement project. BHC behavioral health consultant; PCBH primary care behavioral health. BHT utilization BHT roles were not specified by overall program management. Furthermore, as previously mentioned, technicians were reassigned from the specialty mental health clinic to primary care to assist with the anticipated increase in services. These BHTs were enlisted personnel who receive extensive training in psychological assessment and intervention as part of their technical school education (approximately 3–4 months). This includes rigorous evaluation and on-the-job training over the first several years of employment. Although there are no specific educational requirements beyond a high school diploma or equivalent, many of these technicians obtain associate’s, bachelor’s, or other professional degrees, ultimately becoming certified alcohol and drug abuse counselors and providing a full range of psychological care under supervision in a deployed setting [24]. The technicians employed in this study had already passed the certification required by the Air Force for involvement in patient care, practiced only under the scope of a licensed provider, and received pay commiserate with their enlisted rank. For this project, they received additional competency-based phased training designed to parallel USAF training for BHCs [18]. At one facility, the technician was trained to be directly involved in full-time patient care and operated in this capacity approximately 50% of the utilization year. Due to successful feedback and results, that technician trained technicians at other sites. This included providing a patient with informed consent, reviewing screening measures, and conducting a focused biopsychosocial, functional assessment (essentially, the first 15–20 min of a patient encounter in PCBH). Afterward, the technician would brief the BHC on the case, who would then meet with the patient for treatment planning for the remaining 5–10 min. Such operations were designed to parallel the traditional use of medical technicians in primary care while remaining consistent with the PCBH model of brief, solution-focused appointments [25]. An advantage of this model included flexibility (i.e., an atypical deviation in length of visit), which did not delay access for subsequent appointments and availability of PCBH personnel for warm-handoffs, maintaining the population health focus of PCBH [11, 12]. Although other behavioral health models of care have used provider extenders for telephonic care management and structured screening [26], technicians in this model participated in the face-to-face functional assessment of the patient’s concerns. Strategic messaging Before the pilot study began, the medical facilities launched a comprehensive strategic messaging campaign to alert the population and military leadership about the changes to accessing mental health care. Mental health and PCBH staff briefed commanders and their personnel about these changes, created base newspaper articles, and informed patients about PCBH services via online secure messaging. Examples are provided in the Supplementary Material. Measures Process metrics The following data were collected for PCBH and mental health clinics for both participating (n = 3) and nonparticipating (n = 69) MTFs: number of patient encounters per PCBH and mental health clinics, number of unique patients per PCBH and mental health clinics, access to care (days it took to be seen for initial appointment), and the amount of money reimbursed by TRICARE for outpatient mental health therapy spent in the local community. These process metrics were made available to the project manager overseeing this QI study for the fiscal year prior to implementation of the pilot program and the fiscal year during implementation. Data were provided on a quarterly basis as part of ongoing and standard AFMS program evaluation practices for behavioral health care operations. Patient satisfaction The Anonymous Patient Satisfaction Survey for Internal Behavioral Health Consultant Services has been used for program evaluation within the USAF for many years and found to show good internal consistency and concurrent validity [5, 22]. It has been revised and edited to include additional aspects of patient satisfaction and increase the range of response options due to noted concerns about ceiling effects [22]. Five items measured patients’ satisfaction with the BHC on various clinical skills (e.g., effort to listen, perceived knowledge of concern, likelihood of recommending services) and two additional questions measured the patient’s overall satisfaction and willingness to recommend services. Questions were measured on a seven-point Likert scale with responses ranging from 0 = extremely dissatisfied to 6 = extremely satisfied, 0 = definitely would not to 6 = definitely would, and 0 = extremely poor to 6 = extremely good. The current version was further revised for this study to allow patients to indicate whether or not a technician was involved in the encounter, and if so, to evaluate their satisfaction with the technician’s assessment of the presenting concern and effort to listen. Internal consistency in the current sample was α = .95. This survey was collected as part of standard clinical operations throughout the process improvement initiative. At each site, the Family Health Clinic front desk staff were instructed to give the surveys to each patient who had a PCBH appointment to reduce social desirability (i.e., avoiding the provision of feedback directly to their BHC). The completed surveys, excluding patient identifying information, were returned to the PCBH administrative staff after a sufficient number of surveys had been collected to prevent identification of specific respondents. These responses were then entered by administrative staff and sent to the process improvement project manager. These procedures are similar to previous naturalistic collection of patient satisfaction within the MHS [22]. This revised survey was only used during implementation of the technician training program from July to September 2015. Data analytic plan The use of retrospective data limited certain statistical analysis. Due to the significantly unequal cell sizes in comparing participating with nonparticipating sites, analysis of variance (ANOVA) for process metrics was not recommended [27]. In addition, the limited degrees of freedom available for time-series designs precluded formal testing of pre- and post-changes [28]. Thus, the limited descriptive data available are presented for these process metrics. Because each site differed in empanelment size and because total number of encounters for three participating sites versus 69 nonparticipating sites involved significantly different frames of reference, changes from pre- to post-implementation were converted to aggregated percentage change scores. In regard to patient satisfaction data, more robust statistical analyses could be performed. Unfortunately, due to the small number of participating sites, comparisons across sites could not be statistically tested. All data were analyzed for missingness and normality. Independent samples t-tests were conducted to examine differences in patient satisfaction when a technician was involved in care versus when one was not. A Bonferroni correction was applied given multiple planned analysis (p < .05/8 = .006). Hierarchical linear regression was employed to understand factors of patient satisfaction with both their BHC and technician that influenced their overall satisfaction and willingness to recommend services. RESULTS Basic information on staff and empanelment size for selected sites is listed in Table 1. As mentioned previously, additional staff was not recruited for this participating site; instead, sites managed their personnel to support additional resources in primary care consistent with implementation guidance described above. It should also be noted that overall, staffing across all three locations decreased over 20%, with only one individual site reporting an increase in staffing across both PCBH and specialty mental health clinics—and of less than 0.4 full-time equivalent providers (6% increase). As a fidelity check, all sites reported adherence to program implementation guidance as feasible, showing an increase in PCBH providers, with the exception of one site where there was a decrease in providers. However, this decrease was at a considerably smaller rate (5%) than seen in the specialty mental health clinic at the same location (47%), representing a prioritization of maintaining manning in PCBH. Table 1 | Pilot site characteristicsa Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) aPercentage change from pre- to post-intervention listed in FY2015 (Post) columns. bPer 2010 U.S. Census (population per square mile). View Large Table 1 | Pilot site characteristicsa Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) Empanelment size Site population densityb Mental health clinic providers (Full-time equivalents) PCBH providers (Full-time equivalents) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) FY2014 (Pre) FY2015 (Post) 13,579 13,347 (−2%) 1,113 5.12 5.12 (0%) 1.0 1.38 (+38%) 25,624 24,861 (−3%) 1,270 8.5 4.5 (−47%) 2.45 2.34 (−5%) 54,624 46,997 (−13%) 2,944.6 9.38 6.55 (−30%) 0.88 2.5 (+284%) aPercentage change from pre- to post-intervention listed in FY2015 (Post) columns. bPer 2010 U.S. Census (population per square mile). View Large Descriptive results for key process metrics are shown in Table 2. As shown, results were positive but limited by the exploratory nature of available data analysis and percentage change was used due to unequal facility sizes. Participating sites saw an increase in both total patient encounters and total patients served during the implementation year. Furthermore, they had a much greater increase by percentage volume compared with non-participating sites. Pilot sites also saw a decrease in no-show rates for both PCBH and specialty mental health clinics, and this decrease in rates again exceeded those in nonparticipating sites. In regard to access to care, specialty mental health clinics at participating sites saw a slight decrease in access to care (longer wait times, equivalent to less than 10% of a day) while nonparticipating sites saw shorter wait times. However, the opposite effect was observed for PCBH clinics. Participating sites, while already increasing patient volume, experienced improved access to care and nonparticipating PCBH sites saw longer wait times. In addition, where nonparticipating sites experienced a 16% increase in community purchased care costs (the amount of money that TRICARE spends on behavioral health care outside the MTF) between pre- and post-implementation fiscal years, pilot sites saw a 4% decrease in purchased care costs during this same time period. This 20% difference in community health care costs accounts for a potential savings of nearly $7 million across the entire AFMS per year [23]. In regard to technician involvement, the initial site in which a technician was involved in direct patient care had a 61% increase in average daily patient visits when comparing the quarter proceeding technician training (4.73) with the quarter following technician training (7.6). Table 2 | Key process metrics for PCBH enhancement pilot Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 PCBH primary care behavioral health. aAverage percentage across pilot sites. Percentage change rounded to nearest hundredth reported as whole number. View Large Table 2 | Key process metrics for PCBH enhancement pilot Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 Pilot sites Nonparticipating sites Process metric Pre-pilot Post-pilot %Δa Pre-pilot Post-pilot %Δa Total encounters  PCBH 4,219 8,511 +99 48,856 59,726 +22  Mental health 23,214 24,952 +12 348,053 357,275 +3 Total patients  PCBH 2,683 6,464 +140 30,116 34,209 +14  Mental health 6,132 12,865 +138 53,248 52,844 −1 No-show rate (%)a  PCBH 14.3 10.6 −4 8.9 8.8 −1  Mental health 12.1 8.3 −30 7.2 7.0 −3 Access to carea  PCBH 6.88 5.22 −12 4.09 4.24 +4  Mental health 6.06 6.13 +1 6.04 5.71 −6 Purchased care ($M) 3.30 3.17 −4 36.1 41.7 +16 PCBH primary care behavioral health. aAverage percentage across pilot sites. Percentage change rounded to nearest hundredth reported as whole number. View Large In regard to patient satisfaction, during the 3-month collection period, there were approximately 2,078 encounters and 329 patients returned completed surveys. Although this response rate is low (16%), it is on par with other primary care settings [22, 29]. Patient satisfaction was analyzed comparing encounters with a technician compared with those without a technician aggregated across all three sites due to sample size within specific sites. Three cases were missing data on key study variables; after eliminating these cases, data were missing in only 3% of cases and data were considered missing completely at random (n = 326; Little’s MCAR test all ns for key study variables); thus, expectation maximization imputation methods were utilized to handle missing data consistent with prudent statistical techniques [30]. Of the 326 patient encounters, patients indicated technician involvement in 77 (24%) encounters. As shown in Table 3, patients who worked with a BHT rated their willingness to recommend services as greater than those patients who did not meet with a BHT. Table 3 | Patient satisfaction in technician-involved versus nontechnician-involved encounters Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 BHC behavioral health consultant. aDegrees of freedom adjusted due to heterogeneity of variances. *p < .05 using Bonferroni correction for planned analysis. View Large Table 3 | Patient satisfaction in technician-involved versus nontechnician-involved encounters Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 Technician mean (SD) Nontechnician mean (SD) t 1. Amount of time for appt 5.23 (0.93) 5.29 (0.94) 0.49 2. BHC’s effort to listen 5.47 (0.85) 5.55 (0.76) 0.77 3. BHC’s knowledge of presenting concern 5.29 (0.96) 5.46 (0.81) 1.58 4. Quality of care/intervention 5.16 (0.99) 5.39 (0.82) 2.09 5. Overall treatment plana 5.00 (1.14) 5.32 (0.86) 2.26 6. Overall patient satisfaction 5.25 (1.00) 5.36 (0.88) 0.90 7. Likelihood to recommend IBHC servicesa 5.32 (0.91) 4.94 (1.29) 2.94* 8. Patient health rating 3.54 (1.23) 3.61 (1.33) 0.36 BHC behavioral health consultant. aDegrees of freedom adjusted due to heterogeneity of variances. *p < .05 using Bonferroni correction for planned analysis. View Large Hierarchical linear regression was used to understand the impact of specific aspects of patient satisfaction (e.g., amount of time spent with the BHC, provider or technician’s effort to listen) on overall patient satisfaction for those encounters with a technician. All aspects of patient satisfaction for the provider were entered on an initial step and two items assessing the technician’s role (their effort to listen and assessment of the problem) were entered on a second step. Results are shown in Table 4. Inclusion of the BHT items accounted for an additional 3% of the variance in patient satisfaction, F change (2, 70) = 7.76, p = .001; of note, their effort to listen was significantly associated with patient satisfaction, β = .36, p < .001. Table 4 | Hierarchical linear regression predicting patient satisfaction from provider and technician factors R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** BHC behavioral health consultant. *p < .01, **p < .001. View Large Table 4 | Hierarchical linear regression predicting patient satisfaction from provider and technician factors R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** R2 ΔR2 β Final β Step 1 (Provider factors) .82 .82**  Amount of time for appt .43* .35*  BHC effort to listen .13 .02  Knowledge of presenting concern −.03 −.05  Quality of care/intervention .12 .17  Overall treatment plan .34* .31* Step 2 (Technician factors) .85 .03*  Technician effort to listen .36** .36**  Assessment of presenting concern −.15 −.15 Final model F(7, 70) = 56.92** BHC behavioral health consultant. *p < .01, **p < .001. View Large DISCUSSION The results of this preliminary pilot are promising and have implications for civilian health care organizations. Findings reinforce the benefits of PCBH, with a focus on understudied practice-level factors [31–33]. Specifically, findings demonstrate the ability of PCBH to reduce specialty mental health burden, improving population reach and access. This appears to have the added benefit of increasing patient compliance through improved appointment attendance. These benefits were seen not only in PCBH, but also in specialty mental health clinics at participating sites, suggesting advantages of a stepped care approach that improved communication, coordination, and reach of behavioral health services within health care operations. Improved coordination between primary care and specialty mental health may also have contributed to a decrease in no-show rates and promote patient engagement. Reduced no-shows and improved access, combined with increased population penetration, were also accompanied by large cost savings by reducing the use of outside community providers with higher per-encounter reimbursement costs. In this regard, the findings provide initial, but potentially compelling evidence for cost-saving models which may influence the insurance market landscape in health care to encourage further systematic research in this understudied area of systems-level factors. Sites incorporating PCBH into their patient flow for specialty mental health referrals saved money during a time when costs continued to increase at other facilities and while experiencing greater demand for services. Cost savings were even stronger in the final two fiscal quarters of the implementation project, suggesting that as clinic populations become more familiar with new procedures and increase their service uptake, costs may decline even further. The positive findings associated with technicians are also promising preliminary results. Although there may be some apprehension about a patient having to meet with yet another individual to describe their presenting concern, findings were not consistent with this being seen as a barrier. Given the parallel process to typical primary care operations, patients may not have been acutely aware of the difference. This was likely reinforced by training the BHC to provide a biopsychosocial conceptualization to the patient when entering the room, reaffirming that the technician had fully briefed the provider on the case, allowing the patient to validate that report, and then move immediately into intervention. In this way, the patient would not need to “tell their story” yet again while still feeling heard. In fact, patient satisfaction was improved by a technician’s effort to listen—and this survey was completed at the end of the entire encounter; thus, it is likely that the ability of the BHC to convey the technicians’ report reinforced a feeling of team-based care. Results further indicated that the quality of care with both a technician and BHC is perceived as similar or perhaps preferred to care with that same BHC only, while providing potential benefits that may indirectly lead to improved utilization. In particular, incorporating technicians in patient care appears to provide opportunities to increase provider efficiency and patient access—resulting in at least three extra appointments daily in the current study—while maintaining the experience of care. Several possibilities may explain these findings, although these links have not been empirically tested. The first is that by having this model whereby someone from the PCBH team is always available, even if an appointment has been scheduled, allows for both the warm-handoffs and the scheduled follow-ups to occur without decreasing either patient or PCP access, which can be a barrier to referral [22]. Second, this model may increase comfort with PCBH as “routine” for patients and PCPs, as we know that individuals bring certain schemas to various situations and that these influence their perceptions and expectations in these environments [34]. Finally, research suggests barriers to BHC service utilization are higher among nonprovider members of the primary care team [22]. Having a technician available may also facilitate more interactions between team members by promoting relationships between team members with similar levels of training and responsibility (e.g., BHT to medical technician, BHCs to PCPs). Future research should directly test these potential explanations. This preliminary study is not without limitations. The sample size and time points for which data were collected limit the ability to utilize more advanced statistical analysis. Furthermore, the low response rate for patient satisfaction may limit our ability to evaluate the representativeness of the sample, despite its consistency with response rates in previous studies in similar settings. It also introduces the potential for selection and nonresponse biases in the sample. It is important to note that in an effort to preserve “real-world” practices, few additional constraints were placed on pilot sites. Although this improves generalizability, it also significantly increases heterogeneity of the intervention sample. Thus, it is quite possible that a number of other variables (e.g., fidelity to model, staff training and expertise, population demographics) may have contributed to the positive findings. Without direct measures of these confounds, the ability to draw causal inferences is more limited. Despite these constraints, we feel these results offer important encouragement for further research in the field of PCBH at the systemic level. It is important to emphasize that these changes were incorporated in a system where there are clearly defined competency-based expectations for PCBH providers and clinics. These expectations are easily translatable to new facilities, as the Department of Defense has expanded many of these clinics only in the past few years [18]. As discussed above, these MTFs have considerable variability in size, staffing, and resources and are in locations with significant diversity in patient empanelment, environment, and presenting concerns. The MHS provides care not only for service members, but also for family members and retirees, whose diversity mirrors the civilian population. Thus, the USAF’s PCBH training and implementation program provides a “roadmap” for both civilian and military health care settings to allow for further testing and evaluation. In addition, this study utilized a pre- and post-test intervention design with control group comparison for analyzing results. Although statistical analysis is limited by the small intervention sample size and limited time points, these preliminary findings are reinforced by prudent methodological design. Although these facilities are heterogeneous and were not selected entirely at random, it is important to highlight that results were compared against the population of 69 other MTFs who also vary in fidelity to the model on measures such as staff training and expertise, population demographics, and a host of other potential variables of interest. Thus, these positive findings across this naturalistic variability speak to the potential generalizability of these findings. Finally, these benefits were achieved at “zero cost” to the existing PCBH and specialty mental health clinics. No additional personnel were recruited or hired across all three sites; manning decreased while it was reallocated. No additional compensation was provided to these sites to support these changes. Thus, there is little net cost to implementing these changes relative to the significant potential benefit demonstrated by these preliminary positive findings, which may seek to encourage further systematic research in this area. Future research should seek to address the limitation of the small intervention sample by further expanding this QI initiative based on these promising preliminary findings. Furthermore, efforts should be made to incorporate these changes into new PCBH programs and to parallel civilian-integrated health care settings. This will not only improve generalizability, but many of the same observed benefits in this setting are directly translatable outside of a military setting. Focusing on these practice-level decisions has the potential to shape the opportunities for success in the area of integrated behavioral health care. SUPPLEMENTARY MATERIAL Supplementary material is available at Translational Behavioral Medicine online. Acknowledgments: This project would not have been possible with the clinical team responsible for executing the quality improvement project at local facilities. Listed in alphabetical order: Capt (Dr.) Adam Dell, Capt (Dr.) Abby Fields, SrA Romia Jones, Mr. Michael Jordan, SSgt Ebony Santiago, SrA Samantha Webber, and SSgt Natalie Yarbrough. We also extend our sincere thanks to Dr. Jisuk Park who assisted with preliminary review of the study findings. There was no funding associated with this project. This article was authored by employees of the U.S. government. Any views expressed herein are those of the authors and do not necessarily represent the views of the U.S. government or the Department of Defense. Compliance with Ethical Standards Primary Data: This manuscript is not under consideration at another journal and has not been previously published. Portions of these data have been presented at various conferences, including the Society of Behavioral Medicine. As above, we are employees of the U.S. Department of Defense, but we have control of the primary data presented for review if desired. Conflict of Interest: The authors report no conflicts of interest. Ethical Approval: This study has been reviewed by the Wilford Hall Institutional Review Board and the Uniformed Services University Institutional Review Board and was conducted in compliance with ethical standards for human subjects’ research. Due to the nature of this study, informed consent and animal welfare statements do not apply. References 1. Burkey MD , Kaye DL , Frosch E . Training in integrated mental health-primary care models: A national survey of child psychiatry program directors . Acad Psychiatry . 2014 ; 38 ( 4 ): 485 – 488 . Google Scholar CrossRef Search ADS PubMed 2. Gomez D , Bridges AJ , Andrews AR et al. Delivering parent management training in an integrated primary care setting: description and preliminary outcome data . Cogn Behav Pract . 2014 ; 21 : 296 – 309 . Google Scholar CrossRef Search ADS 3. Vogel ME , Kanzler KE , Aikens JE , Goodie JL . Integration of behavioral health and primary care: Current knowledge and future directions . J Behav Med . 2017 ; 40 ( 1 ): 69 – 84 . Google Scholar CrossRef Search ADS PubMed 4. Ayalon L , Areán PA , Linkins K , Lynch M , Estes CL . Integration of mental health services into primary care overcomes ethnic disparities in access to mental health services between black and white elderly . Am J Geriatr Psychiatry . 2007 ; 15 ( 10 ): 906 – 912 . Google Scholar CrossRef Search ADS PubMed 5. Runyan CN , Fonseca VP , Meyer JG , Oordt MS , Talcott GW . A novel approach for mental health disease management: The Air Force Medical Service’s interdisciplinary model . Dis Manag . 2003 ; 6 ( 3 ): 179 – 188 . Google Scholar CrossRef Search ADS PubMed 6. Bryan CJ , Corso ML , Corso KA , Morrow CE , Kanzler KE , Ray-Sannerud B . Severity of mental health impairment and trajectories of improvement in an integrated primary care clinic . J Consult Clin Psychol . 2012 ; 80 ( 3 ): 396 – 403 . Google Scholar CrossRef Search ADS PubMed 7. Cigrang JA , Rauch SA , Mintz J et al. ; STRONG STAR Consortium . Treatment of active duty military with PTSD in primary care: A follow-up report . J Anxiety Disord . 2015 ; 36 : 110 – 114 . Google Scholar CrossRef Search ADS PubMed 8. Goodie JL , Isler WC , Hunter C , Peterson AL . Using behavioral health consultants to treat insomnia in primary care: a clinical case series . J Clin Psychol . 2009 ; 65 ( 3 ): 294 – 304 . Google Scholar CrossRef Search ADS PubMed 9. Berwick DM , Nolan TW , Whittington J . The triple aim: Care, health, and cost . Health Aff (Millwood) . 2008 ; 27 ( 3 ): 759 – 769 . Google Scholar CrossRef Search ADS PubMed 10. Mauksch L , Peek CJ , Fogarty CT . Seeking a wider lens for scientific rigor in emerging fields: The case of the primary care behavioral health model . Fam Syst Health . 2017 ; 35 ( 3 ): 251 – 256 . Google Scholar CrossRef Search ADS PubMed 11. Horevitz E , Organista KC , Arean PA . Depression treatment uptake in integrated primary care: How a “Warm Handoff” and other factors affect decision making by Latinos . Psychiatr Serv . 2015 ; 66 ( 8 ): 824 – 830 . Google Scholar CrossRef Search ADS PubMed 12. Reiter JT , Dobmeyer AC , Hunter CL . The primary care behavioral health (PCBH) model: An overview and operation definition . J Clin Psychol Med Settings . 2018 . doi: 10.1007/s10880-017-9531-x . [Epub ahead of print]. 13. Bryan CJ , Corso KA , Corso ML , Kanzler KE , Ray-Sannerud B , Morrow CE . Therapeutic alliance and change in suicidal ideation during treatment in integrated primary care settings . Arch Suicide Res . 2012 ; 16 ( 4 ): 316 – 323 . Google Scholar CrossRef Search ADS PubMed 14. Hunter CL , Goodie JL . Behavioral health in the Department of Defense Patient-Centered Medical Home: History, finance, policy, work force development, and evaluation . Transl Behav Med . 2012 ; 2 ( 3 ): 355 – 363 . Google Scholar CrossRef Search ADS PubMed 15. Defense Health Agency . Evaluation of the TRICARE Program: Fiscal Year 2017 Report to Congress . Falls Church, VA : Defense Health Agency ; 2017 . 16. Funderburk JS , Dobmeyer AC , Hunter CL , Walsh CO , Maisto SA . Provider practices in the primary care behavioral health (PCBH) model: An initial examination in the Veterans Health Administration and United States Air Force . Fam Syst Health . 2013 ; 31 ( 4 ): 341 – 353 . Google Scholar CrossRef Search ADS PubMed 17. Air Force Medical Operations Agency . Primary Care Behavioral Health Services: Behavioral Health Optimization Program Field Manual . Joint Base San Antonio – Lackland : Air Force Medical Service ; 2014 . 18. Corso KA , Bryan CJ , Corso ML et al. Therapeutic alliance and treatment outcome in the primary care behavioral health model . Fam Syst Health . 2012 ; 30 ( 2 ): 87 – 100 . Google Scholar CrossRef Search ADS PubMed 19. Dobmeyer AC , Hunter CL , Corso ML et al. Primary care behavioral health provider training: Systematic development and implementation in a large medical system . J Clin Psychol Med Settings . 2016 ; 23 ( 3 ): 207 – 224 . Google Scholar CrossRef Search ADS PubMed 20. Robinson PJ , Strosahl KD . Behavioral health consultation and primary care: Lessons learned . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 58 – 71 . Google Scholar CrossRef Search ADS PubMed 21. Shim R , Rust G . Primary care, behavioral health, and public health: Partners in reducing mental health stigma . Am J Public Health . 2013 ; 103 ( 5 ): 774 – 776 . Google Scholar CrossRef Search ADS PubMed 22. Landoll RR , Nielsen MK , Waggoner KK . US air force behavioral health optimization program: Team members’ satisfaction and barriers to care . Fam Pract . 2017 ; 34 ( 1 ): 71 – 76 . Google Scholar CrossRef Search ADS PubMed 23. Nielsen MK. Revolutionizing Mental Health Care Delivery in the United States Air Force by Shifting the Access Point to Primary Care . Maxwell AFB, AL : Air Command and Staff College ; 2016 . 24. Department of the Air Force . AFSC 4C0X1 Mental Health Service Specialty: Career Field Education and Training Plan . Washington, DC : Headquarters Air Force ; 2015 . 25. Kellermann AL , Saultz JW , Mehrotra A , Jones SS , Dalal S . Primary care technicians: A solution to the primary care workforce gap . Health Aff (Millwood) . 2013 ; 32 ( 11 ): 1893 – 1898 . Google Scholar CrossRef Search ADS PubMed 26. Oslin DW , Ross J , Sayers S , Murphy J , Kane V , Katz IR . Screening, assessment, and management of depression in VA primary care clinics. The Behavioral Health Laboratory . J Gen Intern Med . 2006 ; 21 ( 1 ): 46 – 50 . Google Scholar CrossRef Search ADS PubMed 27. Wang Y , Rodriguez de Gil PR , Chen Y et al. Comparing the performance of approaches for testing the homogeneity of variance assumption in one-factor ANOVA models . Educ Psychol Meas . 2017 ; 77 : 305 – 329 . Google Scholar CrossRef Search ADS 28. Lopez Bernal J , Cummins S , Gasparrini A . Interrupted time series regression for the evaluation of public health interventions: A tutorial . Int J Epidemiol . 2016 ; 46 : 348 – 355 . 29. Panattoni L , Stone A , Chung S , Tai-Seale M . Patients report better satisfaction with part-time primary care physicians, despite less continuity of care and access . J Gen Intern Med . 2015 ; 30 ( 3 ): 327 – 333 . Google Scholar CrossRef Search ADS PubMed 30. Kline R. Principles and Practice of Structural Equation Modeling . New York, NY : Guilford ; 2005 . 31. Cummings NA , O’Donohue WT , Cummings JL . The financial dimension of integrated behavioral/primary care . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 31 – 39 . Google Scholar CrossRef Search ADS PubMed 32. Miller BF , Mendenhall TJ , Malik AD . Integrated primary care: An inclusive three-world view through process metrics and empirical discrimination . J Clin Psychol Med Settings . 2009 ; 16 ( 1 ): 21 – 30 . Google Scholar CrossRef Search ADS PubMed 33. Muse AR , Lamson AL , Didericksen KW , Hodgson JL . A systematic review of evaluation research in integrated behavioral health care: Operational and financial characteristics . Fam Syst Health . 2017 ; 35 ( 2 ): 136 – 154 . Google Scholar CrossRef Search ADS PubMed 34. Zacks JM , Speer NK , Swallow KM , Braver TS , Reynolds JR . Event perception: A mind-brain perspective . Psychol Bull . 2007 ; 133 ( 2 ): 273 – 293 . Google Scholar CrossRef Search ADS PubMed Published by Oxford University Press on behalf of the Society of Behavioral Medicine 2018. This work is written by (a) US Government employee(s) and is in the public domain in the US.

Journal

Translational Behavioral MedicineOxford University Press

Published: May 4, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off