Background: Cardiovascular disease (CVD) is the leading cause of death in the US and incurs high health care costs. While many initiatives promote the implementation of ABCS (aspirin therapy, blood pressure control, cholesterol management, and smoking cessation) measures, most primary care practices (PCPs) lack quality improvement (QI) support and resources to achieve meaningful targets. The Healthy Hearts for Oklahoma (H2O) Study proposes to build a QI infrastructure by (1) constructing a sustainable Oklahoma Primary Healthcare Improvement Collaborative (OPHIC) to support dissemination and implementation (D&I) of QI methods; (2) providing QI support in PCPs to better manage patients at risk for CVD events. Parallel to infrastructure building, H2O aims to conduct a comprehensive evaluation of the QI support D&I in primary care and assess the relationship between QI support uptake and changes in ABCS measures. Methods: H2O has partnered with public health agencies and communities to build OPHIC and facilitate QI. H2O has 263 small primary care practices across Oklahoma that receive the bundled QI intervention to improve ABCS performance. A stepped-wedge designed is used to evaluate D&I of QI support. Changes in ABCS measures will be estimated as a function of various components of the QI support and capacity and readiness of PCPs to change. Notes from academic detailing and practice facilitation sessions will be analyzed to help interpret findings on ABCS performance. Discussion: H2O program is designed to improve cardiovascular health and outcomes for more than 1.25 million Oklahomans. The infrastructure established as a result of this funding will help reach medically underserved Oklahomans, particularly among rural and tribal populations. Lessons learned from this project will guide future strategies for D&I of evidence-based practices in PCPs. Trained practice facilitators will continue to serve as critical resource to assists small, rural PCPs in adapting to the ever-changing health environment and continue to deliver quality care to their communities. Keywords: Primary care, Quality improvement, Practice facilitation, Cardiovascular disease, Patient-centered outcomes, Implementation and dissemination * Correspondence: firstname.lastname@example.org College of Medicine, Department of Family and Preventive Medicine, The University of Oklahoma Health Sciences Center, 900 NE 10th St, Oklahoma City, OK 73104, USA Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Chou et al. BMC Health Services Research (2018) 18:404 Page 2 of 13 Background medical and surgical treatments of cardiovascular events, Cardiovascular disease (CVD) is the leading cause of 44% could be attributed to changes in risk factors, in- death in the US and accounts for 17% of national health cluding reductions in total cholesterol, systolic blood expenditures. Each year, more than two million adults in pressure control, reduced smoking prevalence, and in- the US experience a heart attack or stroke, with more crease in physical activity . Recommended strategies than 800,000 dying from CVD . By 2030, 40.5% of the to improve adherence to ABCS guidelines include US population is projected to have some form of CVD “team-based care, patient-centered medical homes, use and between 2010 and 2030, total direct medical costs of of health information technology (HIT), and interven- CVD are projected to triple from $273 billion to $818 tions to promote adherence.” These interventions should billion, with indirect costs, due to lost productivity, in- be “supported, evaluated, and disseminated rapidly to in- creasing from $172 billion to $276 billion . crease use of effective ABCS care practices .” Specific to the state of Oklahoma, which has ranked While ABCS guidelines are well defined, adherence in near the bottom across a host of health indicators, the the clinical setting is less than optimal. National data problem is even more alarming. CVD is the leading from the Million Hearts Initiative suggests that 54% of cause of death among Oklahomans [3, 4]. Based on the individuals at increased risk of CVD events are taking 2016 United Health Foundation rankings, Oklahoma has aspirin, 53% of those with hypertension have adequately the third highest CVD mortality rate in the US, with controlled blood pressure, 32% of individuals with high 325.9 CVD deaths per 100,000 population, compared to cholesterol are effectively managed, and 22% of people Minnesota, which has the lowest CVD death rate, with trying to quit smoking get counseling or treatment . 188.2 CVD deaths per 100,000 population . In 2014, Furthermore, a large number of US adults are unaware there were 5256 deaths among males and 4613 deaths of their high blood pressure and high cholesterol. Based among females attributed to heart disease in Oklahoma on data from the 2003–2010 National Health and Nutri- . The percentage of preventable CVD deaths is par- tion Examination Survey, investigators estimated that ticularly high among minority subgroups, with 65–70% among the 66.9 million US adults aged 18 years and of death among Black, American Indian, Asian/Pacific older with hypertension, 14.1 million (39%) are not Islander, and Hispanic males classified as preventable aware of their hypertension, 5.7 million (16%) are aware compared to roughly 45% of deaths among non- of their hypertension but are not receiving pharmaco- Hispanic Caucasian males . Percentages of prevent- logic treatment, and 16.0 million (45%) are aware of able CVD deaths are lower among females, but the same their hypertension and are being treated with medication trends are evident with 45–50% of deaths among Black, . Similarly, a large percentage of adults are unaware of American Indian, Asian/Pacific Islander, and Hispanic their high cholesterol status, at approximately 40%, and females classified as preventable compared to approxi- this rate is nearly 50% among Hispanic adults [8, 9]. mately 25% among non-Hispanic Caucasian females . While risk factor reduction, focused on ABCS measures, In response to the CVD burden and alarming projec- is recognized as a valuable approach to reducing CVD tions of CVD-related morbidity, mortality, and costs, the deaths, adherence to ABCS screening and treatment Department of Health and Human Services, along with guidelines in primary care is deficient, with high per- other government and private agencies, launched the centages of patients unaware of their elevated risk status Million Hearts Initiative in 2011. The goal of the Million and clinicians juggling competing demands and prioritiz- Hearts Initiative is to prevent one million heart attacks ing more acute illness over preventive screenings. New and strokes by 2017 through a focus on community- and strategies to engage patients in and provide resources to clinic-based strategies to manage “ABCS”– aspirin for practices for ABCS screening and management are ur- high-risk patients, blood pressure control, cholesterol gently needed to address these challenges. management, and smoking cessation . The initiative The Healthy Hearts for Oklahoma (H2O) Study, one focuses on the implementation of proven, effective, and of the seven collaboratives funded by the Agency for inexpensive interventions with two primary targets: (1) Healthcare Research and Quality (AHRQ) Evidence- Improve clinical management of low-dose aspirin use, NOW Initiative, proposes to build a quality improve- blood-pressure control, cholesterol management, and ment (QI) infrastructure in the state by (1) constructing smoking cessation; and (2) Expand community initiatives a sustainable Primary Healthcare Improvement Center to reduce smoking, improve nutrition, and reduce blood (OPHIC) that serves as a resource to support dissemin- pressure. Ford et al. investigated the impact of surgical ation and implementation (D&I) of QI methods in Okla- and medical treatments, relative to the reduction in cor- homa; (2) facilitating the implementation of a bundled onary risk factors, on coronary deaths between 1980 and QI intervention in primary care practices to improve the 2000. They found that while 47% of the reduction in the management of patients at risk for CVD events. Parallel age-adjusted death rate for CVD could be attributed to to infrastructure building, H2O aims to conduct a Chou et al. BMC Health Services Research (2018) 18:404 Page 3 of 13 comprehensive evaluation of the bundled QI interven- resources that will assist in planning QI activities using a tion implementation in primary care and hypothesizes listserv. that the QI intervention is associated with improvement in ABCS measures. Analysis Descriptive statistics will be used to summarize the Methods number of personnel and time required to recruit, train, QI infrastructure and deploy a sufficient number of ADs and PFs to sup- Resource center port H2O throughout the state. Retention of personnel H2O is developing a statewide D&I resource center, and geographic coverage areas of ADs and PFs, over the OPHIC, located within the state’s only comprehensive course of the study, will be summarized. academic health center, with the capability to track patient-centered outcomes research (PCOR) results, as- Implementation of multi-component QI support strategy sess needs of practices and communities, and provide QI implementation support strategy corresponding D&I support to community clinicians and H2O delivers to each participating practice the following practices. QI intervention components: (1) academic detailing pro- Specific to H2O, OPHIC is tasked with providing QI vided by a primary care physician, (2) baseline and and medical informatics support to primary care prac- monthly performance feedback, (3) practice facilitation tices. The OPHIC QI staff includes Practice Facilitator provided by a trained and certified PF, (4) Health Infor- Coordinators (PFCs), Practice Facilitators (PFs), Aca- mation Exchange (HIE) and HIT support, and (5) a Col- demic Detailers (ADs), and HIT Regional Extension laboration Website and listserv through which to share Center-Practice Advisors (REC-PAs). OPHIC recruits, best practices. Figure 1 provides the conceptual model, trains, and certifies these personnel in QI methods. As anchored in Solberg’s Practice Change Model, that eluci- the success of increasing QI capacity in small practices dates the likely effects of each component of the imple- relies on automated data collection, performance report- mentation support strategy on a practice’s QI priority, ing, and tracking, OPHIC, with its HIT consultants de- change capacity, and care process contents . velops technical specifications for data collection and reporting, provides technical assistance to the PFs and Academic detailing Academic detailing has been shown REC-PAs to guide practices with clinical data capture in effective for changing certain clinician behaviors includ- the electronic health records (EHR) that meets specifica- ing delivery of smoking cessation counseling  and tions to calculate clinical quality measures. OPHIC will appropriate use of antibiotics , though it was inef- also connect practices to knowledge and educational fective in increasing cervical cancer screening rates  Fig. 1 Conceptual model for implementation strategy Chou et al. BMC Health Services Research (2018) 18:404 Page 4 of 13 and implementing depression management guidelines documentation and reporting of ABCS data, and train . A Cochrane Collaboration review by O’Brien et al. practices to use HIE to generate performance reports. concluded that “educational outreach visits, particularly when combined with social marketing, appear to be a Collaboration website and listserv The website will in- promising approach to modifying professional behavior, clude dashboard pages for each participating practice especially prescribing .” H2O AD visits involve con- and county, which will be used primarily by practices versations with practice clinicians and staff about: 1) evi- and PFs. There will also be a set of project pages display- dence; 2) current practice; and 3) characteristics of high ing de-identified, comparative run charts and other pro- performing practices. Academic detailing begins with a ject data as well as resources and resource links. The kick-off meeting to elicit a preliminary QI plan for the listserv will be updated on a weekly basis with questions, practice. The AD uses evidence-based summaries and tips, and resource links. decision support tools in their work with the practice. The ADs will make at least two visits to each practice Practice recruitment and enrollment during the intervention period and stay in contact with Figure 2 presents the location of primary care practices the practice throughout the project. in Oklahoma by county. In total, it is estimated that Oklahoma has 2047 primary care practices that care for Performance feedback Performance feedback has been adults, but fewer than 25 have more than 10 primary demonstrated as one of the most effective mechanisms care clinicians. There are 46 Medicare certified Rural to motivate clinicians and practices to change [16–19]. Health Clinics , 17 community health centers pro- Performance feedback for this study is provided in two viding services with 58 sites , 52 Indian Health Ser- ways. First, reports are generated from the HIE and/or vices or American Indian tribal clinics, two Department EHR based upon patients’ meeting ABCS performance of Defense clinics, 13 Department of Veterans Affairs benchmarks. The practices receive baseline performance clinics, and 75 free clinics. Only practices with an EHR reports and then monthly reports for 1 year post imple- and a willingness to connect to the HIE were eligible to mentation of QI strategy. Second, we will disseminate participate in H2O. “best practices” from high performing practices and To recruit these practices, H2O collaborated with pro- share “lessons learned [20–24].” fessional associations, health systems, payers, the practice- based research network, and the Oklahoma City Area Practice facilitation Practice facilitation has proven use- Inter-Tribal Health Board. Incentives for participation in- ful for helping primary care practices with implementation cluded: (1) updates on the new blood pressure and lipid of new processes of care [25–27]. PFs embedded in the guidelines and ABCS decision aids; (2) in-practice QI sup- practice act as “change agents” and facilitate individualized port to enhance capacity, (3) assistance with Physician solutions through rapid plan-do-study-act QI cycles. The Quality Reporting System requirements for Medicare in- presence of a PF also serves as a reminder of the practice’s centives, (4) credits for MOC Part IV and for Continuing commitment to make changes and increases their capacity Medical Education, (5) assistance achieving Meaningful to do so . Assumptions inherent in the use of PFs in Use of EHR certification for enhanced payment, (6) assist- primary care are that many primary care practices are in- ance qualifying for the Medicare Transition of Care and adequately resourced, lack the experience and skills to Care Coordination payments, and (7) reimbursement for sustain a major QI initiative, and are so different from one expenses relating to the evaluation component. Moreover, another that implementation must be customized. The re- for practices in counties with a County Health Improve- lationships established by the PF with members of the ment Organization (CHIO), H2O worked with the CHIOs practice appear to be critical to their effectiveness . to provide an incentive of $1000 for each participating While facilitation is more expensive than most other QI practice to use for county-wide cardiovascular risk reduc- approaches, reductions in inappropriate testing may more tion campaigns. A PF contacted interested practices to ar- than offset these costs. For example, Hogg’s work showed range a kick-off visit to complete the enrollment process. a 1.4 return on investment on implementing preventive services . Analysis HIE and HIT support Advanced information systems Descriptive statistics will be used to summarize the de- will be required to provide ABCS performance reports ployment of the implementation support strategy among . H2O personnel help practices make more effective the enrolled practices over the course of the study. The use of their EHRs and participate in HIEs. The REC-PAs extent to which the implementation support strategy is visit each practice on a as needed-basis during the inter- implemented at each practice will be quantified by the PFs vention period to help practices maximize electronic based on categorical (qualitative) assessment variables. Chou et al. BMC Health Services Research (2018) 18:404 Page 5 of 13 Fig. 2 Primary care practices servicing adults in Oklahoma Counties Evaluation practices face competing priorities, a practice must iden- A comprehensive, systematic evaluation of the imple- tify a specific QI initiative that will most benefit the prac- mentation strategy among participating primary care tice’s mission and be supported by sufficient resources, practices will be conducted to assess uptake of the im- staff commitment, and buy-in. Second, a practice must plementation strategy as well as practice performance have the capacity and capability to change. This might in- and outcomes. clude a culture that supports innovation, regular QI team meetings, ability to generate performance reports, and tak- Logic model ing pride in seeing outcomes improve [10, 35, 36]. Third, Figure 3 presents the overall logic model guiding the “care process content” refers to processes such as delivery evaluation. The model includes the following compo- system design, decision support, and information systems, nents: inputs, outputs, external factors, and outcomes. etc. as well as any specific resource required to improve a Inputs include components of the implementation particular process. Addressing each of the output compo- support strategy and organizational contextual factors nents would result in significant, sustainable improve- that may facilitate or impede their uptake. Categories of ments in quality of care. inputs are adapted from the Consolidated Framework Inputs and outputs are influenced by external factors or for Implementation Research (CFIR). CFIR has outer setting, such as characteristics of the county in grouped these factors into a number of domains, provid- which the practice is located and community resources ing a “menu” for managers and operations leaders from (e.g., the availability of a CHIO). Inputs, outputs, and ex- which to select those that fit the particular setting and ternal factors, all affect outcomes. This evaluation aims situation to explain QI initiatives, guide diagnostic as- to examine two sets of outcomes: (1) the extent to which sessments of implementation, and evaluate implementa- the interventions have been implemented, as measured tion progress and outcomes. Using CFIR, we aim to by ABCS practice performance; and (2) patient-oriented identify contextual factors that affect other inputs, out- health outcomes (e.g., including utilization of EDs and puts, and outcomes in the following domains: (1) inter- hospitals, cardiovascular events, and deaths). vention characteristics, (2) characteristics of the individual implementers, (3) inner setting, (4) environ- Sampling methods ment; and (5) process of implementation (Table 1). The original design included a targeted enrollment of Outputs from the implementation support strategy, fa- 300 practices, but was revised to reflect a target of 250 cilitated by organizational contextual factors, are the three practices early in the implementation phase to address requirements for improvement identified in the concep- feasibility concerns. According to the design, the prac- tual framework: (1) priority for change; (2) change process tices are nested within counties, which are nested within capability; and (3) care process content . First, as 5 geographic sectors (4 quadrants of the state plus 2 Chou et al. BMC Health Services Research (2018) 18:404 Page 6 of 13 Fig. 3 Logic model metropolitan areas of Oklahoma and Tulsa counties). A system. ABCS performance will be evaluated at each of total of 50 practices were sampled per quadrant and 25 the 263 practices during each 3-month period where the practices were sampled per metro area (Fig. 2). The quad- time of initiation of implementation support is randomly rant area boundaries were based on the Area Health Edu- assigned. Each of 20 PFs were assigned to geographic cation Centers (AHEC) boundaries . When developing sub-regions, nested within the five geographic sectors, in the sampling scheme, a convenience sample of practices the state. The sub-regions reflected collections of coun- was drawn from within each county with the exception of ties that were feasible to access by a given PF. Within counties that share a CHIO. Counties that share a CHIO each sub-region, the consenting practices were randomly were considered a single sampling unit; therefore, practices assigned to begin the intervention program in Wave 1, were sampled from each of the 75 counties or paired county 2, 3, or 4 (Fig. 4). The targeted randomization was a total units. At the completion of recruitment in November 2015, of 13 randomized practices per PF with three to four a total of 263 practices consented and were recruited (Fig. 4). practices initiating the intervention program per Wave per PF. Random assignments were made in a manner to Evaluation design ensure a balance of Wave assignments for each PF and A stepped-wedge cluster randomized trial design was sub-region. In addition, during the 12-month implemen- used to evaluate the program. ABCS outcomes would be tation support period, two of the four programs were in- evaluated every 3 months beginning in month 7 of the troduced in each 6-month period, where blood pressure project following an initial 6-month period for recruit- management and smoking cessation support were intro- ment of the practices and practice units within each duced together and aspirin use and cholesterol manage- county, validation of the HIE data, and development of ment were introduced together. Within each of the four computing code for data abstraction from the HIE Waves, half of the counties introduced blood pressure Chou et al. BMC Health Services Research (2018) 18:404 Page 7 of 13 Table 1 Definitions for Consolidated Framework for Implementation Research (CFIR) Domains CFIR Domain Definition Innovation characteristics Innovation characteristics include the innovation itself, evidence strength and quality, relative advantage, complexity, design quality and packaging, etc. The innovation in this context is the implementation support strategy. For the purpose of this project, we will measure two characteristics of the implementation support strategy: complexity and relative advantage. Characteristics of the individual implementer Characteristics of clinicians and staff in a given practice who implement the support strategy include knowledge and understanding of the strategy, mindfulness, and personal attributes such as attitude, motivation, values, competency capacity, and learning style. Inner setting Organizational structure and mechanisms describe practices’ teamwork and communications, organizational culture, climate and readiness for implementation. Organizational climate illustrates practices’ tension for change, relative priority, incentives and rewards, goals and feedback, and learning. Environment The environment takes into account the location where the practice is situated, and the practice’s relationships with other organizations such as membership in a quality improvement network, health system, or professional society. Process of implementation The implementation process involves 4 stages: planning, engaging, executing, and reflecting and evaluating. Practices will work with ADs and PFs to select scheme, methods, and tasks for implementing the ABCS during the planning stage. The planning is followed by engaging the opinion leaders, internal implementation leaders, champions, and external change agents. The implementation plan is executed and evaluated with quantitative and qualitative feedback about the progress and quality of implementation. Abbreviations: ABCS aspirin, blood pressure, cholesterol and smoking measures, AD academic detailer, PF practice facilitator management and smoking cessation support first and questions from the Change Process Capacility Question- the other half introduced aspirin use and cholesterol naire (CPCQ) and National Ambulatory Medical Care management first, where the order assignment was ran- Survey (NAMCS). The 32-item CPCQ assesses outputs by domly determined to ensure a balance by PF sub-region. measuring three componenets of change capacity [38, 39]. The randomization sequences were generated using ran- The 25-item NAMCS assesses the degree of EHR adoption dom number generation in Excel. and functionalities at each practice (Additional file 1). To assess external factors and the environment, we document Surveys the county in which the practice is located and if the Guided by the logic model, three survey instruments have county has a CHIO to facilitate QI and if the practice is a been developed to identify inputs, implementer characteris- member of a current or previous QI network. Additionally tics, inner setting, and process that may be associated with we use the “5Ps Microsystem Dashboard” developed by each practice’s readiness to change. The first instrument, Dartmouth Institute for Health Policy and Clinical Practice Characteristics Survey, collects practice demo- Practice to describe the practice. This survey is completed graphic information as well as responses to validated by practice leadership/management. Fig. 4 Stepped wedge cluster randomized study design Chou et al. BMC Health Services Research (2018) 18:404 Page 8 of 13 The second instrument, the Practice Members Survey, organizational contexts and outputs. These data are col- assesses perceptions of the change process and context lected by PFs during visits to practices at three time among clinicians and staff using the 23-item Adapative points: (1) baseline, (2) at the end of the implementation Reserve questionnaire. Adaptive Reserve includes items of support strategy, and (3) 6 months post- that demonstrate a practice's ability to make and sustain implementation. Secondary data on performance (e.g., change, such as practice relationship infrastructure, ABCS) and patient-oriented outcomes (e.g., alignment of management functions in which clinical hospitalization, length of stay) is extracted from the care, practice operations, and financial functions share EHRs and the HIE. Practice performance, i.e., the num- and reflect a consistent vision, leadership, teamwork, ber of patients who achieved and provider performance work environment, and culture of learning . This on these measures, will be calculated for each practice survey is completed by multiple members of the same (Table 2). practice who occupy the clinical/administrative hier- archy (Additional file 2). Power analysis and effect sizes The third instrument, Electronic Practice Record, as- A series of simulation studies were performed to investi- sesses utility of implementation support strategy. PFs gate power of the planned evaluation study . A logis- and ADs contribute their notes to structured and semi- tic regression model was used to randomly generate data structured items during facilitation and academic detail- corresponding to the assumed percentage of patients ing sessions for each of ABCS metrics. satisfying each ABCS criterion prior to and after the im- plementation support period as specified in Table 3. The Data sources model was specified according to the mean model intro- The H2O master dataset will include both primary and duced by Hussey and Hughes  and included a ran- secondary data sources. The aforementioned surveys will dom effect corresponding to practice (the unit of be compiled to include variables describing randomization), a fixed implementation support effect Table 2 Outcome measures from HIE and/or EHR data Measure (Source) Numerator Denominator* Exclusion Data Source Aspirin Patients in denominator with Patients 18+ years of age On another anticoagulant, Health (PQRS 204/NQF 0068) documented use of aspirin or with Ischemic Vascular GI bleeding history, Information other antithrombotic Disease diagnosis, or aspirin allergy Exchange (HIE) hospital discharge for acute myocardial infarction, coronary artery bypass graft, or percutaneous coronary interventions Blood Pressure Patients in denominator whose Patients aged 18 through End stage HIE Management 1 blood pressure was adequately 85 years with a diagnosis renal disease, dialysis, renal (PQRS 236/NQF 0018) controlled (< 140/90) of hypertension transplant, or diagnosis of pregnancy Blood Pressure Patients in denominator whose Patients aged 18 through Lacking a DM HIE Management 2 blood pressure was adequately 85 with a diagnosis of or CKD diagnosis controlled (age 18–59 and/or hypertension people with diabetes or chronic kidney disease < 140/90; age 60–85 < 150/90) Cholesterol Patients in the denominator whose Patients aged 20 through HIE and Management 1 risk-stratified fasting LDL is at or 79 years of age who had chart audits (PQRS 316) below the recommended LDL goal a fasting LDL performed Cholesterol Patients in the denominator who Patients aged 20 through HIE and Management 2 were prescribed the recommended 79 years of age who had chart audits dose of statin based on risk status a fasting LDL performed Smoking Cessation Patients in denominator who were Patients age 18+ years Documentation of medical HIE Support screened about tobacco use one or reason(s) for not screening (PQRS 226/NQF 0028) more times within 24 months AND for tobacco use who received tobacco cessation counseling intervention if identified as a tobacco user Note: Population denominators will be based on active patient—defined as patients who have been seen in the practice within the previous 18 months Abbreviations: CABG coronary artery bypass grafting, CKD chronic kidney disease, COPD chronic obstructive pulmonary disease, CVA cerebral vascular accident, DM diabetes mellitus, ED emergency department, GI gastric intestinal, LDL low-density lipoprotein, LOS length of stay, MI myocardial infarction, PQRS physician quality reporting system, NQF National Quality Forum Chou et al. BMC Health Services Research (2018) 18:404 Page 9 of 13 Table 3 Summary of the assumed intervention effect sizes and power Endpoint Patient Cohort Characteristics Baseline Post-intervention Power (defining cohort eligible for intervention) Percentage Percentage A: Aspirin (aspirin or other antithrombotic prescribed) Aged 18 years and older with 60% 70% > 95% ischemic vascular disease without a contraindication to aspirin B1: Blood Pressure Management 1 Aged 18–85 who had a diagnosis 65% 70% > 95% (blood pressure adequately controlled) of hypertension B2: Blood Pressure Management 2 Aged 18–85 who had a 65% 70% > 95% (blood pressure adequately controlled diagnosis of hypertension as per co-morbidity-adjusted targets) C1: Cholesterol Management 1:** Aged 20–79 15% 25% > 95% (fasting LDL test AND ≤ LDL goal) C2: Cholesterol Management 2:** Aged 20–79 15% 25% > 95% (fasting LDL test AND prescribed statin if indicated) S: Smoking cessation support (screening about tobacco AND Aged 18 years and older 60% 70% > 95% received cessation counseling if tobacco user) Estimates for the baseline percentages for aspirin use, blood pressure management, and smoking cessation support are based on Oklahoma Foundation for Medical Quality (OFMQ) primary care practice initiatives in the state of Oklahoma ** The C1 and C2 measures were calculated as the product of the probability of having a fasting LDL test (estimated to be 0.3) multiplied by the probability of having an LDL measure below the target (estimated to be 0.50) or being prescribed a statin if indicated (estimated to be 0.5) conditional on having a fasting LDL test Abbreviation: LDL low-density lipoprotein where the implementation support covariate was mod- 6-month initiative from previous experience. We there- eled using a fractional covariate of (0.25, 0.5, 0.75, and 1. fore estimated a 10% drop-out rate in this 1-year project. 0) for the 3, 6, 9 and 12-month post-initiation practice Among the 263 recruited practices, we expect to retain performance measures to account for an implementation at least 234 practices for the entire duration of the support strategy that is not fully effective until the 12- 3-year evaluation program. month time point following initiation. Note that for half of the practices, the fractional covariate is delayed by an Analytic plan additional 6 months relative to the Series implementa- The evaluation plan has 2 analytic goals: (1) determine tion scheme given that one pair of targets (blood pres- the impact of the implementation support strategy on sure/smoking or aspirin/cholesterol) is delayed by 6 the practice’s capacity to change, performance indicators months in each practice. The model also included a ran- and patient outcomes; and (2) assess the role of internal dom error term. The correlation among measures made and external contextual factors and intervention charac- on patients from the same practice was assumed to be teristics on the uptake of the implementation support 0.10 or less . After the data were generated, General- strategy and on practice performance and outcomes. ized Estimating Equations (GEE) methodology was used to fit a logistic regression model to test the significance (1) Impact of the implementation support strategy on of the implementation support strategy effect. The per- the practice’s capacity to change, performance centage of simulated data sets that resulted in a signifi- indicators and patient outcomes cant intervention effect was recorded as the power of the specified study design. The power analysis assumes a The effect of the implementation support strategy on 2-sided 0.05 alpha level. For each ABCS endpoint, 500 the mean capacity to change score (priority for improve- simulated data sets were generated. Estimated power is ment, change capacity, and care process content), or the summarized for each target in Table 3. The power of the log odds of dichotomous indicators of change, will be es- implemented analyses will be higher than that from the timated using GEE to fit linear and logistic regression simulation studies, given that all active patients in the models, respectively, based on a within-cluster and targeted clinical population for a particular ABCS criter- between-cluster analysis . As discussed by Hussey ion during a 3-month period will be analyzed per prac- and Hughes , GEE has several attractive properties tice unit. that are relevant to the analysis of data arising from Practices drop out of QI projects when there is a sig- stepped wedge designs, including accommodating gener- nificant disruption in the practice’s internal (e.g., signifi- alized linear models that are appropriate for continuous, cant clinician or staff turnover, new EHR) or external count and dichotomous outcome measures, as well as (e.g., new ownership) environment. On average, we have robustness to variance and covariance misspecification. experienced a drop-out rate of about 5% during a The small sample limitations of GEE-based estimation Chou et al. BMC Health Services Research (2018) 18:404 Page 10 of 13 will not be of concern in this study given that 263 prac- Poisson regression models, with a log link, will be used tices will serve as the unit of randomization and 11 time to estimate the effect of the implementation support points will be utilized in the design. The outcome vari- strategy on the incidence rate of patient outcomes, (ED able will be a continuous score measure or a Yes/No in- visits, preventable hospitalizations, heart attacks, strokes, dicator variable measures at a practice unit level. The GI bleeds, deaths, etc.), where practice unit data are exposure of interest will be a dummy variable indicating summarized for each 3-month time period. The outcome the implementation support strategy exposure, which will be the number of patient events reported among the will be coded according to the randomized assignment active adult patients in a practice unit and the offset will (intent-to-treat analysis). Practice unit-level data will be be the total number of patient-months of time at risk dur- analyzed where the nested clustering of practice units ing the 3-month time period for the practice unit. GEE nested within practices, nested within sub-regions will will be used to account for the correlation among nested be accounted for using a structured working correlation practice units within practices within counties and modi- structure . Cluster sizes, by county, will vary due to fying factors will be investigated. the targeted sampling of metro counties (25 practices per county) and more rural counties (one to five prac- (2) Role of internal and external contextual factors and tices per county) and therefore, a jackknife estimate of intervention characteristics on the uptake of the the variance will be used to maintain the size of the test implementation support strategy and on practice . The time since the initiation of the implementation performance and outcomes. support strategy will also be considered as an independent factor in the regression model, along with a time by imple- General regression modeling strategies described by mentation support strategy interaction, to investigate the Baron and Kenny  will be used to investigate modify- effect of the implementation support strategy relative to ing and mediating factors, as presented in the Logic the time since initiation (i.e., delayed treatment effect). If a Model, controlling for clustering. In brief, factors that significant time by implementation support strategy effect modify the effect of the implementation support strategy is found, the analyses will be stratified by time period to will be indicated by significant interaction terms in the estimate the time-specific implementation support strat- regression model. For example, organizational context- egy effect. Time-varying secular trends will be considered ual factor scores will be investigated as modifying factors in the model, including the implementation of other to determine if there is evidence that the effect of the health intervention programs in the studied sub-regions. implementation support strategy is greater among those Geographic region will also be considered as a modifying practice units that have higher scores on the contextual factor where region is fit as a series of four indicator vari- factors. Mediating variables will be investigated by fitting ables (indicator for each quadrant, with the combined a series of four regression models. First, the effect of the Tulsa and Oklahoma City metro areas serving as the refer- implementation support strategy on the health outcome ence group). If appropriate based on estimates, a di- measure will be estimated. Second, the association be- chotomous region variable indicating metro versus tween the implementation support strategy (independent non-metro locations will be considered. A Type III factor) and the potential mediating factor (outcome) will Wald statistic will be used to determine the overall be estimated. Third, the association between the mediat- significance of the interaction between implementa- ing factor (independent factor) and the health outcome tion support strategy and the five degrees of freedom variable will be estimated. Finally, the model will be refit categorical class variable for region. A 2-sided 0.05 to include the implementation support strategy and me- alpha level will be used to define statistical signifi- diating term as independent factors with the health out- cance. An intention-to-treat paradigm will be followed come measure modeled as the outcome variable. A where data from all practice units are analyzed ac- change in the implementation support strategy coeffi- cording to the randomized assignment regardless of cient, indicating a shift towards no effect on the out- adherence or implementation quality. come, with adjustment for the mediating factor will Logistic regression models will be fit to estimate the provide evidence of a mediating association. effect of the implementation support strategy on the log odds of attaining an ABCS criterion for a given patient. Discussion The outcome variable will be a Yes/No indication of Impact attaining an ABCS criterion measured at a patient level. Oklahoma is among the poorest states in the nation, Individual-level data will be analyzed where the nested with a population of 3.85 million, 75.5% of which is clustering of patients within practice units, nested within white, 7.6% black, 9.0% American Indian, 1.9% Asian, practices and counties will be accounted for using a and 9.3% Hispanic . The state has 38 federally recog- structured working correlation structure . nized American Indian tribes and the majority of its Chou et al. BMC Health Services Research (2018) 18:404 Page 11 of 13 counties are rural. Most rural communities have a large patient outcomes. Until recently, the only method of proportion of vulnerable and underserved patients. Rele- obtaining QI data was to perform individual chart ab- vant to the CVD management goals of H2O, 23.3% of stractions of paper or electronic records. This is costly, Oklahomans are current smokers, 35.5% have hyperten- time-consuming, and invasive, although if done by pro- sion, 31.1% are physically inactive, and 32.2% are obese fessionals, remains the gold standard. This barrier may . In a 2011 government report, the quality of health be ameliorated with the use of EHR and connectivity to care in Oklahoma was rated as “weak,” particularly in an HIE. The PFs and REC-PAs will provide technical as- chronic disease management . The H2O study, lever- sistance to practices without EHRs to implement the aging the establishment of OPHIC, puts forth a coordi- system, and to practices with EHRs to generate practice- nated and systematic effort to engage and provide QI level performance reports electronically. As EHRs are resources to practices, in both rural and urban counties, connected to a HIE, these reports should be more accur- that serve vulnerable populations. Moreover, one of the ate because they can be based upon a numerator that in- reasons accounting for poor QI is the lack of infrastruc- cludes services received at multiple sites of care and a ture and appropriate technology. H2O provides technical denominator that includes the practice’s whole panel ra- and financial assistance to help practices with EHR sup- ther than a convenience sample. port and connection to a HIE. Third, recruitment and deployment of PF and REC- Over the course of the study, PFs, ADs, and PFCs sup- PAs to ensure an effective geographic distribution may port practices assisting them in establishing QI pro- be barrier given the size of the state. To minimize travel cesses, empaneling and risk stratifying their patients, time and maximize time in the practices, the hiring of and providing care coordination for those at the highest PFs was strategic in terms of location selection from all CVD risk. Such planned care has been demonstrated to quadrants of the state with the involvement of the reduce hospitalizations, ED visits, utilization, and use of AHECs , CHIOs, and practices. specialists when more active primary care is provided. In addition, practices may have the opportunity to connect Conclusion to the preventive services registry or implement en- H2O creates an infrastructure that supports a market hanced referral systems, both available through the HIE. for practice- and community-oriented professional de- All of these strategies will likely remain with the practice velopment/employment, and a well-tested feasible QI after study completion. model that is well-aligned with the public health system Enrollment in H2O also positions practices to new pay- in Oklahoma. The infrastructure and the QI model are ment opportunities with OPHIC’s potential capability to likely to be sustainable, especially if the infrastructure negotiate purchasing of advanced primary care services. may facilitate linkage between county health targets with Of the 263 practices, 195 will have 12 or more months at healthcare financing structure for primary care through the end of the study period to implement these advanced payment reform for value-based purchasing. Achieving primary care services and be prepared for a shared savings the study aims will benefit a state by improving its poor payment plan similar to that tested in the multi-payer health outcomes and providing additional resources. Comprehensive Primary Care program, or to participate in an Accountable Care Organization. Additional files Limitations Notwithstanding the benefits, the study notes a number Additional file 1: H2O Practice Characteristics Survey. (PDF 700 kb) of barriers that require mitigating strategies. First, there Additional file 2: H2O Practice Member Survey. (PDF 784 kb) may be potential barriers for practices to participate. Pri- mary care practices are often understaffed and face com- Abbreviations peting demands. They may experience difficulty meeting ABCS: Aspirin therapy, blood pressure control, cholesterol management, QI initiative deadlines and having their interest in sus- smoking cessation; AD: Academic detailers; AHEC: Area Health Education Centers; AHRQ: Agency for Healthcare Research and Quality; taining QI wane over time. Automation of clinical meas- CFIR: Consolidated framework for implementation research; ure data collection and reporting, on-site consultation CVD: Cardiovascular disease; D&I: Dissemination and implementation; by an AD for the clinicians, and the weekly presence or ED: Emergency department; EHR: Electronic health records; H2O: Healthy Hearts Oklahoma; HIE: Health information exchange; HIT: Health information availability of a PF will make the uptake of QI efforts technology; HITREC: Health Information Technology Regional Extension more feasible. Individualizing the implementation ap- Center; NAMCS: National Ambulatory Medical Care Survey; NHANES: National proach and routinely sharing lessons across practices Health and Nutrition Examination Survey; OPHIC: Oklahoma Primary Healthcare Improvement Center; ORCA: Organizational Readiness Change may also enhance the practice’s readiness to change. Assessment; PF: Practice facilitators; PFC: Practice facilitator coordinators; Second, a major barrier to QI has been the time and QI: Quality improvement; REC-PA: Regional Extension Center-Practice effort required to document practice performance and Advisors Chou et al. BMC Health Services Research (2018) 18:404 Page 12 of 13 Acknowledgements Results from the Hispanic Community Health Study/Study of Latinos. J Am We gratefully acknowledge the support of the entire project leadership, Heart Assoc. 2015;4(7) our practice facilitators, academic detailers, participating clinicians and 10. Solberg LI. Improving medical practice: a conceptual framework. Ann Fam their office staff, and numerous collaborators. We also extend our Med. 2007;5(3):251–6. appreciation to our partners, the Oklahoma Foundation for Medical 11. Goldstein MG, Niaura R, Willey C, Kazura A, Rakowski W, DePue J, Park E. An Quality (OFMQ), and MyHealth. academic detailing intervention to disseminate physician-delivered smoking cessation counseling: smoking cessation outcomes of the Physicians Funding Counseling Smokers Project. Prev Med. 2003;36(2):185–96. This research is supported by AHRQ R18 grant #1R18HS023919–01 (PI: D. Duffy). 12. Ilett KF, Johnson S, Greenhill G, Mullen L, Brockis J, Golledge CL, Reid DB. Modification of general practitioner prescribing of antibiotics by use of a Availability of data and materials therapeutics adviser (academic detailer). Br J Clin Pharmacol. 2000;49(2):168–73. Data sharing is not applicable as this article presents a study protocol. At the 13. Young JM, Ward JE. Randomised trial of intensive academic detailing to completion of the study, the dataset will be made publically available promote opportunistic recruitment of women to cervical screening by through Agency for Healthcare Research and Quality (AHRQ). general practitioners. Aust N Z J Public Health. 2003;27(3):273–81. 14. Brown JB, Shye D, McFarland BH, Nichols GA, Mullooly JP, Johnson RE. Authors’ contributions Controlled trials of CQI and academic detailing to implement a clinical AC composed the first draft of the manuscript, and with JM developed the practice guideline for depression. Jt Comm J Qual Improv. 2000;26(1):39–54. conceptual framework. JM, FDD, ZN, and SC developed the infrastructure 15. Thomson O’Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle N, plan and related components. AC, JS, JH, and JM developed the evaluation Harvey EL. Educational outreach visits: effects on professional practice and and data analysis plans. All authors read and approved the final manuscript. health care outcomes. Cochrane Database Syst Rev. 2000;2:CD000409. 16. Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and Ethics approval and consent to participate feedback: effects on professional practice and health care outcomes. This study was reviewed and approved by the University of Oklahoma Cochrane Database Syst Rev. 2006;2:CD000259. Institutional Review Board (IRB # 5251). All primary care practices that agreed 17. Kanouse DE, Jacoby I. When does information change practitioners’ to participate had undergone an informed consent process to complete behavior? Int J Technol Assess Health Care. 1988;4(1):27–33. enrollment. Practice facilitation and data collection began only after all 18. Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. required IRB paperwork has been filed. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285(22):2871–9. Competing interests 19. Thomson O'Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle N, The authors declare that they have no competing interests. Harvey EL. Audit and feedback versus alternative strategies: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2000;2:CD000260. Publisher’sNote 20. Brodsky KL, Baron RJ. A “best practices” strategy to improve quality in Springer Nature remains neutral with regard to jurisdictional claims in Medicaid managed care plans. J Urban Health. 2000;77(4):592–602. published maps and institutional affiliations. 21. Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Author details 1 Affairs. Arch Surg. 2002;137(1):20–7. College of Medicine, Department of Family and Preventive Medicine, The 22. Krakauer H, Lin MJ, Schone EM, Park D, Miller RC, Greenwald J, Bailey RC, University of Oklahoma Health Sciences Center, 900 NE 10th St, Oklahoma 2 Rogers B, Bernstein G, Lilienfeld DE, et al. ‘Best clinical practice’: assessment City, OK 73104, USA. School of Community Medicine, The University of of processes of care and of outcomes in the US Military Health Services Oklahoma Health Sciences Center, 4502 E 41st St, Tulsa, OK 74135, USA. 3 System. J Eval Clin Pract. 1998;4(1):11–29. College of Public Health, The University of Oklahoma Health Sciences 23. Mold JW, Gregory ME. Best practices research. Fam Med. 2003;35(2):131–4. Center, 801 NE 13th St, Oklahoma City, OK 73104, USA. 24. Oddone E, Brass LM, Booss J, Goldstein L, Alley L, Horner R, Rosen A, Kaplan L. Quality Enhancement Research Initiative in stroke: prevention, treatment, Received: 7 November 2017 Accepted: 4 May 2018 and rehabilitation. Med Care. 2000;38(6 Suppl 1):I92–104. 25. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012; References 10(1):63–74. 1. Frieden TR, Berwick DM. The “Million Hearts” initiative–preventing heart 26. Nagykaldi Z, Mold JW, Aspy CB. Practice facilitators: a review of the attacks and strokes. N Engl J Med. 2011;365(13):e27. literature. Fam Med. 2005;37(8):581–8. 2. Heidenreich PA, Trogdon JG, Khavjou OA, Butler J, Dracup K, Ezekowitz MD, 27. Nagykaldi Z, Mold JW, Robinson A, Niebauer L, Ford A. Practice Finkelstein EA, Hong Y, Johnston SC, Khera A, et al. Forecasting the future of facilitators and practice-based research networks. J Am Board Fam Med. cardiovascular disease in the United States: a policy statement from the 2006;19(5):506–10. American Heart Association. Circulation. 2011;123(8):933–44. 28. Rogers EM. Diffusion of innovations, 5th ed. New York: Free Press; 2003. 3. American Health Rankings. The United Health Foundation; 2016. 29. Rasmussen WD. Taking the university to the people : seventy-five years of 4. Center for Health Statistics, Oklahoma State Department of Health (OSDH). cooperative extension. 1st ed. Ames: Iowa State University Press; 1989. Vital Statistics 2014. Oklahoma Statistics on Health Available for Everyone 30. Hogg W, Baskerville N, Lemelin J. Cost savings associated with improving (OK2SHARE). 2014. appropriate and reducing inappropriate preventive care: cost-consequences 5. Ford ES, Ajani UA, Croft JB, Critchley JA, Labarthe DR, Kottke TE, Giles WH, analysis. BMC Health Serv Res. 2005;5(1):20. Capewell S. Explaining the decrease in U.S. deaths from coronary disease, 31. Nagykaldi Z, Mold JW. The role of health information technology in the 1980-2000. N Engl J Med. 2007;356(23):2388–98. translation of research into practice: an Oklahoma Physicians Resource/ 6. Wright J. Status of ABCS. Centers for Medicare and Medicaid Services (CMS); 2014. Research Network (OKPRN) study. J Am Board Fam Med. 2007;20(2): 7. CDC. Vital signs: awareness and treatment of uncontrolled hypertension 188–95. among adults–United States, 2003-2010. MMWR. 2012;61:703. 32. State Health Facts: Number of Medicare Certified Rural Health Clinics 8. Muntner P, Levitan EB, Brown TM, Sharma P, Zhao H, Bittner V, Glasser S, [http://kff.org/other/state-indicator/total-rural-health-clinics/]. Kilgore M, Yun H, Woolley JM, et al. Trends in the prevalence, awareness, treatment and control of high low density lipoprotein-cholesterol among 33. State Health Facts: Number of Federally-Funded Federally Qualified Health United States adults from 1999-2000 through 2009-2010. Am J Cardiol. Centers [http://kff.org/other/state-indicator/total-fqhcs/]. 2013;112(5):664–70. 34. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. 9. Rodriguez CJ, Cai J, Swett K, Gonzalez HM, Talavera GA, Wruck LM, Fostering implementation of health services research findings into Wassertheil-Smoller S, Lloyd-Jones D, Kaplan R, Daviglus ML. High practice: a consolidated framework for advancing implementation cholesterol awareness, treatment, and control among Hispanic/Latinos: science. Implement Sci. 2009;4:50. Chou et al. BMC Health Services Research (2018) 18:404 Page 13 of 13 35. Langley GJ. The improvement guide : a practical approach to enhancing organizational performance. 1st ed. San Francisco: Jossey-Bass Publishers; 1996. 36. Solberg LI, Brekke ML, Fazio CJ, Fowles J, Jacobsen DN, Kottke TE, Mosser G, O'Connor PJ, Ohnsorg KA, Rolnick SJ. Lessons from experienced guideline implementers: attend to many factors and use multiple strategies. Jt Comm J Qual Improv. 2000;26(4):171–88. 37. Area Health Education Centers [http://www.healthsciences.okstate.edu/ research/ahec/regional_centers.cfm]. 38. Solberg LI, Stuck LH, Crain AL, Tillema JO, Flottemesch TJ, Whitebird RR, Fontaine PL. Organizational factors and change strategies associated with medical home transformation. Am J Med Qual. 2015;30(4):337–44. 39. Solberg LI, Asche SE, Margolis KL, Whitebird RR. Measuring an organization's ability to manage change: the change process capability questionnaire and its use for improving depression care. Am J Med Qual. 2008;23(3):193–200. 40. Nutting PA, Crabtree BF, Stewart EE, Miller WL, Palmer RF, Stange KC, Jaen CR. Effect of facilitation on practice outcomes in the National Demonstration Project model of the patient-centered medical home. Ann Fam Med. 2010;8(Suppl 1):S33–44. S92 41. Dimairo MB, Walters M, Stephen J. Sample size determination through power simulation; practical lessons from a stepped wedge cluster randomised trial (SW CRT). Trials. 2011;12(Suppl 1) 42. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91. 43. Thompson DM, Fernald DH, Mold JW. Intraclass correlation coefficients typical of cluster-randomized studies: estimates from the Robert Wood Johnson Prescription for Health projects. Ann Fam Med. 2012;10(3):235–40. 44. Chao EC. Structured correlation in models for clustered data. Stat Med. 2006;25(14):2450–68. 45. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51(6):1173–82. 46. State & County QuickFacts [https://www.census.gov/quickfacts/OK]. 47. Key Health Data About Oklahoma [http://www.healthyamericans.org/states/ ?stateid=OK]. 48. 2011 National Healthcare Quality Report In.: Agency for Healthcare Research and Quality 2012.
BMC Health Services Research – Springer Journals
Published: Jun 4, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
All the latest content is available, no embargo periods.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud