Stakeholder perspectives on costs and resource expenditures: tools for addressing economic issues most relevant to patients, providers, and clinics

Stakeholder perspectives on costs and resource expenditures: tools for addressing economic issues... Abstract Cost and other resources required are often primary considerations in whether a potential program or policy will be adopted or implemented and an important element in determining value. However, few economic analyses are conducted from the perspective of patient/family or small-scale stakeholders such as local clinics. We outline and discuss alternative cost assessment and resource expenditures options from the perspective of these small, proximal stakeholders. The perspective of these persons differs from larger societal or health plan perspectives, and often differs across individuals in terms of what they value and the types of expenditures about which they are concerned. We discuss key features of the perspectives of patients, health care clinics, and local leaders, and present brief examples and sample templates for collection of consumer/stakeholder relevant cost and return on investment issues. These tools can be used prospectively and iteratively during program planning, intervention delivery, summative analysis, and preparation for dissemination stages. There is an important need for this type of feasible, pragmatic, rapid, and iterative cost and resource expenditure analysis directly relevant to patients/families, small local stakeholders and their organizations. Future research on and use of these approaches is recommended. Implications Practice: Cost templates can be used by patients, providers, and clinics to estimate the costs most relevant to them of participating in and delivering an intervention. Policy: Policymakers who want to support health promotion through implementation of translational programs should consider the costs relevant to stakeholders of such interventions and implications for sustainability. Research: Future research should examine the feasibility and utility of using templates to estimate and report the costs of implementing translational programs. BACKGROUND In an era of value-based decision-making, cost, staff time, and other resources required are often primary considerations in whether a potential program or policy will be adopted or implemented [1, 2]. With health care costs disproportionately rising in relation to other domestic expenditure categories, there is increasing interest and tacit incentives for maximizing patient health outcomes relative to dollars spent [1, 3]. Decreasing per capita health care cost is also a core element of the Triple Aim of Healthcare, endorsed by the Centers for Medicaid and Medicare services and most other health care payers and health systems—to improve the patient experience of care, to improve the health of populations, and to reduce the cost of health care [4]. As such, an integral component of measuring value is capturing the cost of health care and behavioral interventions from perspectives relevant to various stakeholders. The recommendations of the Second Panel on Cost-Effectiveness in Health and Medicine [5] provide authoritative guidance on cost-effectiveness analyses and primary outcomes studies. In this paper, we focus on much less discussed resources required for implementation, from the perspective of different local stakeholders. Most cost-effectiveness analyses focus on the large-scale societal perspective or that of large organizations such as national health insurers [6]. Few economic evaluations have addressed the perspective, values and concerns at the unit level of clinics, providers, patients, and community groups [7]. This absence of cost and resource expenditure estimation tools for use in community practice is unfortunate given the relevance of such tools to the evolving field of patient-centered outcomes research, such as that advocated by PCORI [8], as well as community-based participatory research, stakeholder engagement, and Clinical Translational Science Award hubs [9–12]. These groups could greatly benefit from better information associated with cost and resource allocation, along with the use predictive modeling techniques that is feasible, understandable, and relevant to their concerns [2, 13]. Cost analyses are seldom reported in behavioral interventions [14–17]. When cost analyses are reported, they are frequently conducted from the perspective of the payer or society as a whole as opposed to measures more relevant to implementing partners and stakeholders [2, 6, 7]. Many of these analyses are also overly complex for nonexperts to understand, or not relevant to specific consumers, including patients and families, clinicians, and small clinics (as opposed to large health plans). Accurately capturing costs associated with community-based interventions can be resource-intensive and complex [1]. Other perceived challenges to incorporating cost analyses in implementation evaluations are the need for highly trained professionals such as economists or accountants along with the lack of standardized measures. Practitioners often report cost as the primary factor for which interventions they select to implement [2]. However, researchers rarely disseminate cost information or consider its importance when developing new interventions. Ribisl et al. propose an approach whereby intervention developers assess what organizations adopting their interventions are willing to pay and then design interventions that are responsive to this price range [2]. The ultimate goal is to develop effective and affordable programs, such as lean interventions, or “minimal interventions needed for change” [18] which have been discussed in this journal. The take-home message from these approaches is that it is never too early to be thinking about dissemination (including adoption, implementation, and sustainability) [19] and that addressing cost—from the perspective of patients and consumers, potential adopting stakeholders and organizations—is central to this [15, 17]. Both Peek et al. [20] and Ribisl et al. [2] have described the need for better understanding of the resource and costs requirements to enhance translation of relevant and effective programs into community practices. In addition, Ritzwoller et al. have conducted several studies using pragmatic approaches to cost collection from the perspective of an adopting organization as well as assessment of costs from the perspective of the participant [7, 16, 21]. This line of research emphasizes estimation of replication costs in different kinds of settings and conditions and use of sensitivity analysis [16]. However, there is not one place that integrates these approaches and provides template(s) for identifying, collecting, and presenting results in ways that are responsive and relevant to stakeholders. This paper addresses the gap in practical resources to assess intervention costs and resources required from the perspectives of patients, families, and local clinical and community stakeholders. PURPOSE The purposes of this paper are to: (a) describe costs from the perspective of three different key stakeholders in today’s community-based participatory research and patient/stakeholder engagement research: patients and families [8]; clinical providers or staff implementing a policy or program; and local clinic, worksite, or community organizations [22]; (b) provide and illustrate use of templates for identifying and involving “small” stakeholders in cost definition and assessment; and (c) discuss directions for future research in assessing costs relevant to translational research. FRAMEWORK Perspective This paper focuses on three stakeholder groups: patients and their families, clinicians and implementing staff, and small organizations (clinics, worksites, and community organizations). Table 1 highlights the key issues for the three stakeholder perspectives including time costs, summary of economic costs, and opportunity cost (the next best use of time). Table 1 Common Issues for Various Perspectives and Stakeholders Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing View Large Table 1 Common Issues for Various Perspectives and Stakeholders Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing View Large Patients and families Resource considerations for patients include time spent participating in an intervention, including follow-up activities and assignments, as well as time required to travel to and from intervention sites when applicable. Costs to patients and families include transportation costs and any applicable out of pocket costs of intervention participation [7, 23]. Considerations for families vary by the extent to which they are involved in the intervention and/or transportation to and from it. Opportunity costs include alternative uses of time and interference with activities of daily living including time away from work (which may have financial implications as well). Clinicians and implementing staff Time considerations for clinicians and other implementing staff (e.g., social workers, community health workers) include the time spent implementing an intervention (including planning, training, and follow-up), as well as time spent recruiting participants and time spent (or saved) performing other tasks required by the program such as longer clinical visits or additional chart reviews. Opportunity costs to clinicians and staff include the next best use of time spent implementing an intervention, addressing other important clinical or patient/employee issues, such as lost clinical productivity and the associated ability to bill, or longer work hours resulting in less time for other activities of daily living. Time and opportunity costs may vary by level and type of staff and whether roles are consistent with or conflicting with usual work. In some cases, support staff may be able to take on activities that enable providers or higher paid staff to focus on clinical duties and may save time elsewhere or develop generalized strategies. Clinic, worksite, and local community groups Time costs to clinics, worksites, and/or local community groups includes hiring and training costs (and rehiring and retraining costs in cases of turnover), as well as program development costs. Economic costs may include personnel costs (intervention delivery personnel, supervision and administrative support, human resources, and training costs), and direct program costs (costs of resources used that are not already part of existing operating expenses), such as materials, equipment, office supplies, or software necessary to deliver the intervention. Opportunity costs include lost clinical productivity and the opportunity to bill, or other interventions/initiatives in which to participate. Templates We used elements from a five-step process developed by Ritzwoller et al. for assessing costs of implementing behavioral interventions in a pragmatic way [16]. The steps include defining the perspective from which the analysis will be conducted, identifying components necessary for intervention replication, selecting practical methods for cost capture, data analyses, and importantly, conducting sensitivity analyses to estimate replication costs given different conditions. Sensitivity analyses involve calculating costs under different assumptions or scenarios, such as having implementation done by staff at different pay grades, or identifying relevant patients by chart review versus electronic health record registries. Usually, sensitivity analyses are used to provide both a “high” and a “low” estimate to bound the likely costs of a program under different conditions or assumptions. Examples of sensitivity analyses include estimating the range of potential costs for various circumstances, as well as the ability to scale to different settings and populations or be delivered by different types of staff (e.g., physician, care manager, office staff) [16, 24]. The latter is important because specifying the conditions of a replication using sensitivity analyses can address the concerns, perspectives, and constraints of different potential adopters and adopting settings. The following templates are for implementation of an intervention including adaptation to local sites. Templates have been pilot tested with a convenience sample of clinicians, implementation experts, and patients. The templates are intended to encourage implementers to consider all relevant costs and are intended to be adapted and edited to best capture time and resources. Data can be obtained using a variety of procedures, including observation of activities, structured questions, and financial records of the organization. Research costs are not included; however, promotion and recruitment costs are considered, as they apply to interventions regardless of whether there is a research component. Promotion and recruitment can be costly and often neglected components of an intervention [16]. Operational templates for identifying, assessing, and summarizing relevant costs Tables 2–4 are templates intended to provide an estimate of time and economic costs relevant to three respective stakeholders: patients and families; clinicians and implementing staff; and clinics, worksites, or community organizations. These templates are intended to serve as a pragmatic guide for understanding and estimating time and economic costs, not as a rigorous or costly economic evaluation using precise techniques such as micro-costing. It is important to note that while costs for behavioral intervention programs can be divided into three or four categories (development, recruitment, implementation/intervention, and if relevant, research) [16], our templates focus exclusively on the implementation/replication costs and include a category for adaptation/local customization costs. These templates can also be used to compare differences from the perspective of the implementation site (e.g., differing implications between small practices that lack sophisticated electronic health records and large integrated health systems or academic medical centers). Table 2 Patient and Family Perspective: Time and Economic Costs of Participating in a Health Program This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ View Large Table 2 Patient and Family Perspective: Time and Economic Costs of Participating in a Health Program This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ View Large Table 3 Time and Economic Costs of Intervention Participation from the Clinician and Implementation Perspective This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 3 Time and Economic Costs of Intervention Participation from the Clinician and Implementation Perspective This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 4 Time and Economic Costs of Program from the Clinic, Worksite, or Local Community Group Perspective This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 4 Time and Economic Costs of Program from the Clinic, Worksite, or Local Community Group Perspective This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Participants are encouraged to fill out the cost template relevant to the perspective(s) of interest and select the costs within each template that apply to the program or policy being implemented. Again, these templates can be filled out by stakeholders with minimal instructions, completed in interview format, or used as tracking data sheets in time sampling methods. Table 2 contains items relevant to patients and families in estimating their costs of participating in a given intervention or enable program implementers to estimate the cost to the patient or participant and their family. Table 3 contains time and economic costs of intervention participation from the clinician and implementation perspective. Each column represents a different type of team member that may be involved in implementation (e.g., clinician, medical assistant, office staff) to allow for estimates of time costs by role. Table 4 contains time and economic costs relevant to a clinic, worksite, or local community group that are in addition to the costs and resources required of the personnel in Table 3. When relevant, Table 4 can be combined with Table 3 to capture total costs to the clinic including personnel costs. Application examples Clinician and local clinic example The My Own Health Record (MOHR) project included an assessment of the implementation and replication costs of collecting and providing feedback to clinic staff of brief surveys of patient-reported health behaviors, priorities, and concerns [25, 26]. The key goal of the MOHR project was to demonstrate the feasibility for wide-scale application of flexible, minimally burdensome assessment of and feedback on patient health behaviors (smoking, physical activity, dietary patterns) and behavioral health issues (depression, anxiety, alcohol use) and patients’ priorities for behavior change [25]. The interactive MOHR-automated assessment and feedback system was implemented via the clinic’s choice of modalities and timing: Internet collection before a visit, phone data collection by health system callers, via computer tablets in the waiting or exam room; or by staff administration for especially low literacy patients for those of exceptionally low health literacy or who did not speak English or Spanish. MOHR cost collection procedures were adapted from the procedures used by Ritzwoller et al. in prior research studies, and adapted for this context [7, 16, 21]. Individualized cost collection templates similar to those and an initial version of those presented above were developed for each participating clinic. This customization was important both to reduce burden and make data collection relevant and restricted to components of the MOHR program implemented in each clinic. Cost data were collected using a combination of procedures including interviews of implementation staff and observation of clinic processes (sometimes this involved multiple staff who had the most direct experience with different program components). MOHR engaged multiple clinic stakeholders on the front-end about the importance of and ways to best collect relevant costs to document time to implement MOHR procedures. This was done to maximize engagement and make the results were more directly usable by different types of primary care clinics, and potential funding and reimbursement agencies. Individualized cost data collection templates were developed and used for each clinic, based on this feedback and the modalities, timing and location of MOHR patient data collection, and the types of clinic staff involved in program implementation. Costs were assessed at two time points during the intervention, relatively early in the implementation and later in the implementation after stability was achieved. These cost collection procedures worked well and were consistently implemented in very diverse primary care clinics (e.g., community health centers, large health system settings, small rural clinics). We think that our success was due to both the effort put into developing simple, individually tailored cost templates and engagement of multiple clinic staff in helping us understand the processes their clinic used to implement the MOHR patient survey and feedback procedures. Patients and families There is a particular gap in the literature of ways to collect costs and resources required by patients and their families. To address this gap, we have provided the easy-to-use template in Table 2, and provide an example of costs from this perspective. Using an example from the literature, Ungar and Coyte assessed the cost of pediatric asthma care from several perspectives including the patient and family perspective [23]. The patient and family perspective included cost to parents for medication, health services not covered by insurance, and other out-of-pocket costs related to obtaining care. Indirect costs incurred by the family were also captured in the form of lost wages resulting time required to care for a child with asthma including obtaining care, travel, and waiting time. Data were collected by interviewing parents of asthma patients at three time points. Interview questions included demographics, symptom frequency, health services utilization, medication consumption, copayments and out-of-pocket costs for health services not covered by insurance, travel time and costs as well as waiting time [23]. DISCUSSION There is a pressing need for affordable and practical patient/family and small business perspective resource expenditure and cost assessments. The vast majority of existing academic, societal, and business examples have been from the perspective of large organizations or governmental agencies. The methods for such economic analyses are almost always implemented by health economists, accountants, or similar “expert” personnel; collected under often intrusive and time-consuming conditions; and conducted in the context of well-funded investigations. Many of the methods for these sophisticated and comprehensive cost (and cost-effectiveness, cost-utility, return on investment, and similar methods) analyses are relatively well worked out, but often not easy to apply [6, 27]. Many of these methods are not possible, relevant to or understandable by clinicians or small local stakeholders and their organizations, not to mention patients/families, citizen groups, or nonprofessionals. In an era of value-based decision-making and disproportionate increases in health care costs, there are compelling reasons to measure and maximize patient health outcomes and assess the cost of health care and behavioral interventions [1–3]. Identifying and utilizing pragmatic process and outcome measures by engaging stakeholders, including patients, implementation staff, and settings, early in the process is likely to lead to results that are more relevant, understandable, practical, and usable for stakeholders [25]. While precise economic studies can be costly and complex, there is also value in conducting cost studies that favor pragmatic measures that are intuitive and easily usable [1]. For some types of economic analysis, highly trained professionals such as economists may be needed. Krist et al. found that utilization of formal expenditure methods to collect intervention-specific data in primary care practices was feasible but that practitioners did not have the knowledge needed to collect reliable data independently [28]. As such, it is important to balance the level of precision needed with the appropriate economic guidance and expertise. To our knowledge, there are no papers that directly address these issues or provide templates and examples usable by nonresearchers. There have been important related articles, especially Ribisl [2] and the idea of assessing cost and willingness to pay used prospectively to create lean interventions, but our article addresses these issues from perspective of three important stakeholder groups (patients and families; health care teams; and community leaders). Also, Ritzwoller et al. have provided relevant examples of assessing costs in low resource settings, but for behavioral medicine research projects [7, 16, 21]. The approach and templates we discuss here are related to, but different from the Ribisl and Ritzwoller contributions in key concerns, themes; personnel that collect the data, and user perspective. In a related, but distinct approach, some evaluators have utilized micro-costing approaches which involve the detailed enumeration of each resource utilized for a defined intervention and its cost [29]. Micro-costing can be cost and time-intensive and is best suited to complement primary effectiveness studies, or for new interventions such as community health interventions when the average cost is unknown, to examine within-intervention variation, or when standard cost estimates are unlikely to be available [29]. While micro-costing analyses can be useful, they require expertise and assessments not feasible for simple analyses for individual clinics, organizations, patients, and community groups without resources or time to do an expensive, rigorous analysis. Our approach presented here can be viewed as an alternative and more simplified application of micro-costing [29], but from different stakeholder perspectives. The MOHR study by Krist et al. used a similar approach by combining economic analyses and primary care practice consultation, and an expenditure data collection instrument was created [25]. However, they concluded that even using their approach, “most practices and researchers lack the knowledge, expertise, and resources to collect such data independently” [25]. Hovmand has presented a relatively new and somewhat related approach to working with community groups to evaluate potential plans and programs, both prior to adoption and at the end of a program, that utilizes system dynamics modeling but with the assumptions and input of community members rather than researchers or economists [13]. To our knowledge, Hovmand’s intriguing approach has only been primarily used to predict and assess outcomes anticipated and unanticipated, rather than implementation costs [1, 13]. Our stakeholder-based cost assessment perspective and associated templates may also be especially relevant to PCORI grants and more generally, useful in patient-/stakeholder-oriented research. Possible extensions of our stakeholder-based cost assessments might include value-based assessment of perceived burden [1] and outcomes most valued to create individually (or local setting) specific stakeholder-based cost-effectiveness and cost-benefit estimates. Our stakeholder-specific cost and resource expenditure collection templates are derived from published work and have been piloted as described above. They appear to be reasonably low cost and feasible, but further research and quality improvement work is needed to identify types of costs and time expenditures that are most accurately (and inaccurately) assessed, types of training or guideline materials that are optimal, and methods through which these cost assessment approaches are best used (e.g., observation, interview, time sampling). Similarly, and most important from a pragmatic and “cost of cost-collection” perspective of concern to low resource settings and low budget evaluations, research is needed on conditions under which these templates can be usefully employed by different types of assessors. In particular, some may have concerns that busy practitioners cannot take even the time required to complete the template in Table 3. In such cases, it may be possible to either have other staff such as medical assistants or community health workers do observations of time required, or to have other staff such as office managers or nurses who work closely with providers complete the template and have it quickly reviewed by the provider. Limitations of our approach, templates, and recommendations are that, while based on our experience and use in prior studies and reports, they are not yet established and validated procedures or tools. We explicitly invite use, discussion, and refinement by others. In particular, there is a need to investigate how much these three different types of stakeholders can do on their own, with minimal personal or interactive assistance, and when and how much they need a professional to supervise or conduct assessments for them. CONCLUSION In conclusion, to enhance research translation and decision-making, there is both a need and opportunity for practical, stakeholder-based cost assessment to help local decision-makers, including patients and families, local worksites, and small clinics to evaluate locally and personally relevant costs. We are making the templates in this article available without charge to the public and encourage their use in both research and applied settings. The ultimate use criterion is whether these groups find our approach and templates feasible, useful, and helpful in their decision-making. Acknowledgements The authors would like to thank Robyn Wearner, MA, RD, for her contributions to the manuscript. No funding was received for this manuscript. The authors confirm that the findings reported have not been previously published and the manuscript is not being simultaneously submitted elsewhere. Additionally, there has been no previous reporting of data. The authors have full control of all primary data and agree to allow the journal to review their data if requested. Compliance with Ethical Standards Conflict of Interest: No conflicts of interest to report. Author contributions: W. C. Jones Rhodes and R. E. Glasgow conceptualized and drafted the manuscript. D. P. Ritzwoller and W. C. Jones Rhodes developed the templates. All authors critically revised the manuscript for intellectual content, read and approved the final manuscript. Ethical Approval: This work does not involve human subjects research and therefore does not require IRB approval. This work does not involve animals, so there are no concerns to report regarding the welfare of animals. Informed Consent: Informed consent is not applicable or required. References 1. Lee VS , Kawamoto K , Hess R et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality . JAMA . 2016 ; 316 ( 10 ): 1061 – 1072 . Google Scholar Crossref Search ADS PubMed 2. Ribisl KM , Leeman J , Glasser AM . Pricing health behavior interventions to promote adoption: Lessons from the marketing and business literature . Am J Prev Med . 2014 ; 46 ( 6 ): 653 – 659 . Google Scholar Crossref Search ADS PubMed 3. Porter ME . What is value in health care ? N Engl J Med . 2010 ; 363 ( 26 ): 2477 – 2481 . Google Scholar Crossref Search ADS PubMed 4. Institute for Healthcare Improvement . Available at www.ihi.org. Accessibility verified November 30, 2016 . 5. Sanders GD , Neumann PJ , Basu A et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine . JAMA . 2016 ; 316 ( 10 ): 1093 – 1103 . Google Scholar Crossref Search ADS PubMed 6. Gold M , Siegel J , Russell L , Weinstein M. Cost-effectiveness in health and medicine: report of the panel on cost-effectiveness in health and medicine . New York, NY : Oxford University Press ; 1996 . 7. Ritzwoller DP , Sukhanova AS , Glasgow RE et al. Intervention costs and cost-effectiveness for a multiple-risk-factor diabetes self-management trial for Latinas: Economic analysis of Viva Bien ! Transl Behav Med . 2011 ; 1 ( 3 ): 427 – 435 . Google Scholar Crossref Search ADS PubMed 8. Patient-Centered Outcomes Research Institute . Available at www.pcori.org. Accessibility verified December 5, 2016 . 9. National Center for Advancing Translational Sciences . Clinical and Translational Science Awards (CTSA) Program . 2016 . National Institutes of Health . Available at https://ncats.nih.gov/ctsa. Accessibility verified December 12, 2017. 10. Trochim WM , Rubio DM , Thomas VG . Evaluation guidelines for the Clinical and Translational Science Awards (CTSAs) . Clin Transl Sci . 2013 ; 6 ( 4 ): 303 – 309 . Google Scholar Crossref Search ADS PubMed 11. Grazier KL , Trochim WM , Dilts DM , Kirk R . Estimating return on investment in translational research methods and protocols . Eval Health Prof . 2013 ; 36 ( 4 ): 478 – 491 . Google Scholar Crossref Search ADS PubMed 12. Glasgow RE . What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation . Health Educ Behav . 2013 ; 40 ( 3 ): 257 – 265 . Google Scholar Crossref Search ADS PubMed 13. Hovmand PS. Community based system dynamics . New York, NY : Springer ; 2014 . 14. Gaglio B , Shoup JA , Glasgow RE . The RE-AIM framework: a systematic review of use over time . Am J Public Health . 2013 ; 103 ( 6 ): e38 – e46 . Google Scholar Crossref Search ADS PubMed 15. Crowley DM , Jones D . Financing prevention: opportunities for economic analysis across the translational research cycle . Transl Behav Med . 2016 ; 6 ( 1 ): 145 – 152 . Google Scholar Crossref Search ADS PubMed 16. Ritzwoller DP , Sukhanova A , Gaglio B , Glasgow RE . Costing behavioral interventions: a practical guide to enhance translation . Ann Behav Med . 2009 ; 37 ( 2 ): 218 – 227 . Google Scholar Crossref Search ADS PubMed 17. Saldana L , Chamberlain P , Bradford WD , Campbell M , Landsverk J . The cost of implementing new strategies (COINS): a method for mapping implementation resources using the stages of implementation completion . Child Youth Serv Rev . 2014 April 1 ; 39 : 177 – 182 . Google Scholar Crossref Search ADS 18. Glasgow RE , Fisher L , Strycker LA et al. Minimal intervention needed for change: definition, use, and value for improving health and health research . Transl Behav Med . 2014 ; 4 ( 1 ): 26 – 33 . Google Scholar Crossref Search ADS PubMed 19. Klesges LM , Estabrooks PA , Dzewaltowski DA , Bull SS , Glasgow RE . Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination . Ann Behav Med . 2005 ; 29 ( suppl ): 66 – 75 . Google Scholar Crossref Search ADS PubMed 20. Peek CJ , Glasgow RE , Stange KC , Klesges LM , Purcell EP , Kessler RS . The 5 R’s: an emerging bold standard for conducting relevant research in a changing world . Ann Fam Med . 2014 ; 12 ( 5 ): 447 – 455 . Google Scholar Crossref Search ADS PubMed 21. Ritzwoller DP , Glasgow RE , Sukhanova AY et al. Economic analyses of the Be Fit Be Well program: a weight loss program for community health centers . J Gen Intern Med . 2013 ; 28 ( 12 ): 1581 – 1588 . Google Scholar Crossref Search ADS PubMed 22. Minkler M , Wallerstein N . Community-based participatory research for health: from process to outcomes . 2 nd ed. San Francisco, CA : Jossey-Bass ; 2008 . 23. Ungar WJ , Coyte PC . Prospective study of the patient-level cost of asthma care in children . Pediatr Pulmonol . 2001 ; 32 ( 2 ): 101 – 108 . Google Scholar Crossref Search ADS PubMed 24. Shoup JA , Madrid C , Koehler C , Lamb C , Ellis J , Ritzwoller DP , Daley MF . Effectiveness and cost of influenza vaccine reminders for adults with asthma or chronic obstructive pulmonary disease . Am J Manag Care . 2015 ; 21 ( 7 ): e405 – e413 . Google Scholar PubMed 25. Krist AH , Glenn BA , Glasgow RE et al. Designing a valid randomized pragmatic primary care implementation trial: the my own health report (MOHR) project . Implement Sci . 2013 June 25 ; 8 : 73 . Google Scholar Crossref Search ADS 26. Glasgow RE , Kessler RS , Ory MG , Roby D , Gorin SS , Krist A . Conducting rapid, relevant research: lessons learned from the My Own Health Report project . Am J Prev Med . 2014 ; 47 ( 2 ): 212 – 219 . Google Scholar Crossref Search ADS PubMed 27. Drummond M. Methods for the economic evaluation of health care programmes . 4 th ed. Oxford, UK : Oxford University Press ; 2015 . 28. Krist AH , Cifuentes M , Dodoo MS , Green LA . Measuring primary care expenses . J Am Board Fam Med . 2010 ; 23 ( 3 ): 376 – 383 . Google Scholar Crossref Search ADS PubMed 29. Frick KD . Microcosting quantity data collection methods . Med Care . 2009 ; 47 ( 7 suppl 1 ): S76 – S81 . Google Scholar Crossref Search ADS PubMed © Society of Behavioral Medicine 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Translational Behavioral Medicine Oxford University Press

Stakeholder perspectives on costs and resource expenditures: tools for addressing economic issues most relevant to patients, providers, and clinics

Loading next page...
 
/lp/ou_press/stakeholder-perspectives-on-costs-and-resource-expenditures-tools-for-7pi14O0LbX
Copyright
© Society of Behavioral Medicine 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
ISSN
1869-6716
eISSN
1613-9860
D.O.I.
10.1093/tbm/ibx003
Publisher site
See Article on Publisher Site

Abstract

Abstract Cost and other resources required are often primary considerations in whether a potential program or policy will be adopted or implemented and an important element in determining value. However, few economic analyses are conducted from the perspective of patient/family or small-scale stakeholders such as local clinics. We outline and discuss alternative cost assessment and resource expenditures options from the perspective of these small, proximal stakeholders. The perspective of these persons differs from larger societal or health plan perspectives, and often differs across individuals in terms of what they value and the types of expenditures about which they are concerned. We discuss key features of the perspectives of patients, health care clinics, and local leaders, and present brief examples and sample templates for collection of consumer/stakeholder relevant cost and return on investment issues. These tools can be used prospectively and iteratively during program planning, intervention delivery, summative analysis, and preparation for dissemination stages. There is an important need for this type of feasible, pragmatic, rapid, and iterative cost and resource expenditure analysis directly relevant to patients/families, small local stakeholders and their organizations. Future research on and use of these approaches is recommended. Implications Practice: Cost templates can be used by patients, providers, and clinics to estimate the costs most relevant to them of participating in and delivering an intervention. Policy: Policymakers who want to support health promotion through implementation of translational programs should consider the costs relevant to stakeholders of such interventions and implications for sustainability. Research: Future research should examine the feasibility and utility of using templates to estimate and report the costs of implementing translational programs. BACKGROUND In an era of value-based decision-making, cost, staff time, and other resources required are often primary considerations in whether a potential program or policy will be adopted or implemented [1, 2]. With health care costs disproportionately rising in relation to other domestic expenditure categories, there is increasing interest and tacit incentives for maximizing patient health outcomes relative to dollars spent [1, 3]. Decreasing per capita health care cost is also a core element of the Triple Aim of Healthcare, endorsed by the Centers for Medicaid and Medicare services and most other health care payers and health systems—to improve the patient experience of care, to improve the health of populations, and to reduce the cost of health care [4]. As such, an integral component of measuring value is capturing the cost of health care and behavioral interventions from perspectives relevant to various stakeholders. The recommendations of the Second Panel on Cost-Effectiveness in Health and Medicine [5] provide authoritative guidance on cost-effectiveness analyses and primary outcomes studies. In this paper, we focus on much less discussed resources required for implementation, from the perspective of different local stakeholders. Most cost-effectiveness analyses focus on the large-scale societal perspective or that of large organizations such as national health insurers [6]. Few economic evaluations have addressed the perspective, values and concerns at the unit level of clinics, providers, patients, and community groups [7]. This absence of cost and resource expenditure estimation tools for use in community practice is unfortunate given the relevance of such tools to the evolving field of patient-centered outcomes research, such as that advocated by PCORI [8], as well as community-based participatory research, stakeholder engagement, and Clinical Translational Science Award hubs [9–12]. These groups could greatly benefit from better information associated with cost and resource allocation, along with the use predictive modeling techniques that is feasible, understandable, and relevant to their concerns [2, 13]. Cost analyses are seldom reported in behavioral interventions [14–17]. When cost analyses are reported, they are frequently conducted from the perspective of the payer or society as a whole as opposed to measures more relevant to implementing partners and stakeholders [2, 6, 7]. Many of these analyses are also overly complex for nonexperts to understand, or not relevant to specific consumers, including patients and families, clinicians, and small clinics (as opposed to large health plans). Accurately capturing costs associated with community-based interventions can be resource-intensive and complex [1]. Other perceived challenges to incorporating cost analyses in implementation evaluations are the need for highly trained professionals such as economists or accountants along with the lack of standardized measures. Practitioners often report cost as the primary factor for which interventions they select to implement [2]. However, researchers rarely disseminate cost information or consider its importance when developing new interventions. Ribisl et al. propose an approach whereby intervention developers assess what organizations adopting their interventions are willing to pay and then design interventions that are responsive to this price range [2]. The ultimate goal is to develop effective and affordable programs, such as lean interventions, or “minimal interventions needed for change” [18] which have been discussed in this journal. The take-home message from these approaches is that it is never too early to be thinking about dissemination (including adoption, implementation, and sustainability) [19] and that addressing cost—from the perspective of patients and consumers, potential adopting stakeholders and organizations—is central to this [15, 17]. Both Peek et al. [20] and Ribisl et al. [2] have described the need for better understanding of the resource and costs requirements to enhance translation of relevant and effective programs into community practices. In addition, Ritzwoller et al. have conducted several studies using pragmatic approaches to cost collection from the perspective of an adopting organization as well as assessment of costs from the perspective of the participant [7, 16, 21]. This line of research emphasizes estimation of replication costs in different kinds of settings and conditions and use of sensitivity analysis [16]. However, there is not one place that integrates these approaches and provides template(s) for identifying, collecting, and presenting results in ways that are responsive and relevant to stakeholders. This paper addresses the gap in practical resources to assess intervention costs and resources required from the perspectives of patients, families, and local clinical and community stakeholders. PURPOSE The purposes of this paper are to: (a) describe costs from the perspective of three different key stakeholders in today’s community-based participatory research and patient/stakeholder engagement research: patients and families [8]; clinical providers or staff implementing a policy or program; and local clinic, worksite, or community organizations [22]; (b) provide and illustrate use of templates for identifying and involving “small” stakeholders in cost definition and assessment; and (c) discuss directions for future research in assessing costs relevant to translational research. FRAMEWORK Perspective This paper focuses on three stakeholder groups: patients and their families, clinicians and implementing staff, and small organizations (clinics, worksites, and community organizations). Table 1 highlights the key issues for the three stakeholder perspectives including time costs, summary of economic costs, and opportunity cost (the next best use of time). Table 1 Common Issues for Various Perspectives and Stakeholders Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing View Large Table 1 Common Issues for Various Perspectives and Stakeholders Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing Patient and family perspective • Time spent participating in program or activities (including follow-up “homework”) • Transportation cost • Cost of intervention (if any) or direct out of pocket participation costs (if applicable) • Opportunity cost—alternative uses of time (what has to be given up in order to participate) Clinicians and implementing staff perspective • Time spent implementing the program (including planning, training, implementation, and follow-up) • Costs of resources used (materials, equipment, phone or web charges) • Promotion or recruitment costs (could be considered part of or separate from the first bullet) • Time spent (or saved) doing other activities necessitated by the program (e.g., longer visits, extra chart reviews, etc.) • Opportunity cost (lost clinical productivity and thus ability to bill, or longer hours, other) Clinic, worksite, and local community group perspective Costs: • Program development costs • Staffing (intervention delivery personnel, supervision and administrative support) • Hiring and training, and retraining costs • Other direct program costs • Opportunity costs (clinical productivity could be billing for, other interventions could participate in) • Scalability costs/marginal costing View Large Patients and families Resource considerations for patients include time spent participating in an intervention, including follow-up activities and assignments, as well as time required to travel to and from intervention sites when applicable. Costs to patients and families include transportation costs and any applicable out of pocket costs of intervention participation [7, 23]. Considerations for families vary by the extent to which they are involved in the intervention and/or transportation to and from it. Opportunity costs include alternative uses of time and interference with activities of daily living including time away from work (which may have financial implications as well). Clinicians and implementing staff Time considerations for clinicians and other implementing staff (e.g., social workers, community health workers) include the time spent implementing an intervention (including planning, training, and follow-up), as well as time spent recruiting participants and time spent (or saved) performing other tasks required by the program such as longer clinical visits or additional chart reviews. Opportunity costs to clinicians and staff include the next best use of time spent implementing an intervention, addressing other important clinical or patient/employee issues, such as lost clinical productivity and the associated ability to bill, or longer work hours resulting in less time for other activities of daily living. Time and opportunity costs may vary by level and type of staff and whether roles are consistent with or conflicting with usual work. In some cases, support staff may be able to take on activities that enable providers or higher paid staff to focus on clinical duties and may save time elsewhere or develop generalized strategies. Clinic, worksite, and local community groups Time costs to clinics, worksites, and/or local community groups includes hiring and training costs (and rehiring and retraining costs in cases of turnover), as well as program development costs. Economic costs may include personnel costs (intervention delivery personnel, supervision and administrative support, human resources, and training costs), and direct program costs (costs of resources used that are not already part of existing operating expenses), such as materials, equipment, office supplies, or software necessary to deliver the intervention. Opportunity costs include lost clinical productivity and the opportunity to bill, or other interventions/initiatives in which to participate. Templates We used elements from a five-step process developed by Ritzwoller et al. for assessing costs of implementing behavioral interventions in a pragmatic way [16]. The steps include defining the perspective from which the analysis will be conducted, identifying components necessary for intervention replication, selecting practical methods for cost capture, data analyses, and importantly, conducting sensitivity analyses to estimate replication costs given different conditions. Sensitivity analyses involve calculating costs under different assumptions or scenarios, such as having implementation done by staff at different pay grades, or identifying relevant patients by chart review versus electronic health record registries. Usually, sensitivity analyses are used to provide both a “high” and a “low” estimate to bound the likely costs of a program under different conditions or assumptions. Examples of sensitivity analyses include estimating the range of potential costs for various circumstances, as well as the ability to scale to different settings and populations or be delivered by different types of staff (e.g., physician, care manager, office staff) [16, 24]. The latter is important because specifying the conditions of a replication using sensitivity analyses can address the concerns, perspectives, and constraints of different potential adopters and adopting settings. The following templates are for implementation of an intervention including adaptation to local sites. Templates have been pilot tested with a convenience sample of clinicians, implementation experts, and patients. The templates are intended to encourage implementers to consider all relevant costs and are intended to be adapted and edited to best capture time and resources. Data can be obtained using a variety of procedures, including observation of activities, structured questions, and financial records of the organization. Research costs are not included; however, promotion and recruitment costs are considered, as they apply to interventions regardless of whether there is a research component. Promotion and recruitment can be costly and often neglected components of an intervention [16]. Operational templates for identifying, assessing, and summarizing relevant costs Tables 2–4 are templates intended to provide an estimate of time and economic costs relevant to three respective stakeholders: patients and families; clinicians and implementing staff; and clinics, worksites, or community organizations. These templates are intended to serve as a pragmatic guide for understanding and estimating time and economic costs, not as a rigorous or costly economic evaluation using precise techniques such as micro-costing. It is important to note that while costs for behavioral intervention programs can be divided into three or four categories (development, recruitment, implementation/intervention, and if relevant, research) [16], our templates focus exclusively on the implementation/replication costs and include a category for adaptation/local customization costs. These templates can also be used to compare differences from the perspective of the implementation site (e.g., differing implications between small practices that lack sophisticated electronic health records and large integrated health systems or academic medical centers). Table 2 Patient and Family Perspective: Time and Economic Costs of Participating in a Health Program This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ View Large Table 2 Patient and Family Perspective: Time and Economic Costs of Participating in a Health Program This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ This template can help you to estimate and understand the costs of participating in a health program, including your time.  • Your answers do not need to be exact - your best guess will do.  • Think about how many hours each week you spend participating in the program, doing follow-up “homework” and how much time it takes you to get to and from the location.  • If family members come with you or help you participate, record how much time they spend. How much time do YOU spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes How much time does YOUR FAMILY spend on this program each week? This includes time to:  Meet for the actual program: _______minutes  Work on “homework” for the project: _______minutes  Get to and from the location: _______minutes  Other activities related to the program: _______minutes Are you paying for transportation to be part of the project? (circle one) ____Yes ____ No  If yes, how much do you think you spend on transportation or gas each week? $______ Is there anything else you paying for to be part of the project? (circle one) ____Yes ____ No  If yes, what else do you pay for? How much do you think you spend to be part of the project?  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________  Type of Cost (e.g materials for the program)_______________$________ NOTES: Please list any questions, comments, other things we should know about your answers above: _____________________________________________________________________________________________________ View Large Table 3 Time and Economic Costs of Intervention Participation from the Clinician and Implementation Perspective This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 3 Time and Economic Costs of Intervention Participation from the Clinician and Implementation Perspective This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. The program staff member in each location who has the best knowledge of the program in your setting should provide the information requested in the template.  • In some cases, different persons may need to complete different sections.  • The program staff member collecting the data should primarily rely on their knowledge and understanding of the program in their location to fill in the estimates below.   Do not worry about having to be precise: Your best estimate is sufficient.  • If helpful, you can also use a combination of observation of activities, interviews with staff and records of your organization.  • For each job title, add the total number of hours per person per week. Next, multiply the number of hours per week by the number of implementation staff. To quantify staffing costs, multiply this figure by the average hourly rate for each job title. Clinician and Delivery Staff Perspective Staff Time - All hours (list hours per week) Job Title: Physician Medical Assistant Front office staff Billing/ coding Care Manager (Health educator; patient navigator) Behavioral Health Provider Other roles (please specify) Number of staff who deliver the program with each job title Average hourly rate (including benefits) Time spent attending meetings (include implementation team meetings, development, staff meetings related to implementation, other relevant activities) Training and supervisions related to program (or other phases - planning, follow-up) Recruitment (including participant screening and recruiting patients) Assessment: Pre, during post, follow-up baseline (enrollment) data. Note: Include only assessment related to the intervention, please exclude assessments that are for research purposes only. Intervention:  - Time for notes, charting, preparation for meetings  - Patient training on equipment  - Coordination of services (pharmacy, labs, etc),  - Intervention delivery documentation or quality control  - Other activities directly related to intervention delivery Total hours per week If/when the intervention needs to be changed or adapted to fit the participants or your setting, specify time needed and over what time period. NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 4 Time and Economic Costs of Program from the Clinic, Worksite, or Local Community Group Perspective This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Table 4 Time and Economic Costs of Program from the Clinic, Worksite, or Local Community Group Perspective This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. This template estimates costs to the clinic, worksite or local community group beyond the time costs of staff implementation. This cost data collection template is designed to estimate the resources associated with the implementation of a given program. It is meant to be edited and adapted to enable you to best capture time and resources relevant to your program. Your best guess is sufficient here. If helpful, you can also use a combination of observation of activities, interviews with staff and financial records of the organization. Clinic, worksite, and local community group Number of new employees hired (implementation-specific) Average cost of hiring a new employee (if available from Human Resources) $ Training costs (cost of training program and/or staff time to deliver training, travel) Note: Exclude trainee time costs (included above in Table 3) $ Costs of program related equipment (only include what is not already used for current role) –e.g. computers, tablets, phones, hardware $ Software, licenses (only include what is not already used for current role) $ Travel expenses (specific to the implementation process) Note: Exclude trainee time costs if included in staff time above $ Cost of materials specific to the program (e.g. marketing materials, rental equipment, use of space that is not already available (e.g. renting space outside the clinic or office location; do not include overhead or indirect costs) $ NOTES, COMMENTS: Please list any questions, comments, other things we should know about your answers above. View Large Participants are encouraged to fill out the cost template relevant to the perspective(s) of interest and select the costs within each template that apply to the program or policy being implemented. Again, these templates can be filled out by stakeholders with minimal instructions, completed in interview format, or used as tracking data sheets in time sampling methods. Table 2 contains items relevant to patients and families in estimating their costs of participating in a given intervention or enable program implementers to estimate the cost to the patient or participant and their family. Table 3 contains time and economic costs of intervention participation from the clinician and implementation perspective. Each column represents a different type of team member that may be involved in implementation (e.g., clinician, medical assistant, office staff) to allow for estimates of time costs by role. Table 4 contains time and economic costs relevant to a clinic, worksite, or local community group that are in addition to the costs and resources required of the personnel in Table 3. When relevant, Table 4 can be combined with Table 3 to capture total costs to the clinic including personnel costs. Application examples Clinician and local clinic example The My Own Health Record (MOHR) project included an assessment of the implementation and replication costs of collecting and providing feedback to clinic staff of brief surveys of patient-reported health behaviors, priorities, and concerns [25, 26]. The key goal of the MOHR project was to demonstrate the feasibility for wide-scale application of flexible, minimally burdensome assessment of and feedback on patient health behaviors (smoking, physical activity, dietary patterns) and behavioral health issues (depression, anxiety, alcohol use) and patients’ priorities for behavior change [25]. The interactive MOHR-automated assessment and feedback system was implemented via the clinic’s choice of modalities and timing: Internet collection before a visit, phone data collection by health system callers, via computer tablets in the waiting or exam room; or by staff administration for especially low literacy patients for those of exceptionally low health literacy or who did not speak English or Spanish. MOHR cost collection procedures were adapted from the procedures used by Ritzwoller et al. in prior research studies, and adapted for this context [7, 16, 21]. Individualized cost collection templates similar to those and an initial version of those presented above were developed for each participating clinic. This customization was important both to reduce burden and make data collection relevant and restricted to components of the MOHR program implemented in each clinic. Cost data were collected using a combination of procedures including interviews of implementation staff and observation of clinic processes (sometimes this involved multiple staff who had the most direct experience with different program components). MOHR engaged multiple clinic stakeholders on the front-end about the importance of and ways to best collect relevant costs to document time to implement MOHR procedures. This was done to maximize engagement and make the results were more directly usable by different types of primary care clinics, and potential funding and reimbursement agencies. Individualized cost data collection templates were developed and used for each clinic, based on this feedback and the modalities, timing and location of MOHR patient data collection, and the types of clinic staff involved in program implementation. Costs were assessed at two time points during the intervention, relatively early in the implementation and later in the implementation after stability was achieved. These cost collection procedures worked well and were consistently implemented in very diverse primary care clinics (e.g., community health centers, large health system settings, small rural clinics). We think that our success was due to both the effort put into developing simple, individually tailored cost templates and engagement of multiple clinic staff in helping us understand the processes their clinic used to implement the MOHR patient survey and feedback procedures. Patients and families There is a particular gap in the literature of ways to collect costs and resources required by patients and their families. To address this gap, we have provided the easy-to-use template in Table 2, and provide an example of costs from this perspective. Using an example from the literature, Ungar and Coyte assessed the cost of pediatric asthma care from several perspectives including the patient and family perspective [23]. The patient and family perspective included cost to parents for medication, health services not covered by insurance, and other out-of-pocket costs related to obtaining care. Indirect costs incurred by the family were also captured in the form of lost wages resulting time required to care for a child with asthma including obtaining care, travel, and waiting time. Data were collected by interviewing parents of asthma patients at three time points. Interview questions included demographics, symptom frequency, health services utilization, medication consumption, copayments and out-of-pocket costs for health services not covered by insurance, travel time and costs as well as waiting time [23]. DISCUSSION There is a pressing need for affordable and practical patient/family and small business perspective resource expenditure and cost assessments. The vast majority of existing academic, societal, and business examples have been from the perspective of large organizations or governmental agencies. The methods for such economic analyses are almost always implemented by health economists, accountants, or similar “expert” personnel; collected under often intrusive and time-consuming conditions; and conducted in the context of well-funded investigations. Many of the methods for these sophisticated and comprehensive cost (and cost-effectiveness, cost-utility, return on investment, and similar methods) analyses are relatively well worked out, but often not easy to apply [6, 27]. Many of these methods are not possible, relevant to or understandable by clinicians or small local stakeholders and their organizations, not to mention patients/families, citizen groups, or nonprofessionals. In an era of value-based decision-making and disproportionate increases in health care costs, there are compelling reasons to measure and maximize patient health outcomes and assess the cost of health care and behavioral interventions [1–3]. Identifying and utilizing pragmatic process and outcome measures by engaging stakeholders, including patients, implementation staff, and settings, early in the process is likely to lead to results that are more relevant, understandable, practical, and usable for stakeholders [25]. While precise economic studies can be costly and complex, there is also value in conducting cost studies that favor pragmatic measures that are intuitive and easily usable [1]. For some types of economic analysis, highly trained professionals such as economists may be needed. Krist et al. found that utilization of formal expenditure methods to collect intervention-specific data in primary care practices was feasible but that practitioners did not have the knowledge needed to collect reliable data independently [28]. As such, it is important to balance the level of precision needed with the appropriate economic guidance and expertise. To our knowledge, there are no papers that directly address these issues or provide templates and examples usable by nonresearchers. There have been important related articles, especially Ribisl [2] and the idea of assessing cost and willingness to pay used prospectively to create lean interventions, but our article addresses these issues from perspective of three important stakeholder groups (patients and families; health care teams; and community leaders). Also, Ritzwoller et al. have provided relevant examples of assessing costs in low resource settings, but for behavioral medicine research projects [7, 16, 21]. The approach and templates we discuss here are related to, but different from the Ribisl and Ritzwoller contributions in key concerns, themes; personnel that collect the data, and user perspective. In a related, but distinct approach, some evaluators have utilized micro-costing approaches which involve the detailed enumeration of each resource utilized for a defined intervention and its cost [29]. Micro-costing can be cost and time-intensive and is best suited to complement primary effectiveness studies, or for new interventions such as community health interventions when the average cost is unknown, to examine within-intervention variation, or when standard cost estimates are unlikely to be available [29]. While micro-costing analyses can be useful, they require expertise and assessments not feasible for simple analyses for individual clinics, organizations, patients, and community groups without resources or time to do an expensive, rigorous analysis. Our approach presented here can be viewed as an alternative and more simplified application of micro-costing [29], but from different stakeholder perspectives. The MOHR study by Krist et al. used a similar approach by combining economic analyses and primary care practice consultation, and an expenditure data collection instrument was created [25]. However, they concluded that even using their approach, “most practices and researchers lack the knowledge, expertise, and resources to collect such data independently” [25]. Hovmand has presented a relatively new and somewhat related approach to working with community groups to evaluate potential plans and programs, both prior to adoption and at the end of a program, that utilizes system dynamics modeling but with the assumptions and input of community members rather than researchers or economists [13]. To our knowledge, Hovmand’s intriguing approach has only been primarily used to predict and assess outcomes anticipated and unanticipated, rather than implementation costs [1, 13]. Our stakeholder-based cost assessment perspective and associated templates may also be especially relevant to PCORI grants and more generally, useful in patient-/stakeholder-oriented research. Possible extensions of our stakeholder-based cost assessments might include value-based assessment of perceived burden [1] and outcomes most valued to create individually (or local setting) specific stakeholder-based cost-effectiveness and cost-benefit estimates. Our stakeholder-specific cost and resource expenditure collection templates are derived from published work and have been piloted as described above. They appear to be reasonably low cost and feasible, but further research and quality improvement work is needed to identify types of costs and time expenditures that are most accurately (and inaccurately) assessed, types of training or guideline materials that are optimal, and methods through which these cost assessment approaches are best used (e.g., observation, interview, time sampling). Similarly, and most important from a pragmatic and “cost of cost-collection” perspective of concern to low resource settings and low budget evaluations, research is needed on conditions under which these templates can be usefully employed by different types of assessors. In particular, some may have concerns that busy practitioners cannot take even the time required to complete the template in Table 3. In such cases, it may be possible to either have other staff such as medical assistants or community health workers do observations of time required, or to have other staff such as office managers or nurses who work closely with providers complete the template and have it quickly reviewed by the provider. Limitations of our approach, templates, and recommendations are that, while based on our experience and use in prior studies and reports, they are not yet established and validated procedures or tools. We explicitly invite use, discussion, and refinement by others. In particular, there is a need to investigate how much these three different types of stakeholders can do on their own, with minimal personal or interactive assistance, and when and how much they need a professional to supervise or conduct assessments for them. CONCLUSION In conclusion, to enhance research translation and decision-making, there is both a need and opportunity for practical, stakeholder-based cost assessment to help local decision-makers, including patients and families, local worksites, and small clinics to evaluate locally and personally relevant costs. We are making the templates in this article available without charge to the public and encourage their use in both research and applied settings. The ultimate use criterion is whether these groups find our approach and templates feasible, useful, and helpful in their decision-making. Acknowledgements The authors would like to thank Robyn Wearner, MA, RD, for her contributions to the manuscript. No funding was received for this manuscript. The authors confirm that the findings reported have not been previously published and the manuscript is not being simultaneously submitted elsewhere. Additionally, there has been no previous reporting of data. The authors have full control of all primary data and agree to allow the journal to review their data if requested. Compliance with Ethical Standards Conflict of Interest: No conflicts of interest to report. Author contributions: W. C. Jones Rhodes and R. E. Glasgow conceptualized and drafted the manuscript. D. P. Ritzwoller and W. C. Jones Rhodes developed the templates. All authors critically revised the manuscript for intellectual content, read and approved the final manuscript. Ethical Approval: This work does not involve human subjects research and therefore does not require IRB approval. This work does not involve animals, so there are no concerns to report regarding the welfare of animals. Informed Consent: Informed consent is not applicable or required. References 1. Lee VS , Kawamoto K , Hess R et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality . JAMA . 2016 ; 316 ( 10 ): 1061 – 1072 . Google Scholar Crossref Search ADS PubMed 2. Ribisl KM , Leeman J , Glasser AM . Pricing health behavior interventions to promote adoption: Lessons from the marketing and business literature . Am J Prev Med . 2014 ; 46 ( 6 ): 653 – 659 . Google Scholar Crossref Search ADS PubMed 3. Porter ME . What is value in health care ? N Engl J Med . 2010 ; 363 ( 26 ): 2477 – 2481 . Google Scholar Crossref Search ADS PubMed 4. Institute for Healthcare Improvement . Available at www.ihi.org. Accessibility verified November 30, 2016 . 5. Sanders GD , Neumann PJ , Basu A et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine . JAMA . 2016 ; 316 ( 10 ): 1093 – 1103 . Google Scholar Crossref Search ADS PubMed 6. Gold M , Siegel J , Russell L , Weinstein M. Cost-effectiveness in health and medicine: report of the panel on cost-effectiveness in health and medicine . New York, NY : Oxford University Press ; 1996 . 7. Ritzwoller DP , Sukhanova AS , Glasgow RE et al. Intervention costs and cost-effectiveness for a multiple-risk-factor diabetes self-management trial for Latinas: Economic analysis of Viva Bien ! Transl Behav Med . 2011 ; 1 ( 3 ): 427 – 435 . Google Scholar Crossref Search ADS PubMed 8. Patient-Centered Outcomes Research Institute . Available at www.pcori.org. Accessibility verified December 5, 2016 . 9. National Center for Advancing Translational Sciences . Clinical and Translational Science Awards (CTSA) Program . 2016 . National Institutes of Health . Available at https://ncats.nih.gov/ctsa. Accessibility verified December 12, 2017. 10. Trochim WM , Rubio DM , Thomas VG . Evaluation guidelines for the Clinical and Translational Science Awards (CTSAs) . Clin Transl Sci . 2013 ; 6 ( 4 ): 303 – 309 . Google Scholar Crossref Search ADS PubMed 11. Grazier KL , Trochim WM , Dilts DM , Kirk R . Estimating return on investment in translational research methods and protocols . Eval Health Prof . 2013 ; 36 ( 4 ): 478 – 491 . Google Scholar Crossref Search ADS PubMed 12. Glasgow RE . What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation . Health Educ Behav . 2013 ; 40 ( 3 ): 257 – 265 . Google Scholar Crossref Search ADS PubMed 13. Hovmand PS. Community based system dynamics . New York, NY : Springer ; 2014 . 14. Gaglio B , Shoup JA , Glasgow RE . The RE-AIM framework: a systematic review of use over time . Am J Public Health . 2013 ; 103 ( 6 ): e38 – e46 . Google Scholar Crossref Search ADS PubMed 15. Crowley DM , Jones D . Financing prevention: opportunities for economic analysis across the translational research cycle . Transl Behav Med . 2016 ; 6 ( 1 ): 145 – 152 . Google Scholar Crossref Search ADS PubMed 16. Ritzwoller DP , Sukhanova A , Gaglio B , Glasgow RE . Costing behavioral interventions: a practical guide to enhance translation . Ann Behav Med . 2009 ; 37 ( 2 ): 218 – 227 . Google Scholar Crossref Search ADS PubMed 17. Saldana L , Chamberlain P , Bradford WD , Campbell M , Landsverk J . The cost of implementing new strategies (COINS): a method for mapping implementation resources using the stages of implementation completion . Child Youth Serv Rev . 2014 April 1 ; 39 : 177 – 182 . Google Scholar Crossref Search ADS 18. Glasgow RE , Fisher L , Strycker LA et al. Minimal intervention needed for change: definition, use, and value for improving health and health research . Transl Behav Med . 2014 ; 4 ( 1 ): 26 – 33 . Google Scholar Crossref Search ADS PubMed 19. Klesges LM , Estabrooks PA , Dzewaltowski DA , Bull SS , Glasgow RE . Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination . Ann Behav Med . 2005 ; 29 ( suppl ): 66 – 75 . Google Scholar Crossref Search ADS PubMed 20. Peek CJ , Glasgow RE , Stange KC , Klesges LM , Purcell EP , Kessler RS . The 5 R’s: an emerging bold standard for conducting relevant research in a changing world . Ann Fam Med . 2014 ; 12 ( 5 ): 447 – 455 . Google Scholar Crossref Search ADS PubMed 21. Ritzwoller DP , Glasgow RE , Sukhanova AY et al. Economic analyses of the Be Fit Be Well program: a weight loss program for community health centers . J Gen Intern Med . 2013 ; 28 ( 12 ): 1581 – 1588 . Google Scholar Crossref Search ADS PubMed 22. Minkler M , Wallerstein N . Community-based participatory research for health: from process to outcomes . 2 nd ed. San Francisco, CA : Jossey-Bass ; 2008 . 23. Ungar WJ , Coyte PC . Prospective study of the patient-level cost of asthma care in children . Pediatr Pulmonol . 2001 ; 32 ( 2 ): 101 – 108 . Google Scholar Crossref Search ADS PubMed 24. Shoup JA , Madrid C , Koehler C , Lamb C , Ellis J , Ritzwoller DP , Daley MF . Effectiveness and cost of influenza vaccine reminders for adults with asthma or chronic obstructive pulmonary disease . Am J Manag Care . 2015 ; 21 ( 7 ): e405 – e413 . Google Scholar PubMed 25. Krist AH , Glenn BA , Glasgow RE et al. Designing a valid randomized pragmatic primary care implementation trial: the my own health report (MOHR) project . Implement Sci . 2013 June 25 ; 8 : 73 . Google Scholar Crossref Search ADS 26. Glasgow RE , Kessler RS , Ory MG , Roby D , Gorin SS , Krist A . Conducting rapid, relevant research: lessons learned from the My Own Health Report project . Am J Prev Med . 2014 ; 47 ( 2 ): 212 – 219 . Google Scholar Crossref Search ADS PubMed 27. Drummond M. Methods for the economic evaluation of health care programmes . 4 th ed. Oxford, UK : Oxford University Press ; 2015 . 28. Krist AH , Cifuentes M , Dodoo MS , Green LA . Measuring primary care expenses . J Am Board Fam Med . 2010 ; 23 ( 3 ): 376 – 383 . Google Scholar Crossref Search ADS PubMed 29. Frick KD . Microcosting quantity data collection methods . Med Care . 2009 ; 47 ( 7 suppl 1 ): S76 – S81 . Google Scholar Crossref Search ADS PubMed © Society of Behavioral Medicine 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

Translational Behavioral MedicineOxford University Press

Published: Sep 8, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off