Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Joint Fire Science Program (JFSP) Fire Science Exchange Network: A National Evaluation of Initiative Impacts

Joint Fire Science Program (JFSP) Fire Science Exchange Network: A National Evaluation of... Abstract The Joint Fire Science Program (JFSP) Fire Science Exchange Network is composed of 15 Exchanges that act as boundary organizations tasked with improving fire science use within their respective regions. A longitudinal survey conducted annually between 2011 and 2015 as part of a larger evaluation effort reveals significant impacts. Results indicate that fire science practitioners who are constituents of Exchanges increased their agreement that scientists respect their experience and input, their regional Exchange improves access to fire science, and they use fire science in their work. Additionally, scientists agreed increasingly that Exchanges are necessary to share fire science information and improve the application of fire science in their region. These positive findings suggest that Exchanges’ boundary organizational activities impact the adoption of fire science research in the field and that Exchanges are fulfilling their role in disseminating and increasing the use of fire science. Boundary organizations, logic model, translational science Management and Policy Implications The Joint Fire Science Program Fire Science Exchange Network is composed of 15 Exchanges across the United States that act as boundary organizations to increase the dissemination and use of fire science. Three key implications for management and policy are highlighted: the logic model framework for program planning and evaluation, the importance of relationship-building between practitioners and scientists, and the use of a variety of educational outreach methods and materials. First, a comprehensive logic model of the Exchange initiative as well as individual Exchange logic models were developed. These models provided structure for prioritizing activities by directly linking them to short-, medium-, and long-term objectives that needed to occur in succession. Thus, logic models helped identify strategies to reach desired changes. Second, Exchanges recognized that rather than being passive recipients of research findings, practitioners needed to communicate to scientists their current and future management needs. Providing opportunities for these professionals to interact improved relationships, practitioners’ perceptions that their experiences were valued, and increased fire science adoption. Finally, the use of a variety of outreach products allowed Exchanges to reach wider audiences with differing learning styles, professional roles, and time constraints. Exchanges were able to provide in-person activities as well as harness emerging social media technologies, maintaining a coherent message across educational outreach platforms. Future outreach initiatives should continue to use a variety of educational strategies to foster interpersonal connections, ensuring that a cohesive strategy (outlined in a logic model) guides implementation. The need to link federally funded research with real-world application and impact has led to an increase in translational science initiatives (National Institute of Health [NIH] 2004, Zerhouni 2007, Hall and Fleishman 2010, Wethington and Dunifon 2012, Calhoun et al. 2013). Broadly, translational science is defined as the transformation of scientific research findings into applied products and procedures for on-the-ground use (Sung et al. 2003, Westfall et al. 2007, Wooten et al. 2013). In the past decade, natural resource and conservation fields have identified gaps between research generated by scientists and the application of that research among practitioners on the job (Laurance et al. 2012, Hulme 2014). Although there is a need to translate and communicate fire science research results for practitioners to encourage greater use of research findings in the field (Cerveny and Ryan 2008, Ascher et al. 2010), the co-production of knowledge whereby practitioners inform the research process is necessary to effectively bridge the gap between fire science production and application (Lemos and Morehouse 2005, White et al. 2010). Therefore, strategies to increase the use of scientific research have sought to improve relationships between scientists and practitioners, specifically through increasing mutual respect while emphasizing practitioners’ information needs and field experiences (McNie 2007, Laurance et al. 2012, Hulme 2014, Patton 2014, Schwandt 2014). The US Department of the Interior’s Joint Fire Science Program (JFSP) was developed in 1998 to promote applied fire science research. After 10 years, however, the JFSP Board realized that the proliferation of fire science research did not necessarily increase the application of research results on the ground (Kocher et al. 2012). In 2009, JFSP conducted a needs assessment to discover why fire science availability alone was not enough to increase the use of fire science among practitioners. The needs assessment found that the primary barriers to fire science application included lack of practitioners’ access to relevant and practical information as well as communication barriers between scientists and practitioners (Kocher et al. 2012). Therefore, the JFSP funded a nationwide network of fire science information Exchanges based on a boundary organization approach. Boundary organizations work to break down demarcations between traditionally siloed research and practitioner communities so that science research outputs become more relevant (Guston 2001). Through increasing mutual respect and interaction between these professional communities, scientists direct their research focus toward practitioners’ information needs. Practitioners, seeing that scientists value their input, increase their trust and use of the resulting science. Thus, improved relationships are necessary to increase the co-production of knowledge and subsequent adoption of practices derived from that communal knowledge (Guston 2001, McNie 2007). As Exchanges increase accessibility to translated fire science research products, they increase interaction between scientists and practitioners, which serves to improve relationships and enhances mutual respect and collaboration (JFSP 2011). To date, the JFSP Fire Science Exchange Network includes 15 Exchanges distributed across the United States. An evaluation of Exchange activities between 2011 and 2015 measured the efficacy of these organizations to improve fire science accessibility, science use and application, and relationships between practitioners and scientists. The following article reports the survey findings of these evaluation efforts. The Evaluation of the JFSP Exchanges In 2010, an independent team of investigators launched an evaluation of the JFSP Fire Science Exchange Network’s outcomes and impacts, based on Theory of Change (TOC) using a logic model framework. The TOC posits that a logical progression exists for achieving outcomes and that programs should work backwards from their overarching goals to determine intermediate and beginning prerequisite steps (Patton 2014, Stufflebeam 2001, Weiss 1997). Thus, TOC is concerned with the causal modeling of outcomes, paying particular attention to the progressive outcomes that build on one another. The logic model is a practical tool for mapping how a program’s activities translate into the short-, medium-, and long-term outcomes to achieve its larger goals (Patton 2014). The TOC and logic model define short-term outcomes as the first steps necessary for change and involve factors such as increased awareness of an issue, knowledge gains, and changes in attitudes. Medium-term outcomes involve changes in behavior and necessitate that short-term outcomes have first been achieved. Changes in behavior indicate a higher level of success that is more difficult to achieve. Similarly, long-term outcomes often reflect ultimate organizational and programmatic goals, such as impacting environmental conditions. The evaluation of Exchanges involved development of a national logic model to map outcomes relevant to the entire initiative, which served to direct strategies and identify indicators for measuring program outcomes and impacts over time (see Figure 1). Figure 1. View largeDownload slide National logic model for JFSP fire science exchange network. Figure 1. View largeDownload slide National logic model for JFSP fire science exchange network. The JFSP Fire Science Exchange Network experienced growth over the course of the evaluation. When the national evaluation began in 2010, the JFSP had funded eight Exchanges that varied in terms of program development and implementation of outreach activities and materials. Between 2010 and 2013, JFSP funded six additional Exchanges, which further emphasized developmental differences among the Exchanges. In 2015, an additional Exchange was added in the northeast region of the United States. A suite of evaluation activities was designed and implemented to track the JFSP initiative’s progress. One of four primary evaluation activities included an annual survey to measure Exchange users’ perceptions of overall outcomes and impacts. Exchanges were required to participate in the survey every two to three years in alignment with their funding cycles. This schedule reduced the burden on Exchanges to participate every year as well as incorporated flexibility as new Exchanges entered the initiative. A national evaluation of Exchanges was possible because each Exchange shared common goals and sought to reach those goals through similar outreach activities. For example, in addition to social media accounts, each Exchange maintained an online presence by hosting a website that provided translated fire science products and educational webinars. Also, each Exchange conducted in-person educational events that included workshops, field tours, field demonstrations, and science-based presentations at conferences and practitioner meetings. The ongoing evaluation focused on how practitioners rated their experiences with their regional Exchange and with fire scientists, and conversely, how scientists rated their experiences with their Exchange and with practitioners. This paper highlights the longitudinal quantitative results from the annual national Fire Exchange evaluative survey. Method An annual survey was developed in 2010 and administered online over the course of five years (2011–2015). The survey was designed to evaluate Exchange constituents’ experiences with their regional Exchange. Experiences included the Exchange website, educational events and products, as well as experiences with other fire science professionals. Since Exchanges are required to participate only during their funding year, for each year the survey was administered, participants represented Exchanges established at different points in time. To account for this variation in establishment, data were separated based on the Exchanges’ year of funding as an indicator of developmental maturity. It was assumed that Exchanges would need time to impact their constituents. Differentiating Exchanges by year of establishment allowed for analyses that compared responses of survey participants affiliated with Exchanges in their first and second years of funding (e.g., Exchanges’ early establishment period) with responses from their fourth year of funding. At the time of these analyses, all but one Exchange had been established at least four years and therefore that Exchange was omitted from this study. The analyses reported here represent data collected from 14 established Exchanges across their initial four years of funding. Participants and Recruitment A total of 3283 participants comprised the combined dataset that spans all survey waves. Of the total participants, 2625 were practitioners and 658 were scientists. The majority of participants in the current sample were male (72.6%). When examining race and ethnicity, the majority of participants were White/Caucasian (90.0%), followed by “Other” (3.2%), Hispanic/Latino (1.9%), Native American/Alaskan native (1.5%), Multi-ethnic (1.4%), Asian/Pacific Islander (1.3%), and Black/African American (0.7%). Average age of participants was 46.6 years, with a standard deviation of 11.1 years. Participants had worked in their field on average for 17.8 years with a standard deviation of 10.7 years. These demographics were proportionally consistent throughout all survey years. The survey design was tailored to target Exchanges’ two primary audiences: practitioners, who consume fire science for professional or land management intentions, and scientists, who are the producers of fire science research. During a survey implementation year, Exchanges recruited participants from their list of members via a snowball sampling strategy with three recruitment stages (Dillman et al. 2010). The initial survey invitation informed participants of the purpose of the survey and provided links to the survey, hosted by Survey Monkey™. These links could be forwarded via email to additional potential survey participants. Two additional invitations to participate were sent at two- and four-week intervals, respectively, following the initial invitation. Measures Survey questions were developed and finalized through collaborative efforts between the university-based independent evaluation team, Exchange personnel, and members of the JFSP Board. In 2010, evaluators examined the approved funding proposals submitted by the first eight Exchanges to identify common proposed activities and desired outcomes. Evaluators created a first draft of survey questions. Multiple revisions of question items were conducted in collaboration with Exchange personnel and representatives of the JFSP Board until a final draft of the survey was approved. Prior to launching the survey, all items were mapped onto the national logic model as a final check to verify alignment and coverage of national outcomes. As minor revisions to the survey occurred over time, a core set of questions remained consistent across annual survey administrations to allow for longitudinal analyses. Institutional Review Board approval was obtained prior to recruiting participants for the initial and each subsequent year of data collection. The survey targeted practitioners and scientists. Participants who indicated that their work involved applying fire science in the field were directed to the practitioner survey, whereas participants indicating they were fire science researchers were directed to the scientist survey. Both surveys featured parallel questions about experiences with Exchanges and the Exchange websites. Additionally, each group was asked about their perceptions of one another. For example, practitioners were asked about their experiences working with fire scientists, whereas scientists were asked about their experiences working with practitioners. Skip patterns were integrated into the survey so that participants without knowledge of their Exchange or who had never visited their Exchange website were redirected from topic-specific questions. Most survey participants (70% for all survey years) were familiar with their regional Exchange and Exchange website. All questions were measured on a 5-point Likert-type scale where 1 = Strongly Disagree and 5 = Strongly Agree. Scales Data from all survey years were combined to facilitate analyses by funding year. Respondents were categorized as affiliates of Exchanges based on their regional Exchange’s first and fourth years of funding. We developed post hoc the scales described here from existing survey questions. As previously described, these items were developed according to our TOC framework, which specified outcome areas of interest related to the effectiveness of the overarching Fire Exchange initiative. Considerations of face validity, missing data per item, and number and commonality of items between practitioner and scientist scales helped finalize our initial scale groupings. Skewness and kurtosis were examined to ensure that item distributions were reasonably balanced (Clark and Watson 1995). A total of five distinct scales were developed: (1) practitioners’ use of fire science and perceived relevance for their work, (2) practitioners’ respect for scientists, (3) scientists’ respect for practitioners, (4) Exchanges as necessary organizations for disseminating trusted and relevant fire science, and (5) Exchange website utility and value in providing fire science. Next, we conducted a Confirmatory Factor Analysis (CFA) to test the underlying dimensions of each of these scales, retaining items with factor loadings greater than .60. This process led to dropping one or two items per proposed scale. We then ran skew and kurtosis analytics on each scale. As expected, creating scales improved the distribution of scores. Finally, Cronbach’s coefficient alphas (Cronbach’s α) were run on each scale to estimate the internal consistency of questions as a measurement of reliability (Carmines and Zeller 1979). Below is a description of each of the final scales used for analyses. Practitioners’ Use of Fire Science and Perceived Relevance for Their Work Practitioners responded to four items developed to measure the extent to which they value fire science and use fire science on the job. The scale included the items I trust fire science research findings, Using fire science information enhances my effectiveness on the job, I often draw upon fire science research when making work-related decisions, and During the past year, I have changed at least one thing in my work based on what I’ve learned about fire science. These items were implemented in the first year of the survey. Cronbach’s α for this scale was 0.70. Practitioners’ Respect for Scientists Practitioners answered five questions that comprised a scale developed to measure practitioners’ respect for scientists. Three of the items included in this scale were positively framed statements: Fire scientists are willing to work directly with me if I have questions about research or how to apply fire science at my job, Fire science researchers/scientists are easy to approach, and Fire scientists value my knowledge and experience as a field professional. Two reverse-coded questions included the statements Fire scientists rarely provide information that helps me address the management problems I face and Fire scientists are reluctant to study problems and issues suggested by local managers/ practitioners. These items were implemented in the first year of the survey. Cronbach’s α for this scale was 0.73. Scientists’ Respect for Practitioners Scientists answered five questions that comprised a scale developed similarly to measure scientists’ respect for practitioners. Three of the questions included positive statements: I often present or publish fire science information for manager/practitioner audiences, Managers/practitioners value my knowledge and experience as a fire scientist, and Interacting with managers/practitioners enhances my effectiveness on the job. Two reverse-coded questions included the statements I am sometimes hesitant to study problems and issues suggested by local managers/practitioners and I prefer that my research be focused on theoretical issues, rather than on applied management problems. These items were included in the first year of the survey. Cronbach’s α for this scale was 0.71. Exchanges as Necessary Organizations for Spreading Trusted and Relevant Fire Science This scale was implemented with both practitioners and scientists. Five questions were designed to measure respondents’ overall experiences with Exchanges and their perceived impact on increasing accessibility and application of trusted and relevant fire science in their respective region. The scale included items The Exchange has helped improve the accessibility of fire science information in my region, The Exchange is needed to help coordinate sharing of fire science information in my region, The Exchange has helped improve communication among fire managers/practitioners and fire researchers/scientists in my region, I would recommend Exchange involvement to my co-workers, and The Exchange has improved the use and application of fire science in my region. These items were implemented in the second year of the survey. Therefore, data exist for all Exchanges following their second funding year. For practitioners, Cronbach’s α for this scale was 0.84. For scientists, Cronbach’s α for this scale was 0.88. Exchange Websites’ Utility and Value Providing Fire Science This scale was implemented with both practitioners and scientists. Participants answered four questions developed to measure perceptions of their regional Exchange’s website. The scale included questions such as My Exchange’s website provides practical information I can use in my job, My Exchange’s website provides a wide variety of fire science information, My Exchange’s website provides information that is current and up-to-date, and My Exchange’s website organizes the information I need in one convenient place. These items were implemented in the second year of the survey. Therefore, data exist for all Exchanges after their second funding year. For practitioners, Cronbach’s α for this scale was 0.74. For scientists, Cronbach’s α for this scale was 0.77. Results Mean responses for scales were compared by Exchanges’ year of establishment to determine if responses to scales significantly changed between two time points. Tables 1 and 2 illustrate mean scores, standard deviations, skew, and kurtosis for scales by Exchange year of practitioners and scientists. As noted earlier, prior to analyses, variables were tested for issues of skew, kurtosis, multicollinearity, and homogeneity of variance, and were found to meet the assumptions for significance testing (Clark and Watson, 1995, Tabachnick and Fidell 2013). Specifically, all skews were less than 1 and all measures of kurtosis were less than 2. Difference of means testing, or independent t-tests, were conducted between scale means of respondents affiliated with Exchanges in their first or second year and means of respondents affiliated with Exchanges in their fourth year. Levene’s test confirmed that all testing pairs met the assumption of homogeneity of variance. Table 1. Means, standard deviations, skew, kurtosis, and p-values from t-tests for practitioner scales for years, 1, 2, and 4. JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 View Large Table 1. Means, standard deviations, skew, kurtosis, and p-values from t-tests for practitioner scales for years, 1, 2, and 4. JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 View Large Table 2. Means, standard deviations, skew, kurtosis, and p-values from t-tests for scientist scales for years, 1, 2, and 4. JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 View Large Table 2. Means, standard deviations, skew, kurtosis, and p-values from t-tests for scientist scales for years, 1, 2, and 4. JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 View Large Difference of Means Tests Practitioner Results Analyses revealed over time that practitioners’ reported use of fire science and perceived respect for scientists became more favorable. That is, practitioners were much more likely to report that fire science enhanced their work and they increased use of fire science on the job in year four (M = 3.81, SD = 0.54) when compared to year one (M = 3.71, SD = 0.57); t(1433) = 2.87, p < 0.01. Similarly, practitioners were significantly more likely to rate that scientists listened to their input and that scientists provided useful information in year four (M = 3.46, SD = 0.57) when compared to year one (M = 3.33, SD = 0.55); t(1419) = 3.97, p < 0.001 (see Table 1). Questions designed to measure experiences with Exchanges and Exchange websites were not added until the second year of the online survey. Therefore, year two and year four comparisons were conducted for these variables. Again, practitioners were significantly more likely to rate their Exchange as necessary for spreading trusted and relevant fire science in year four (M = 4.07, SD = 0.62) when compared to year two (M = 3.81, SD = 0.58); t(722) = 5.71, p < 0.001. Also, practitioners were significantly more likely to rate their Exchange websites’ utility and value positively in year four (M = 3.71, SD = 0.54) when compared to year two (M = 3.61, SD = 0.50); t(659) = 2.56, p < 0.05 (see Table 1). Scientist Results There were no statistically significant changes in mean scores for the scale Scientists’ respect for practitioners because scientists’ ratings were highly favorable in year one (M = 4.22, SD = 0.46) and remained highly favorable in year four (M = 4.20, SD = 0.49). For the other two scales (Exchanges as necessary organizations for spreading trusted and relevant fire science and Exchange websites’ utility and value providing fire science), responses significantly improved from year two to year four. Specifically, respondents were much more likely to endorse that their Exchange improved the dissemination and application of fire science in year four (M = 4.19, SD = 0.56) when compared to year two (M = 3.90, SD = 0.62); t(196) = 3.41, p < 0.001. Similarly, respondents had significantly more positive ratings of their regional Exchange website and its positive impact on their work in year four (M = 3.73, SD = 0.62) when compared to year two (M = 3.51, SD = 0.62); t(176) = 2.39, p < 0.05 (see Table 2). Discussion Large-scale boundary organization initiatives are complex networks that feature multiple program sites in varied developmental stages, tasked with translating science-based research into applicable information (Sung et al. 2003; Westfall et al. 2007; Wooten et al. 2013). It is essential to the success of such initiatives to strategically and routinely assess progress toward desired goals and outcomes for the purpose of targeting improvements. In response to findings that increased production of applicable fire science did not result in increased use of fire science among practitioners, JFSP established the Fire Exchange initiative (Kocher et al. 2012). Exchanges were tasked with improving the dissemination and the application of translated fire science. To meet this goal, Exchange personnel recognized that it was essential to establish Exchanges as reliable sources of relevant fire science as well as to increase positive interactions and improve relationships between practitioners and scientists. As scientists and practitioners increase mutual respect for one another, scientists are more likely to engage in research relevant to practitioners, and practitioners are more likely to adopt and apply these research findings (McNie 2007, Laurance et al. 2012, Hulme 2014, Schwandt 2014). Therefore, when planning programming to bring these professional groups together, Exchange personnel incorporated the perspectives of both the scientists producing fire science research and the practitioners using that research in real-world contexts. To evaluate the activities of the initiative, we constructed a national logic model based on our TOC. This logic model mapped how Exchanges’ shared activities would achieve their overarching goal of bridging the gap between fire science research and practice in order to eventually change environmental conditions. According to the logic model, necessary prerequisite steps included: increasing perceptions that Exchanges are valuable and trusted sources of fire science, increasing mutual respect between scientists and practitioners, and increasing practitioners’ use of fire science in their jobs. The results of five cumulative years of evaluative research of Exchanges’ outcomes and processes indicate that Exchanges are making progress toward reaching short- and medium-term goals necessary to achieve long-lasting environmental change. For practitioners in particular, these impacts are demonstrated by improvements at all levels of Exchange efforts, including improvements in attitudes concerning the value of Exchanges, Exchange websites (Exchanges’ primary mechanism for reaching practitioners), practitioner reports of increased relevance of fire science and its use on the job, and practitioner reports that scientists respect their input on research agendas. Improvements in perceptions of Exchanges and Exchange websites indicate that these organizations increasingly meet constituents’ needs and are viewed as valuable and trusted sources of fire science information. Furthermore, these findings are supported by coincident research that reveals Exchanges are viewed as credible sources of translated fire science and that practitioners are using Exchange products to make decisions on the job (Hunter 2016). That practitioners increasingly endorse the use and application of Exchange fire science in their professional work is an important finding. This indicator of behavioral change, as a result of Exchange resources and activities, often is difficult to achieve and yet essential to achieving long-term changes in conditions (Patton 2014). The current findings, reflected as modest but statistically significant increases in mean scores for reported fire science use among practitioners, reveal that Exchanges are taking the necessary steps to achieve their long-term goals. Improvements in practitioners’ attitudes toward scientists are particularly important in demonstrating increases in mutual respect between practitioners and scientists, which is integral for fire science adoption (McNie 2007). During the first and second years of the evaluation, scientists rated themselves as highly approachable whereas practitioners rated scientists as less approachable (Sicafuse et al. 2012). Qualitative data collected in interviews with Exchange personnel in 2012 revealed that professional communication and trust could be improved between fire practitioners and scientists (Sicafuse et al. 2013, Maletsky et al. 2017). Based on these findings, Exchange personnel sought to pay special attention to improving practitioner and scientist interactions. The longitudinal results reported here suggest that their efforts have been successful in improving practitioners’ perceptions that scientists value their expertise. Additionally, supplemental qualitative data from the annually administered surveys serve to further verify these successes. As one practitioner stated, “The fire science [Exchange] has been a fantastic resource. It’s great to see the communication between [practitioners] and scientists that was almost non-existent when I began my career two decades ago.” According to another practitioner, “The Exchanges play critical roles in developing and maintaining two-way communication between fire scientists and practitioners and deserve continued funding and support through the JFSP.” Finally, as one practitioner reflected, “The highest value of the Fire Science Exchange is the Exchange itself—[that is,] when the scientist, managers and practitioners come together to solve fire challenges where the ‘answers’ are not clear. The more the Exchanges focus on springing forward using the best from traditional western science, traditional ecological knowledge, and evidence based practice and learning, the more likely we are able to advance.” Similarly, for scientists, ratings have increased over time concerning Exchanges’ impacts in their regions and value of Exchange websites. Scientists’ perceptions of practitioners were very positive initially (at the upper end of the scale) and have remained positive over time. This result is not surprising, as many scientists likely became involved with Exchanges to disseminate their research findings and improve the application of fire science research in the field. Exchanges also may attract scientists who are more interested in collaborating with practitioners and pursuing applied science research. Increases in ratings of the Exchanges overall and Exchange websites in particular indicate that scientists also benefit from Exchanges as a forum for sharing their research, identifying new research topics, and connecting with other fire science professionals. The finding that scientists rate Exchanges as valuable in the fire science community is necessary to ensure that scientists remain engaged and willing to receive constructive input and feedback from practitioners (Guston 2001, McNie 2007). Beyond the routine cautions concerning self-reported data, there are limitations to the present study. For example, although Exchanges actively recruited participants during their scheduled survey administration year, the snowball sampling strategy may have led to participation of individuals from off-year neighboring Exchanges. Additionally, although responses were categorized by Exchanges’ establishment year to examine changes in responses over time, individual responses were not tracked. Thus, multiple survey waves likely included both new and returning participants. This precluded the use of more complex statistical techniques, such as time-series analyses. Finally, participants began their interaction with Exchanges with somewhat positive attitudes and behaviors, thus restricting Evaluators’ ability to isolate the effect of Exchanges to positively influence participants through research-related resources and activities. Nevertheless, general increases on TOC variables of interest over time highlight improvements in constituents’ value of Exchange activities and products, as well as improvements in practitioners’ perceptions of scientists and their adoption of fire science. These findings support the longitudinal trends reflected in the data and are congruent with contextual qualitative data collected as part of the overall evaluation effort (Maletsky et al. 2017). Together, these results provide evidence that the magnitudes of change we report are meaningful and aligned with the TOC as illustrated in the Exchange logic model. Response rates to the annual evaluative surveys have varied over time, with more individuals participating in the first survey than in later surveys. In discussions with Fire Science Exchange Network leadership, survey fatigue and saturation issues were highlighted as concerns and the most likely reasons for survey completion decline in later years. Subsequent efforts to address these issues appear to have reversed this trend. These have included decreasing survey length and expanding constituent list-servs. These changes do not impact the analyses reported here but will help improve data collection and subsequent analyses moving forward. Finally, the data reported here are based on individuals familiar with Exchanges. Therefore, nothing definitive can be said about how Exchange participants differ from non-participants in terms of isolating the effects of Exchanges to improve fire science research and application. Current results, however, provide evidence that Exchanges are fulfilling their roles as boundary organizations and effectively disseminating translated fire science. These findings also are supported by results from separate but concurrent evaluation efforts (Hunter 2016). Overall, evaluative research to date indicates that Exchanges are acting successfully as boundary organizations, convening diverse professionals and furthering the adoption of fire science in the field. These are only first steps, and additional evaluation is needed to assess whether these gains translate to positive change in land management and environmental conditions. To this end, the evaluation of the JFSP Fire Science Exchange Network has shifted its focus from measuring short-term goals (e.g., Exchange awareness and cross-professional perceptions) to medium- and long-term goals (e.g., fire science usage, policy change, and change on the ground). The results reported here support the likely success of the Fire Science Exchange Network’s strategies to increase educational outreach, professional relationship-building, and fire science dissemination outcomes. Hence, these Exchanges can provide a roadmap to other boundary organizations that aim to enhance cross-professional communication to increase mutual respect among professionals and increase the application of scientific research in the field. Acknowledgments The authors would like to thank the Joint Fire Science Program (JFSP) Fire Science Exchange Network leadership and staff for overall evaluation support and data collection efforts, with special thanks to John Cissel, who served as JFSP Director at the time this article was written. Literature Cited Ascher , W. , T. Steelman , and R. Healy . 2010 . Knowledge and environmental policy: Re-imagining the boundaries between science and politics . MIT Press , Cambridge, MA . Calhoun , W.J. , K. Wooten , S. Bhavnani , K.E. Anderson , J. Freeman , and A.R. Brasier . 2013 . The CTSA as an exemplar framework for developing multidisciplinary translational teams . Clinical and Translational Science 6 : 60 – 71 . Google Scholar CrossRef Search ADS PubMed Carmines , E.G. , and R.A. Zeller . 1979 . Reliability and validity assessment . SAGE Publications , Thousand Oaks, CA . Google Scholar CrossRef Search ADS Cerveny , L.K. , and C.M. Ryan . 2008 . Agency capacity for recreation science and management: The case of the U.S. Forest Service . USDA For. Serv. Gen. Tech. Rep. PNW-GTR-757 . Clark , L.A. , and D. Watson . 1995 . Construction validity: Basic issues in objective scale development . Psychological Assessment 7 : 309 – 319 . doi: 10.1037/1040-3590.7.3.309 Google Scholar CrossRef Search ADS Dillman , D.A. , J.D. Smyth , and L.M. Christian . 2010 . Internet, phone, mail, and mixed-mode surveys: The tailored design method , 4th ed . Willey , Hoboken, NJ . Guston , D.H . 2001 . Boundary organizations in environmental policy and science: An introduction . Science, Technology, & Human Values 26 ( 4 ): 399 – 408 . Google Scholar CrossRef Search ADS Hall , J.A. , and E. Fleishman . 2010 . Demonstration as a means to translate conservation science into practice . Conservation Biology 24 ( 1 ): 120 – 127 . Google Scholar CrossRef Search ADS PubMed Hulme , P.E . 2014 . Bridging the knowing-doing gap: Know-who, know-what, know-why, know-how and know-when . Journal of Applied Ecology 51 : 1131 – 1136 . Google Scholar CrossRef Search ADS Hunter , M.E . 2016 . Outcomes of fire research: Is science used ? International Journal of Wildland Fire 25 ( 5 ): 495 – 504 . Google Scholar CrossRef Search ADS Joint Fire Science Program (JFSP) . 2011 . Knowledge exchange: A two-way street . Fire Science Digest 1 ( 11 ): 1 – 16 . Kocher , S.D. , E. Toman , S.F. Trainor , V. Wright , J.S. Briggs , C.P. Goebel , E.M. MontBlanc , et al. 2012 . How can we span the boundaries between wildland fire science and management in the United States ? Journal of Forestry 110 ( 8 ): 421 – 428 . Google Scholar CrossRef Search ADS Laurance , W.F. , H. Koster , M. Grooter , A.B. Anderso , P.A. Zuidema , S. Zwick , R.J. Zagt , et al. 2012 . Making conservation research more relevant for conservation practitioners . Biological Conservation 153 : 164 – 168 . Google Scholar CrossRef Search ADS Lemos , M.C. , and B.J. Morehouse . 2005 . The co-production of science and policy in integrated climate assessments . Global Environmental Change 15 ( 1 ): 57 – 68 . Google Scholar CrossRef Search ADS Maletsky , L. , W. Evans , L. Singletary , and L. Sicafuse . 2017 . Joint Fire Science Program Fire Science Exchange Network best practices . Working Paper, University of Nevada , Reno . McNie , E.C . 2007 . Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature . Environ Sci Policy 10 : 17 – 38 . Google Scholar CrossRef Search ADS National Institute of Health [NIH] . 2004 . NIH roadmap: Re-engineering the clinical research enterprise regional translational research centers interim report . Available online at https://commonfund.nih.gov/sites/default/files/rtrc_interimreport.pdf Patton , M.Q . 2014 . What brain sciences reveal about integrating theory and practice . Am. J. Eval . 35 : 237 – 244 . Google Scholar CrossRef Search ADS Schwandt , T.M . 2014 . On the mutually informing relationship between practice and theory in evaluation . Am. J. Eval . 35 : 231 – 236 . Google Scholar CrossRef Search ADS Sicafuse , L. , W. Evans , and L. Singletary . 2012 . Joint Fire Science Program regional consortia 2012 evaluation report: A national cluster evaluation of consortia process and impacts . Available online at the Joint Fire Science Program website: https://www.firescience.gov/documents/2012_JFSP_FSEN_Evaluation_Report.pdf Sicafuse , L. , W. Evans , L. Singletary , and L. Maletsky . 2013 . Interviews with joint fire science program consortia leadership and staff . Available online at the Joint Fire Science Program website: http://www.firescience.gov/documents/JFSP_Consortia_Interviews_report.pdf Stufflebeam , D . 2001 . Evaluation models . New Directions for Evaluation 89 : 7 – 98 . Google Scholar CrossRef Search ADS Sung , N.S. , W.F.J. Crowley , M. Genel , P. Salber , L. Sandy , and L.M. Sherwood . 2003 . Central challenges facing the national clinical research enterprise . JAMA 289 : 1278 – 1287 . Google Scholar CrossRef Search ADS PubMed Tabachnick , B.G. , and L.S. Fidell . 2013 . Using multivariate statistics , 6th ed . Pearson Education , Washington, DC . Weiss , C . 1997 . How can theory-based evaluation make greater headway ? Evaluation Review 21 : 501 – 524 . Google Scholar CrossRef Search ADS Westfall , J.M. , J. Mold , and L. Fagan . 2007 . Practice-based research: “Blue highways” on the NIH roadmap . JAMA 297 : 403 – 406 . Google Scholar CrossRef Search ADS PubMed Wethington , E. , and R.E. Dunifon . 2012 . Research for the public good: Applying the methods of translational research to improve human health and well-being . American Psychological Association , Washington, DC . Google Scholar CrossRef Search ADS White , D.D. , A. Wutich , K.L. Larson , P. Gober , T. Lant , and C. Senneville . 2010 . Credibility, salience, and legitimacy of boundary objects: Water managers’ assessment of a simulation model in an immersive decision theater . Science and Public Policy 37 ( 3 ): 219 – 232 . Google Scholar CrossRef Search ADS Wooten , K.C. , R.M. Rose , G.V. Ostir , W.J. Calhoun , B.T. Ameredes , and A.R. Brasier . 2013 . Assessing and evaluating multidisciplinary translational teams: A mixed methods approach . Evaluation and the Health Professions 37 ( 1 ): 33 – 49 . Google Scholar CrossRef Search ADS PubMed Zerhouni , E.A . 2007 . Translational research: Moving discovery to practice . Clinical Pharmacology & Therapeutics 81 : 126 – 128 . doi: 10.1038/sj.clpt.6100029 Google Scholar CrossRef Search ADS © 2018 Society of American Foresters This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Forestry Oxford University Press

Joint Fire Science Program (JFSP) Fire Science Exchange Network: A National Evaluation of Initiative Impacts

Journal of Forestry , Volume Advance Article (4) – May 22, 2018
Download PDF
8 pages

Loading next page...
 
/lp/ou_press/joint-fire-science-program-jfsp-fire-science-exchange-network-a-e08I0FYaLN

References (30)

Copyright
© 2018 Society of American Foresters
ISSN
0022-1201
eISSN
1938-3746
DOI
10.1093/jofore/fvy009
Publisher site
See Article on Publisher Site

Abstract

Abstract The Joint Fire Science Program (JFSP) Fire Science Exchange Network is composed of 15 Exchanges that act as boundary organizations tasked with improving fire science use within their respective regions. A longitudinal survey conducted annually between 2011 and 2015 as part of a larger evaluation effort reveals significant impacts. Results indicate that fire science practitioners who are constituents of Exchanges increased their agreement that scientists respect their experience and input, their regional Exchange improves access to fire science, and they use fire science in their work. Additionally, scientists agreed increasingly that Exchanges are necessary to share fire science information and improve the application of fire science in their region. These positive findings suggest that Exchanges’ boundary organizational activities impact the adoption of fire science research in the field and that Exchanges are fulfilling their role in disseminating and increasing the use of fire science. Boundary organizations, logic model, translational science Management and Policy Implications The Joint Fire Science Program Fire Science Exchange Network is composed of 15 Exchanges across the United States that act as boundary organizations to increase the dissemination and use of fire science. Three key implications for management and policy are highlighted: the logic model framework for program planning and evaluation, the importance of relationship-building between practitioners and scientists, and the use of a variety of educational outreach methods and materials. First, a comprehensive logic model of the Exchange initiative as well as individual Exchange logic models were developed. These models provided structure for prioritizing activities by directly linking them to short-, medium-, and long-term objectives that needed to occur in succession. Thus, logic models helped identify strategies to reach desired changes. Second, Exchanges recognized that rather than being passive recipients of research findings, practitioners needed to communicate to scientists their current and future management needs. Providing opportunities for these professionals to interact improved relationships, practitioners’ perceptions that their experiences were valued, and increased fire science adoption. Finally, the use of a variety of outreach products allowed Exchanges to reach wider audiences with differing learning styles, professional roles, and time constraints. Exchanges were able to provide in-person activities as well as harness emerging social media technologies, maintaining a coherent message across educational outreach platforms. Future outreach initiatives should continue to use a variety of educational strategies to foster interpersonal connections, ensuring that a cohesive strategy (outlined in a logic model) guides implementation. The need to link federally funded research with real-world application and impact has led to an increase in translational science initiatives (National Institute of Health [NIH] 2004, Zerhouni 2007, Hall and Fleishman 2010, Wethington and Dunifon 2012, Calhoun et al. 2013). Broadly, translational science is defined as the transformation of scientific research findings into applied products and procedures for on-the-ground use (Sung et al. 2003, Westfall et al. 2007, Wooten et al. 2013). In the past decade, natural resource and conservation fields have identified gaps between research generated by scientists and the application of that research among practitioners on the job (Laurance et al. 2012, Hulme 2014). Although there is a need to translate and communicate fire science research results for practitioners to encourage greater use of research findings in the field (Cerveny and Ryan 2008, Ascher et al. 2010), the co-production of knowledge whereby practitioners inform the research process is necessary to effectively bridge the gap between fire science production and application (Lemos and Morehouse 2005, White et al. 2010). Therefore, strategies to increase the use of scientific research have sought to improve relationships between scientists and practitioners, specifically through increasing mutual respect while emphasizing practitioners’ information needs and field experiences (McNie 2007, Laurance et al. 2012, Hulme 2014, Patton 2014, Schwandt 2014). The US Department of the Interior’s Joint Fire Science Program (JFSP) was developed in 1998 to promote applied fire science research. After 10 years, however, the JFSP Board realized that the proliferation of fire science research did not necessarily increase the application of research results on the ground (Kocher et al. 2012). In 2009, JFSP conducted a needs assessment to discover why fire science availability alone was not enough to increase the use of fire science among practitioners. The needs assessment found that the primary barriers to fire science application included lack of practitioners’ access to relevant and practical information as well as communication barriers between scientists and practitioners (Kocher et al. 2012). Therefore, the JFSP funded a nationwide network of fire science information Exchanges based on a boundary organization approach. Boundary organizations work to break down demarcations between traditionally siloed research and practitioner communities so that science research outputs become more relevant (Guston 2001). Through increasing mutual respect and interaction between these professional communities, scientists direct their research focus toward practitioners’ information needs. Practitioners, seeing that scientists value their input, increase their trust and use of the resulting science. Thus, improved relationships are necessary to increase the co-production of knowledge and subsequent adoption of practices derived from that communal knowledge (Guston 2001, McNie 2007). As Exchanges increase accessibility to translated fire science research products, they increase interaction between scientists and practitioners, which serves to improve relationships and enhances mutual respect and collaboration (JFSP 2011). To date, the JFSP Fire Science Exchange Network includes 15 Exchanges distributed across the United States. An evaluation of Exchange activities between 2011 and 2015 measured the efficacy of these organizations to improve fire science accessibility, science use and application, and relationships between practitioners and scientists. The following article reports the survey findings of these evaluation efforts. The Evaluation of the JFSP Exchanges In 2010, an independent team of investigators launched an evaluation of the JFSP Fire Science Exchange Network’s outcomes and impacts, based on Theory of Change (TOC) using a logic model framework. The TOC posits that a logical progression exists for achieving outcomes and that programs should work backwards from their overarching goals to determine intermediate and beginning prerequisite steps (Patton 2014, Stufflebeam 2001, Weiss 1997). Thus, TOC is concerned with the causal modeling of outcomes, paying particular attention to the progressive outcomes that build on one another. The logic model is a practical tool for mapping how a program’s activities translate into the short-, medium-, and long-term outcomes to achieve its larger goals (Patton 2014). The TOC and logic model define short-term outcomes as the first steps necessary for change and involve factors such as increased awareness of an issue, knowledge gains, and changes in attitudes. Medium-term outcomes involve changes in behavior and necessitate that short-term outcomes have first been achieved. Changes in behavior indicate a higher level of success that is more difficult to achieve. Similarly, long-term outcomes often reflect ultimate organizational and programmatic goals, such as impacting environmental conditions. The evaluation of Exchanges involved development of a national logic model to map outcomes relevant to the entire initiative, which served to direct strategies and identify indicators for measuring program outcomes and impacts over time (see Figure 1). Figure 1. View largeDownload slide National logic model for JFSP fire science exchange network. Figure 1. View largeDownload slide National logic model for JFSP fire science exchange network. The JFSP Fire Science Exchange Network experienced growth over the course of the evaluation. When the national evaluation began in 2010, the JFSP had funded eight Exchanges that varied in terms of program development and implementation of outreach activities and materials. Between 2010 and 2013, JFSP funded six additional Exchanges, which further emphasized developmental differences among the Exchanges. In 2015, an additional Exchange was added in the northeast region of the United States. A suite of evaluation activities was designed and implemented to track the JFSP initiative’s progress. One of four primary evaluation activities included an annual survey to measure Exchange users’ perceptions of overall outcomes and impacts. Exchanges were required to participate in the survey every two to three years in alignment with their funding cycles. This schedule reduced the burden on Exchanges to participate every year as well as incorporated flexibility as new Exchanges entered the initiative. A national evaluation of Exchanges was possible because each Exchange shared common goals and sought to reach those goals through similar outreach activities. For example, in addition to social media accounts, each Exchange maintained an online presence by hosting a website that provided translated fire science products and educational webinars. Also, each Exchange conducted in-person educational events that included workshops, field tours, field demonstrations, and science-based presentations at conferences and practitioner meetings. The ongoing evaluation focused on how practitioners rated their experiences with their regional Exchange and with fire scientists, and conversely, how scientists rated their experiences with their Exchange and with practitioners. This paper highlights the longitudinal quantitative results from the annual national Fire Exchange evaluative survey. Method An annual survey was developed in 2010 and administered online over the course of five years (2011–2015). The survey was designed to evaluate Exchange constituents’ experiences with their regional Exchange. Experiences included the Exchange website, educational events and products, as well as experiences with other fire science professionals. Since Exchanges are required to participate only during their funding year, for each year the survey was administered, participants represented Exchanges established at different points in time. To account for this variation in establishment, data were separated based on the Exchanges’ year of funding as an indicator of developmental maturity. It was assumed that Exchanges would need time to impact their constituents. Differentiating Exchanges by year of establishment allowed for analyses that compared responses of survey participants affiliated with Exchanges in their first and second years of funding (e.g., Exchanges’ early establishment period) with responses from their fourth year of funding. At the time of these analyses, all but one Exchange had been established at least four years and therefore that Exchange was omitted from this study. The analyses reported here represent data collected from 14 established Exchanges across their initial four years of funding. Participants and Recruitment A total of 3283 participants comprised the combined dataset that spans all survey waves. Of the total participants, 2625 were practitioners and 658 were scientists. The majority of participants in the current sample were male (72.6%). When examining race and ethnicity, the majority of participants were White/Caucasian (90.0%), followed by “Other” (3.2%), Hispanic/Latino (1.9%), Native American/Alaskan native (1.5%), Multi-ethnic (1.4%), Asian/Pacific Islander (1.3%), and Black/African American (0.7%). Average age of participants was 46.6 years, with a standard deviation of 11.1 years. Participants had worked in their field on average for 17.8 years with a standard deviation of 10.7 years. These demographics were proportionally consistent throughout all survey years. The survey design was tailored to target Exchanges’ two primary audiences: practitioners, who consume fire science for professional or land management intentions, and scientists, who are the producers of fire science research. During a survey implementation year, Exchanges recruited participants from their list of members via a snowball sampling strategy with three recruitment stages (Dillman et al. 2010). The initial survey invitation informed participants of the purpose of the survey and provided links to the survey, hosted by Survey Monkey™. These links could be forwarded via email to additional potential survey participants. Two additional invitations to participate were sent at two- and four-week intervals, respectively, following the initial invitation. Measures Survey questions were developed and finalized through collaborative efforts between the university-based independent evaluation team, Exchange personnel, and members of the JFSP Board. In 2010, evaluators examined the approved funding proposals submitted by the first eight Exchanges to identify common proposed activities and desired outcomes. Evaluators created a first draft of survey questions. Multiple revisions of question items were conducted in collaboration with Exchange personnel and representatives of the JFSP Board until a final draft of the survey was approved. Prior to launching the survey, all items were mapped onto the national logic model as a final check to verify alignment and coverage of national outcomes. As minor revisions to the survey occurred over time, a core set of questions remained consistent across annual survey administrations to allow for longitudinal analyses. Institutional Review Board approval was obtained prior to recruiting participants for the initial and each subsequent year of data collection. The survey targeted practitioners and scientists. Participants who indicated that their work involved applying fire science in the field were directed to the practitioner survey, whereas participants indicating they were fire science researchers were directed to the scientist survey. Both surveys featured parallel questions about experiences with Exchanges and the Exchange websites. Additionally, each group was asked about their perceptions of one another. For example, practitioners were asked about their experiences working with fire scientists, whereas scientists were asked about their experiences working with practitioners. Skip patterns were integrated into the survey so that participants without knowledge of their Exchange or who had never visited their Exchange website were redirected from topic-specific questions. Most survey participants (70% for all survey years) were familiar with their regional Exchange and Exchange website. All questions were measured on a 5-point Likert-type scale where 1 = Strongly Disagree and 5 = Strongly Agree. Scales Data from all survey years were combined to facilitate analyses by funding year. Respondents were categorized as affiliates of Exchanges based on their regional Exchange’s first and fourth years of funding. We developed post hoc the scales described here from existing survey questions. As previously described, these items were developed according to our TOC framework, which specified outcome areas of interest related to the effectiveness of the overarching Fire Exchange initiative. Considerations of face validity, missing data per item, and number and commonality of items between practitioner and scientist scales helped finalize our initial scale groupings. Skewness and kurtosis were examined to ensure that item distributions were reasonably balanced (Clark and Watson 1995). A total of five distinct scales were developed: (1) practitioners’ use of fire science and perceived relevance for their work, (2) practitioners’ respect for scientists, (3) scientists’ respect for practitioners, (4) Exchanges as necessary organizations for disseminating trusted and relevant fire science, and (5) Exchange website utility and value in providing fire science. Next, we conducted a Confirmatory Factor Analysis (CFA) to test the underlying dimensions of each of these scales, retaining items with factor loadings greater than .60. This process led to dropping one or two items per proposed scale. We then ran skew and kurtosis analytics on each scale. As expected, creating scales improved the distribution of scores. Finally, Cronbach’s coefficient alphas (Cronbach’s α) were run on each scale to estimate the internal consistency of questions as a measurement of reliability (Carmines and Zeller 1979). Below is a description of each of the final scales used for analyses. Practitioners’ Use of Fire Science and Perceived Relevance for Their Work Practitioners responded to four items developed to measure the extent to which they value fire science and use fire science on the job. The scale included the items I trust fire science research findings, Using fire science information enhances my effectiveness on the job, I often draw upon fire science research when making work-related decisions, and During the past year, I have changed at least one thing in my work based on what I’ve learned about fire science. These items were implemented in the first year of the survey. Cronbach’s α for this scale was 0.70. Practitioners’ Respect for Scientists Practitioners answered five questions that comprised a scale developed to measure practitioners’ respect for scientists. Three of the items included in this scale were positively framed statements: Fire scientists are willing to work directly with me if I have questions about research or how to apply fire science at my job, Fire science researchers/scientists are easy to approach, and Fire scientists value my knowledge and experience as a field professional. Two reverse-coded questions included the statements Fire scientists rarely provide information that helps me address the management problems I face and Fire scientists are reluctant to study problems and issues suggested by local managers/ practitioners. These items were implemented in the first year of the survey. Cronbach’s α for this scale was 0.73. Scientists’ Respect for Practitioners Scientists answered five questions that comprised a scale developed similarly to measure scientists’ respect for practitioners. Three of the questions included positive statements: I often present or publish fire science information for manager/practitioner audiences, Managers/practitioners value my knowledge and experience as a fire scientist, and Interacting with managers/practitioners enhances my effectiveness on the job. Two reverse-coded questions included the statements I am sometimes hesitant to study problems and issues suggested by local managers/practitioners and I prefer that my research be focused on theoretical issues, rather than on applied management problems. These items were included in the first year of the survey. Cronbach’s α for this scale was 0.71. Exchanges as Necessary Organizations for Spreading Trusted and Relevant Fire Science This scale was implemented with both practitioners and scientists. Five questions were designed to measure respondents’ overall experiences with Exchanges and their perceived impact on increasing accessibility and application of trusted and relevant fire science in their respective region. The scale included items The Exchange has helped improve the accessibility of fire science information in my region, The Exchange is needed to help coordinate sharing of fire science information in my region, The Exchange has helped improve communication among fire managers/practitioners and fire researchers/scientists in my region, I would recommend Exchange involvement to my co-workers, and The Exchange has improved the use and application of fire science in my region. These items were implemented in the second year of the survey. Therefore, data exist for all Exchanges following their second funding year. For practitioners, Cronbach’s α for this scale was 0.84. For scientists, Cronbach’s α for this scale was 0.88. Exchange Websites’ Utility and Value Providing Fire Science This scale was implemented with both practitioners and scientists. Participants answered four questions developed to measure perceptions of their regional Exchange’s website. The scale included questions such as My Exchange’s website provides practical information I can use in my job, My Exchange’s website provides a wide variety of fire science information, My Exchange’s website provides information that is current and up-to-date, and My Exchange’s website organizes the information I need in one convenient place. These items were implemented in the second year of the survey. Therefore, data exist for all Exchanges after their second funding year. For practitioners, Cronbach’s α for this scale was 0.74. For scientists, Cronbach’s α for this scale was 0.77. Results Mean responses for scales were compared by Exchanges’ year of establishment to determine if responses to scales significantly changed between two time points. Tables 1 and 2 illustrate mean scores, standard deviations, skew, and kurtosis for scales by Exchange year of practitioners and scientists. As noted earlier, prior to analyses, variables were tested for issues of skew, kurtosis, multicollinearity, and homogeneity of variance, and were found to meet the assumptions for significance testing (Clark and Watson, 1995, Tabachnick and Fidell 2013). Specifically, all skews were less than 1 and all measures of kurtosis were less than 2. Difference of means testing, or independent t-tests, were conducted between scale means of respondents affiliated with Exchanges in their first or second year and means of respondents affiliated with Exchanges in their fourth year. Levene’s test confirmed that all testing pairs met the assumption of homogeneity of variance. Table 1. Means, standard deviations, skew, kurtosis, and p-values from t-tests for practitioner scales for years, 1, 2, and 4. JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 View Large Table 1. Means, standard deviations, skew, kurtosis, and p-values from t-tests for practitioner scales for years, 1, 2, and 4. JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 JFSP practitioners Year 1 Year 4 Scale Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Fire science use on the job 3.71 (0.57) –.38 .69 1078 3.81 (0.54) –.61 .78 357 <0.01 Respect of scientists 3.33 (0.55) –.24 .38 1062 3.46 (0.57) –.15 1.06 359 <0.001 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.81 (0.58) –.13 .01 439 4.07 (0.62) –.50 .36 285 <0.001 Exchange websites as valuable 3.61 (0.50) –.23 .73 384 3.71 (0.54) –.31 1.04 277 <0.05 View Large Table 2. Means, standard deviations, skew, kurtosis, and p-values from t-tests for scientist scales for years, 1, 2, and 4. JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 View Large Table 2. Means, standard deviations, skew, kurtosis, and p-values from t-tests for scientist scales for years, 1, 2, and 4. JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 JFSP scientists Year 1 Year 4 Mean (SD) Skew Kurt. n Mean (SD) Skew Kurt. n P-value Respect of practitioners 4.22 (0.46) –.34 -.20 256 4.20 (0.49) –.22 –.36 88 0.61 Year 2 Year 4 Mean (SD) Skew Kurt n Mean (SD) Skew Kurt n P-value Exchanges as necessary 3.90 (0.62) –.80 1.71 122 4.19 (0.56) –.31 –.58 76 < 0.001 Exchange websites as valuable 3.51 (0.62) –.53 1.08 104 3.73 (0.62) –.33 .45 74 < 0.05 View Large Difference of Means Tests Practitioner Results Analyses revealed over time that practitioners’ reported use of fire science and perceived respect for scientists became more favorable. That is, practitioners were much more likely to report that fire science enhanced their work and they increased use of fire science on the job in year four (M = 3.81, SD = 0.54) when compared to year one (M = 3.71, SD = 0.57); t(1433) = 2.87, p < 0.01. Similarly, practitioners were significantly more likely to rate that scientists listened to their input and that scientists provided useful information in year four (M = 3.46, SD = 0.57) when compared to year one (M = 3.33, SD = 0.55); t(1419) = 3.97, p < 0.001 (see Table 1). Questions designed to measure experiences with Exchanges and Exchange websites were not added until the second year of the online survey. Therefore, year two and year four comparisons were conducted for these variables. Again, practitioners were significantly more likely to rate their Exchange as necessary for spreading trusted and relevant fire science in year four (M = 4.07, SD = 0.62) when compared to year two (M = 3.81, SD = 0.58); t(722) = 5.71, p < 0.001. Also, practitioners were significantly more likely to rate their Exchange websites’ utility and value positively in year four (M = 3.71, SD = 0.54) when compared to year two (M = 3.61, SD = 0.50); t(659) = 2.56, p < 0.05 (see Table 1). Scientist Results There were no statistically significant changes in mean scores for the scale Scientists’ respect for practitioners because scientists’ ratings were highly favorable in year one (M = 4.22, SD = 0.46) and remained highly favorable in year four (M = 4.20, SD = 0.49). For the other two scales (Exchanges as necessary organizations for spreading trusted and relevant fire science and Exchange websites’ utility and value providing fire science), responses significantly improved from year two to year four. Specifically, respondents were much more likely to endorse that their Exchange improved the dissemination and application of fire science in year four (M = 4.19, SD = 0.56) when compared to year two (M = 3.90, SD = 0.62); t(196) = 3.41, p < 0.001. Similarly, respondents had significantly more positive ratings of their regional Exchange website and its positive impact on their work in year four (M = 3.73, SD = 0.62) when compared to year two (M = 3.51, SD = 0.62); t(176) = 2.39, p < 0.05 (see Table 2). Discussion Large-scale boundary organization initiatives are complex networks that feature multiple program sites in varied developmental stages, tasked with translating science-based research into applicable information (Sung et al. 2003; Westfall et al. 2007; Wooten et al. 2013). It is essential to the success of such initiatives to strategically and routinely assess progress toward desired goals and outcomes for the purpose of targeting improvements. In response to findings that increased production of applicable fire science did not result in increased use of fire science among practitioners, JFSP established the Fire Exchange initiative (Kocher et al. 2012). Exchanges were tasked with improving the dissemination and the application of translated fire science. To meet this goal, Exchange personnel recognized that it was essential to establish Exchanges as reliable sources of relevant fire science as well as to increase positive interactions and improve relationships between practitioners and scientists. As scientists and practitioners increase mutual respect for one another, scientists are more likely to engage in research relevant to practitioners, and practitioners are more likely to adopt and apply these research findings (McNie 2007, Laurance et al. 2012, Hulme 2014, Schwandt 2014). Therefore, when planning programming to bring these professional groups together, Exchange personnel incorporated the perspectives of both the scientists producing fire science research and the practitioners using that research in real-world contexts. To evaluate the activities of the initiative, we constructed a national logic model based on our TOC. This logic model mapped how Exchanges’ shared activities would achieve their overarching goal of bridging the gap between fire science research and practice in order to eventually change environmental conditions. According to the logic model, necessary prerequisite steps included: increasing perceptions that Exchanges are valuable and trusted sources of fire science, increasing mutual respect between scientists and practitioners, and increasing practitioners’ use of fire science in their jobs. The results of five cumulative years of evaluative research of Exchanges’ outcomes and processes indicate that Exchanges are making progress toward reaching short- and medium-term goals necessary to achieve long-lasting environmental change. For practitioners in particular, these impacts are demonstrated by improvements at all levels of Exchange efforts, including improvements in attitudes concerning the value of Exchanges, Exchange websites (Exchanges’ primary mechanism for reaching practitioners), practitioner reports of increased relevance of fire science and its use on the job, and practitioner reports that scientists respect their input on research agendas. Improvements in perceptions of Exchanges and Exchange websites indicate that these organizations increasingly meet constituents’ needs and are viewed as valuable and trusted sources of fire science information. Furthermore, these findings are supported by coincident research that reveals Exchanges are viewed as credible sources of translated fire science and that practitioners are using Exchange products to make decisions on the job (Hunter 2016). That practitioners increasingly endorse the use and application of Exchange fire science in their professional work is an important finding. This indicator of behavioral change, as a result of Exchange resources and activities, often is difficult to achieve and yet essential to achieving long-term changes in conditions (Patton 2014). The current findings, reflected as modest but statistically significant increases in mean scores for reported fire science use among practitioners, reveal that Exchanges are taking the necessary steps to achieve their long-term goals. Improvements in practitioners’ attitudes toward scientists are particularly important in demonstrating increases in mutual respect between practitioners and scientists, which is integral for fire science adoption (McNie 2007). During the first and second years of the evaluation, scientists rated themselves as highly approachable whereas practitioners rated scientists as less approachable (Sicafuse et al. 2012). Qualitative data collected in interviews with Exchange personnel in 2012 revealed that professional communication and trust could be improved between fire practitioners and scientists (Sicafuse et al. 2013, Maletsky et al. 2017). Based on these findings, Exchange personnel sought to pay special attention to improving practitioner and scientist interactions. The longitudinal results reported here suggest that their efforts have been successful in improving practitioners’ perceptions that scientists value their expertise. Additionally, supplemental qualitative data from the annually administered surveys serve to further verify these successes. As one practitioner stated, “The fire science [Exchange] has been a fantastic resource. It’s great to see the communication between [practitioners] and scientists that was almost non-existent when I began my career two decades ago.” According to another practitioner, “The Exchanges play critical roles in developing and maintaining two-way communication between fire scientists and practitioners and deserve continued funding and support through the JFSP.” Finally, as one practitioner reflected, “The highest value of the Fire Science Exchange is the Exchange itself—[that is,] when the scientist, managers and practitioners come together to solve fire challenges where the ‘answers’ are not clear. The more the Exchanges focus on springing forward using the best from traditional western science, traditional ecological knowledge, and evidence based practice and learning, the more likely we are able to advance.” Similarly, for scientists, ratings have increased over time concerning Exchanges’ impacts in their regions and value of Exchange websites. Scientists’ perceptions of practitioners were very positive initially (at the upper end of the scale) and have remained positive over time. This result is not surprising, as many scientists likely became involved with Exchanges to disseminate their research findings and improve the application of fire science research in the field. Exchanges also may attract scientists who are more interested in collaborating with practitioners and pursuing applied science research. Increases in ratings of the Exchanges overall and Exchange websites in particular indicate that scientists also benefit from Exchanges as a forum for sharing their research, identifying new research topics, and connecting with other fire science professionals. The finding that scientists rate Exchanges as valuable in the fire science community is necessary to ensure that scientists remain engaged and willing to receive constructive input and feedback from practitioners (Guston 2001, McNie 2007). Beyond the routine cautions concerning self-reported data, there are limitations to the present study. For example, although Exchanges actively recruited participants during their scheduled survey administration year, the snowball sampling strategy may have led to participation of individuals from off-year neighboring Exchanges. Additionally, although responses were categorized by Exchanges’ establishment year to examine changes in responses over time, individual responses were not tracked. Thus, multiple survey waves likely included both new and returning participants. This precluded the use of more complex statistical techniques, such as time-series analyses. Finally, participants began their interaction with Exchanges with somewhat positive attitudes and behaviors, thus restricting Evaluators’ ability to isolate the effect of Exchanges to positively influence participants through research-related resources and activities. Nevertheless, general increases on TOC variables of interest over time highlight improvements in constituents’ value of Exchange activities and products, as well as improvements in practitioners’ perceptions of scientists and their adoption of fire science. These findings support the longitudinal trends reflected in the data and are congruent with contextual qualitative data collected as part of the overall evaluation effort (Maletsky et al. 2017). Together, these results provide evidence that the magnitudes of change we report are meaningful and aligned with the TOC as illustrated in the Exchange logic model. Response rates to the annual evaluative surveys have varied over time, with more individuals participating in the first survey than in later surveys. In discussions with Fire Science Exchange Network leadership, survey fatigue and saturation issues were highlighted as concerns and the most likely reasons for survey completion decline in later years. Subsequent efforts to address these issues appear to have reversed this trend. These have included decreasing survey length and expanding constituent list-servs. These changes do not impact the analyses reported here but will help improve data collection and subsequent analyses moving forward. Finally, the data reported here are based on individuals familiar with Exchanges. Therefore, nothing definitive can be said about how Exchange participants differ from non-participants in terms of isolating the effects of Exchanges to improve fire science research and application. Current results, however, provide evidence that Exchanges are fulfilling their roles as boundary organizations and effectively disseminating translated fire science. These findings also are supported by results from separate but concurrent evaluation efforts (Hunter 2016). Overall, evaluative research to date indicates that Exchanges are acting successfully as boundary organizations, convening diverse professionals and furthering the adoption of fire science in the field. These are only first steps, and additional evaluation is needed to assess whether these gains translate to positive change in land management and environmental conditions. To this end, the evaluation of the JFSP Fire Science Exchange Network has shifted its focus from measuring short-term goals (e.g., Exchange awareness and cross-professional perceptions) to medium- and long-term goals (e.g., fire science usage, policy change, and change on the ground). The results reported here support the likely success of the Fire Science Exchange Network’s strategies to increase educational outreach, professional relationship-building, and fire science dissemination outcomes. Hence, these Exchanges can provide a roadmap to other boundary organizations that aim to enhance cross-professional communication to increase mutual respect among professionals and increase the application of scientific research in the field. Acknowledgments The authors would like to thank the Joint Fire Science Program (JFSP) Fire Science Exchange Network leadership and staff for overall evaluation support and data collection efforts, with special thanks to John Cissel, who served as JFSP Director at the time this article was written. Literature Cited Ascher , W. , T. Steelman , and R. Healy . 2010 . Knowledge and environmental policy: Re-imagining the boundaries between science and politics . MIT Press , Cambridge, MA . Calhoun , W.J. , K. Wooten , S. Bhavnani , K.E. Anderson , J. Freeman , and A.R. Brasier . 2013 . The CTSA as an exemplar framework for developing multidisciplinary translational teams . Clinical and Translational Science 6 : 60 – 71 . Google Scholar CrossRef Search ADS PubMed Carmines , E.G. , and R.A. Zeller . 1979 . Reliability and validity assessment . SAGE Publications , Thousand Oaks, CA . Google Scholar CrossRef Search ADS Cerveny , L.K. , and C.M. Ryan . 2008 . Agency capacity for recreation science and management: The case of the U.S. Forest Service . USDA For. Serv. Gen. Tech. Rep. PNW-GTR-757 . Clark , L.A. , and D. Watson . 1995 . Construction validity: Basic issues in objective scale development . Psychological Assessment 7 : 309 – 319 . doi: 10.1037/1040-3590.7.3.309 Google Scholar CrossRef Search ADS Dillman , D.A. , J.D. Smyth , and L.M. Christian . 2010 . Internet, phone, mail, and mixed-mode surveys: The tailored design method , 4th ed . Willey , Hoboken, NJ . Guston , D.H . 2001 . Boundary organizations in environmental policy and science: An introduction . Science, Technology, & Human Values 26 ( 4 ): 399 – 408 . Google Scholar CrossRef Search ADS Hall , J.A. , and E. Fleishman . 2010 . Demonstration as a means to translate conservation science into practice . Conservation Biology 24 ( 1 ): 120 – 127 . Google Scholar CrossRef Search ADS PubMed Hulme , P.E . 2014 . Bridging the knowing-doing gap: Know-who, know-what, know-why, know-how and know-when . Journal of Applied Ecology 51 : 1131 – 1136 . Google Scholar CrossRef Search ADS Hunter , M.E . 2016 . Outcomes of fire research: Is science used ? International Journal of Wildland Fire 25 ( 5 ): 495 – 504 . Google Scholar CrossRef Search ADS Joint Fire Science Program (JFSP) . 2011 . Knowledge exchange: A two-way street . Fire Science Digest 1 ( 11 ): 1 – 16 . Kocher , S.D. , E. Toman , S.F. Trainor , V. Wright , J.S. Briggs , C.P. Goebel , E.M. MontBlanc , et al. 2012 . How can we span the boundaries between wildland fire science and management in the United States ? Journal of Forestry 110 ( 8 ): 421 – 428 . Google Scholar CrossRef Search ADS Laurance , W.F. , H. Koster , M. Grooter , A.B. Anderso , P.A. Zuidema , S. Zwick , R.J. Zagt , et al. 2012 . Making conservation research more relevant for conservation practitioners . Biological Conservation 153 : 164 – 168 . Google Scholar CrossRef Search ADS Lemos , M.C. , and B.J. Morehouse . 2005 . The co-production of science and policy in integrated climate assessments . Global Environmental Change 15 ( 1 ): 57 – 68 . Google Scholar CrossRef Search ADS Maletsky , L. , W. Evans , L. Singletary , and L. Sicafuse . 2017 . Joint Fire Science Program Fire Science Exchange Network best practices . Working Paper, University of Nevada , Reno . McNie , E.C . 2007 . Reconciling the supply of scientific information with user demands: An analysis of the problem and review of the literature . Environ Sci Policy 10 : 17 – 38 . Google Scholar CrossRef Search ADS National Institute of Health [NIH] . 2004 . NIH roadmap: Re-engineering the clinical research enterprise regional translational research centers interim report . Available online at https://commonfund.nih.gov/sites/default/files/rtrc_interimreport.pdf Patton , M.Q . 2014 . What brain sciences reveal about integrating theory and practice . Am. J. Eval . 35 : 237 – 244 . Google Scholar CrossRef Search ADS Schwandt , T.M . 2014 . On the mutually informing relationship between practice and theory in evaluation . Am. J. Eval . 35 : 231 – 236 . Google Scholar CrossRef Search ADS Sicafuse , L. , W. Evans , and L. Singletary . 2012 . Joint Fire Science Program regional consortia 2012 evaluation report: A national cluster evaluation of consortia process and impacts . Available online at the Joint Fire Science Program website: https://www.firescience.gov/documents/2012_JFSP_FSEN_Evaluation_Report.pdf Sicafuse , L. , W. Evans , L. Singletary , and L. Maletsky . 2013 . Interviews with joint fire science program consortia leadership and staff . Available online at the Joint Fire Science Program website: http://www.firescience.gov/documents/JFSP_Consortia_Interviews_report.pdf Stufflebeam , D . 2001 . Evaluation models . New Directions for Evaluation 89 : 7 – 98 . Google Scholar CrossRef Search ADS Sung , N.S. , W.F.J. Crowley , M. Genel , P. Salber , L. Sandy , and L.M. Sherwood . 2003 . Central challenges facing the national clinical research enterprise . JAMA 289 : 1278 – 1287 . Google Scholar CrossRef Search ADS PubMed Tabachnick , B.G. , and L.S. Fidell . 2013 . Using multivariate statistics , 6th ed . Pearson Education , Washington, DC . Weiss , C . 1997 . How can theory-based evaluation make greater headway ? Evaluation Review 21 : 501 – 524 . Google Scholar CrossRef Search ADS Westfall , J.M. , J. Mold , and L. Fagan . 2007 . Practice-based research: “Blue highways” on the NIH roadmap . JAMA 297 : 403 – 406 . Google Scholar CrossRef Search ADS PubMed Wethington , E. , and R.E. Dunifon . 2012 . Research for the public good: Applying the methods of translational research to improve human health and well-being . American Psychological Association , Washington, DC . Google Scholar CrossRef Search ADS White , D.D. , A. Wutich , K.L. Larson , P. Gober , T. Lant , and C. Senneville . 2010 . Credibility, salience, and legitimacy of boundary objects: Water managers’ assessment of a simulation model in an immersive decision theater . Science and Public Policy 37 ( 3 ): 219 – 232 . Google Scholar CrossRef Search ADS Wooten , K.C. , R.M. Rose , G.V. Ostir , W.J. Calhoun , B.T. Ameredes , and A.R. Brasier . 2013 . Assessing and evaluating multidisciplinary translational teams: A mixed methods approach . Evaluation and the Health Professions 37 ( 1 ): 33 – 49 . Google Scholar CrossRef Search ADS PubMed Zerhouni , E.A . 2007 . Translational research: Moving discovery to practice . Clinical Pharmacology & Therapeutics 81 : 126 – 128 . doi: 10.1038/sj.clpt.6100029 Google Scholar CrossRef Search ADS © 2018 Society of American Foresters This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Journal of Forestry – Oxford University Press

Published: May 22, 2018

There are no references for this article.