Exploring the importance of evidence in local health and wellbeing strategies

Exploring the importance of evidence in local health and wellbeing strategies Abstract As evidence generators, we need to respond to the changes in the health delivery landscape if we are to continue to support public health decision-makers to make informed and judicious evidence-based choices. This study employs documentary analysis to (i) explore the extent of research evidence use in public health decision-making; (ii) to analyse occurrences of research evidence use in decision-making and (iii) to ascertain whether patterns of evidence use overlap with other area characteristics. Health and Wellbeing Strategies constitute the main source of documentary evidence. Initial results highlight that local areas are undertaking their own programmes of research that are used to inform specific questions, although the methodological robustness of these studies is unknown. There are also commonalities with previous findings, particularly with regard to the underutilization of qualitative research evidence and evidence on the effectiveness of interventions. Using Qualitative Comparative Analysis, this paper also highlights that underutilization of more academic research evidence appears disproportionally weighted towards areas with some of the most complex needs but that are not receiving the highest level of spending to meet these challenges. These areas in particular may be those where knowledge brokerage activities may have the greatest impacts. Management and policy, Public health, Research Introduction Since the introduction of Health and Social Care Act (HSCA) in April 2013, the structure of the English NHS has changed and public health strategizing and decision-making has transferred to primarily involve Local Authorities (local government; LAs) as well as Health and Wellbeing Boards (HWBs). HWBs are intended to support local authorities to cope with new public health responsibilities1 and along with Clinical Commissioning Groups (CCGs), HWBs are expected to produce a Joint Strategic Needs Assessment (JSNA) that identifies the needs of the population, and to create a Joint Health and Wellbeing Strategy(HWS) in response.2 HWSs should ‘translate JSNA findings into clear outcomes the board wants to achieve, which will inform local commissioning—leading to locally led initiatives that meet those outcomes and address the needs’ (p. 9).2 HWSs are expected to be grounded in local needs and to take into account local viewpoints and perspectives.2 As public documents, JSNAs and HWSs must be published,2 and a small number of other preceding studies have drawn on the content of HWSs to address different research questions.3–5 In the context of this study, HWSs are expected to ‘show what evidence has been considered, and what priorities have been agreed and why’ (p. 13),2 and are likely to give one of the most public-facing insights into evidence use patterns in public health and wellbeing decision-making. Public health decision-making structures now differ markedly across the UK’s devolved nations and this study is motivated by a need to introduce greater granularity in understanding the role of the evidence in English public health decision-making. Previous explorations of HWSs have illuminated differences between conceptualizations of ‘evidence’ among evidence users and generators.3 In this study, we aim to build on these foundations through focussing less on the way in which the ‘term’ evidence is used, but on the use of different ‘types’ of evidence. The timing also allows us, in theory, to explore two sets of HWSs, given that HWBs were mandated to publish their first HWS in 2013/14 and many were expected to publish updated strategies between 2015 and 2017. Finally, our chosen synthesis methods also allow us to explore how the characteristics of HWB areas may overlap with different evidence use patterns. Aims and Methods Our core unit of analysis in this study are data extracted from HWSs. A list of HWBs was obtained,6 and a sample of one-third was drawn randomly using STATA.7 At least one HWS was obtained for each HWB and uploaded into EPPI-Reviewer (systematic review software)8 and data extracted and coded. A pre-specified coding template was developed that included each strategy’s duration; the priority issues identified; and a comprehensive list of different types of evidence that were used/referred to. Types of evidence were distinguished by source, methodology and coverage, and reflected sources and forms found to be frequently utilized in public health decision-making (definitions are provided in Table 1).9,10 Each HWS was read in full by either DK or ARG, and supporting information was entered for each code used. Double extraction took place during a piloting stage to ensure consistency of extraction, with an emphasis on recording instances of evidence use as opposed to frequency of occurrence. Table 1 Definitions of evidence types used Type  Working definition  Office for National Statistics (ONS) Statistics  Estimates of population size or population growth or population characteristics deriving from the census or other sources. All evidence on population size was classed under this code. Example: ‘The Office for National Statistics estimates the current population of xxxxx at 231 900’  Specified/referenced local statistics  Statistics about the local population that are either formally referenced or a source is named. For example ‘our residents’ survey in 2014 showed that…’  Unspecified/unreferenced local statistics  Statistics about the local population that are not referenced in any way (either date or source). For example: ‘Significant parts of the housing stock in xxx are in poor condition, with an estimated xxxx properties non-decent’  National Institute for Health and Care Excellence (NICE) Guidance  NICE Guidance or other publication from NICE. For example ‘this strategy is aligned with NICE Guidelines on Anxiety and Depression’. NICE Guidance is usually informed by systematic reviews on the issue and expert and stakeholder consultation  Public Health England (PHE) Guidance or Strategy  PHE formal guidance or reports. Note instances of drawing upon the Public Health Outcomes Framework (PHOF) were cited as drawing on ‘national statistics’ (below) given that PHOF guidance was originally issued by the Department of Health. Many of the latter instances represented intended usage, as opposed to usage that informed the HWS itself  Local Government Association (LGA)  LGA guidance or research; for example LGA guidance on the conduct and purpose of HWS  King’s Fund  King’s fund guidance or research. For example: ‘These findings were reinforced in research by the King’s Fund which looked at four key lifestyle behaviours’  Local Evaluation/existing service  A named evaluation of a service which may be formally referenced. For example: ‘evaluation of Change 4 Life in xxxx showed that…’. Note where there is no reference to a service and there is no other reference, this should go as unreferenced local statistic  External individual expert—not institutional  This refers to input of a local expert—e.g. the Local Police and Crime Commissioner views x as problematic. Where an institution is named—e.g. Hertfordshire police view x as problematic—the code below was used  External expertise—institutional  This includes institutional or academic expertise that is not otherwise disaggregated—e.g. academics from UCL tell us xxxx  Academic research study  Includes journal articles and reports produced by academic institutions—e.g. UCL produced reports—also are included here  Systematic review  Any systematic review (including meta-analyses)—also put in scoping review here  Joint Strategic Needs Assessment  Joint Strategic Needs Assessment  Specific Locally commissioned research  This includes any form of research that has not been captured elsewhere that is carried out locally (e.g. at LA level or lower geography)—interviews, consultations, surveys with open ended questions, surveys that may not be representative or weighted etc – this research is usually carried out by the LA/HWB or commissioned by the LA/HWB  Specified/referenced national statistics  National statistics that have a source—e.g. DWP figures show household income in the UK declined over the past decade  Unspecified/unreferenced national statistics  National statistics not captured elsewhere—e.g. figures show wellbeing in the UK declined over the past decade  National Policy or Reports from National Agencies (Not NICE/PHE)  This includes policy from DH, DCLG or DfE. For example ‘Mental health (and emotional wellbeing) is everyone’s business: individuals, families, employers, educators and communities all need to play their part. (No Health without Mental Health: A cross government mental health outcomes strategy for people of all ages’. (DH 2011))  Specific commissioning tool  Specific commissioning tools. For example: ‘Early Help delivery tool, including the training and support to schools and lead professionals’  Other Local Policy or Strategy  References to other local strategies and plans that support the commissioning or policy-setting  Unspecified Research/Evidence  Evidence with otherwise no source or context e.g. ‘Evidence also suggests that there will be consequences of the economic downturn and the welfare reforms in relation to mental health such as: impact on individuals due to rise in unemployment causing mental health problems; shrinking economy, offering fewer job opportunities and exacerbating poverty and social exclusion; potential reductions in service provision.’  Other  Other forms of evidence contributing to the HWS not captured elsewhere  Type  Working definition  Office for National Statistics (ONS) Statistics  Estimates of population size or population growth or population characteristics deriving from the census or other sources. All evidence on population size was classed under this code. Example: ‘The Office for National Statistics estimates the current population of xxxxx at 231 900’  Specified/referenced local statistics  Statistics about the local population that are either formally referenced or a source is named. For example ‘our residents’ survey in 2014 showed that…’  Unspecified/unreferenced local statistics  Statistics about the local population that are not referenced in any way (either date or source). For example: ‘Significant parts of the housing stock in xxx are in poor condition, with an estimated xxxx properties non-decent’  National Institute for Health and Care Excellence (NICE) Guidance  NICE Guidance or other publication from NICE. For example ‘this strategy is aligned with NICE Guidelines on Anxiety and Depression’. NICE Guidance is usually informed by systematic reviews on the issue and expert and stakeholder consultation  Public Health England (PHE) Guidance or Strategy  PHE formal guidance or reports. Note instances of drawing upon the Public Health Outcomes Framework (PHOF) were cited as drawing on ‘national statistics’ (below) given that PHOF guidance was originally issued by the Department of Health. Many of the latter instances represented intended usage, as opposed to usage that informed the HWS itself  Local Government Association (LGA)  LGA guidance or research; for example LGA guidance on the conduct and purpose of HWS  King’s Fund  King’s fund guidance or research. For example: ‘These findings were reinforced in research by the King’s Fund which looked at four key lifestyle behaviours’  Local Evaluation/existing service  A named evaluation of a service which may be formally referenced. For example: ‘evaluation of Change 4 Life in xxxx showed that…’. Note where there is no reference to a service and there is no other reference, this should go as unreferenced local statistic  External individual expert—not institutional  This refers to input of a local expert—e.g. the Local Police and Crime Commissioner views x as problematic. Where an institution is named—e.g. Hertfordshire police view x as problematic—the code below was used  External expertise—institutional  This includes institutional or academic expertise that is not otherwise disaggregated—e.g. academics from UCL tell us xxxx  Academic research study  Includes journal articles and reports produced by academic institutions—e.g. UCL produced reports—also are included here  Systematic review  Any systematic review (including meta-analyses)—also put in scoping review here  Joint Strategic Needs Assessment  Joint Strategic Needs Assessment  Specific Locally commissioned research  This includes any form of research that has not been captured elsewhere that is carried out locally (e.g. at LA level or lower geography)—interviews, consultations, surveys with open ended questions, surveys that may not be representative or weighted etc – this research is usually carried out by the LA/HWB or commissioned by the LA/HWB  Specified/referenced national statistics  National statistics that have a source—e.g. DWP figures show household income in the UK declined over the past decade  Unspecified/unreferenced national statistics  National statistics not captured elsewhere—e.g. figures show wellbeing in the UK declined over the past decade  National Policy or Reports from National Agencies (Not NICE/PHE)  This includes policy from DH, DCLG or DfE. For example ‘Mental health (and emotional wellbeing) is everyone’s business: individuals, families, employers, educators and communities all need to play their part. (No Health without Mental Health: A cross government mental health outcomes strategy for people of all ages’. (DH 2011))  Specific commissioning tool  Specific commissioning tools. For example: ‘Early Help delivery tool, including the training and support to schools and lead professionals’  Other Local Policy or Strategy  References to other local strategies and plans that support the commissioning or policy-setting  Unspecified Research/Evidence  Evidence with otherwise no source or context e.g. ‘Evidence also suggests that there will be consequences of the economic downturn and the welfare reforms in relation to mental health such as: impact on individuals due to rise in unemployment causing mental health problems; shrinking economy, offering fewer job opportunities and exacerbating poverty and social exclusion; potential reductions in service provision.’  Other  Other forms of evidence contributing to the HWS not captured elsewhere  Different synthesis methods were employed: (i) using the coding template, we provided a narrative account of the context in which evidence was used, focussing on key themes and their recurrence across strategies; (ii) we undertook further analyses through tabulations and charts; (iii) third, we undertook Qualitative Comparative Analyses (QCA). QCA has its basis in set-theoretic logic, and is well-placed in synthesizing data from a small number of cases to uncover complex configurations of conditions observed to overlap with an outcome.11 QCA was used primarily to understand: (i) the HWB characteristics overlapping with evidence use; as well as (ii) exploring overlaps between evidence use patterns and HWB characteristics with LA public health spending allocations. We followed guidance provided elsewhere11 in creating our QCA solution, and ran a series of diagnostic tests to ensure the quality of our solution (no contradictory configurations were detected and no contradictory simplifying assumptions were made in accounting for logical remainders (combinations of conditions not supported by observed cases)). We do not name HWBs and avoid disclosing their identity in our results; while HWSs are public documents, their inclusion in analyses may not have been foreseen by HWBs and it was not within the scope of this work to contact HWBs for further information or clarifications. Results The health of health and wellbeing strategies From our sample of HWBs, we located a strategy published between 2012 and 2014 and an updated strategy published between 2015 and March 2017 for just under half of areas (25/51; see Table 3). One strategy alone was located for 26 HWBs (23 published 2012–14 and 3 published 2015–17). Almost a quarter of HWBs (23.5%) did not have a current strategy in March 2017, with strategies having either expired with no successor (10/12 HWBs) or having been published in 2013/14 with no expiry date (2/12 HWBs). Given that HWBs are required to publish HWSs to outline their long-term direction, this appears to compromise the significance of HWSs (Table 2). The absence of a current strategy was not patterned by the characteristics of the HWB area (e.g. level of deprivation). Table 2 Descriptive characteristics of Health and Wellbeing board areas and the presence of Health and Wellbeing Strategies ID number  HWS discovered published 2013/14  HWS discovered published 2015/16  HWB HAS current HWS in place  Quintile of index of multiple deprivation (1 = lest deprived)  Quintile proportion of older people aged 65 + (1 = fewest older people)  Quintile proportion of black and minority peope (1 = fewest BME people)  Quintile male life exoectancy (1 = lowest life expectancy)  Quintile female life exoectancy (1 = lowest life expectancy)  Rurality  1  Y    No current HWS  1  3  4  5  5  Urban with Major Conurbation  2  Y  Y    1  5  1  5  5  Largely Rural (rural including hub towns 50–79%)  3  Y      1  5  2  4  5  Urban with Significant Rural (rural including hub towns 26–49%)  4  Y  Y    1  2  5  5  5  Urban with Major Conurbation  5    Y  No current HWS  1  1  5  5  5  Urban with Major Conurbation  6  Y  Y    1  5  1  4  4  Largely Rural (rural including hub towns 50–79%)  7  Y  Y    1  5  2  5  5  Mainly Rural (rural including hub towns ≥80%)  8  Y    No current HWS  1  3  2  5  5  Urban with City and Town  9  Y  Y    1  3  3  3  4  Urban with City and Town  10  Y  Y    2  2  5  5  5  Urban with Major Conurbation  11  Y    No current HWS  2  3  4  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  12  Y    No current HWS  2  3  4  5  4  Urban with Major Conurbation  13  Y  Y    2  5  2  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  14  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  15  Y  Y    2  4  3  4  4  Urban with Major Conurbation  16  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  17  Y  Y    2  1  4  3  3  Urban with City and Town  18  Y      2  .  .  .  .    19  Y  Y    2  4  3  3  3  Urban with Major Conurbation  20  Y    No current HWS  2  3  4  4  3  Urban with Major Conurbation  21  Y  Y    2  3  1  1  2  Urban with City and Town  22  Y      3  3  3  2  2  Urban with Major Conurbation  23  Y  Y    3  1  5  5  5  Urban with Major Conurbation  24  Y  Y    3  5  1  3  3  Largely Rural (rural including hub towns 50–79%)  25  Y    No current HWS  3  4  2  2  2  Urban with City and Town  26  Y  Y    3  1  5  4  4  Urban with Major Conurbation  27    Y    3  1  5  2  3  Urban with Major Conurbation  28  Y  Y    3  5  2  3  3  Mainly Rural (rural including hub towns ≥80%)  29  Y      3  2  4  2  3  Urban with Major Conurbation  30  Y      3  4  2  3  2  Urban with Significant Rural (rural including hub towns 26–49%)  31  Y    No current HWS  3  5  1  2  3  Largely Rural (rural including hub towns 50–79%)  32  Y  Y    3  1  4  3  3  Urban with Major Conurbation  33  Y      3  5  1  .  .  Mainly Rural (rural including hub towns ≥80%)  34  Y    No current HWS  3  3  1  1  1  Urban with Major Conurbation  35  Y  Y    4  4  1  2  2  Urban with Minor Conurbation  36    Y    4  1  5  2  2  Urban with City and Town  37  Y  Y    4  2  4  2  2  Urban with City and Town  38  Y    No current HWS  4  3  2  2  2  Urban with City and Town  39  Y      4  2  3  2  2  Urban with City and Town  40  Y    No current HWS  4  2  4  3  2  Urban with City and Town  41  Y  Y    4  1  5  4  3  Urban with Major Conurbation  42  Y  Y    5  1  5  1  1  Urban with City and Town  43  Y  Y    5  5  2  1  1  Urban with City and Town  44  Y      5  2  1  1  1  Urban with City and Town  45  Y      5  4  1  1  1  Urban with City and Town  46  Y  Y    5  2  3  1  1  Urban with City and Town  47  Y  Y    5  1  5  1  1  Urban with Major Conurbation  48  Y  Y    5  2  4  1  1  Urban with Major Conurbation  49  Y    No current HWS  5  2  3  1  1  Urban with Major Conurbation  50  Y  Y    5  3  3  1  1  Urban with City and Town  51  Y      5  4  2  2  2  Urban with Major Conurbation  ID number  HWS discovered published 2013/14  HWS discovered published 2015/16  HWB HAS current HWS in place  Quintile of index of multiple deprivation (1 = lest deprived)  Quintile proportion of older people aged 65 + (1 = fewest older people)  Quintile proportion of black and minority peope (1 = fewest BME people)  Quintile male life exoectancy (1 = lowest life expectancy)  Quintile female life exoectancy (1 = lowest life expectancy)  Rurality  1  Y    No current HWS  1  3  4  5  5  Urban with Major Conurbation  2  Y  Y    1  5  1  5  5  Largely Rural (rural including hub towns 50–79%)  3  Y      1  5  2  4  5  Urban with Significant Rural (rural including hub towns 26–49%)  4  Y  Y    1  2  5  5  5  Urban with Major Conurbation  5    Y  No current HWS  1  1  5  5  5  Urban with Major Conurbation  6  Y  Y    1  5  1  4  4  Largely Rural (rural including hub towns 50–79%)  7  Y  Y    1  5  2  5  5  Mainly Rural (rural including hub towns ≥80%)  8  Y    No current HWS  1  3  2  5  5  Urban with City and Town  9  Y  Y    1  3  3  3  4  Urban with City and Town  10  Y  Y    2  2  5  5  5  Urban with Major Conurbation  11  Y    No current HWS  2  3  4  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  12  Y    No current HWS  2  3  4  5  4  Urban with Major Conurbation  13  Y  Y    2  5  2  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  14  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  15  Y  Y    2  4  3  4  4  Urban with Major Conurbation  16  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  17  Y  Y    2  1  4  3  3  Urban with City and Town  18  Y      2  .  .  .  .    19  Y  Y    2  4  3  3  3  Urban with Major Conurbation  20  Y    No current HWS  2  3  4  4  3  Urban with Major Conurbation  21  Y  Y    2  3  1  1  2  Urban with City and Town  22  Y      3  3  3  2  2  Urban with Major Conurbation  23  Y  Y    3  1  5  5  5  Urban with Major Conurbation  24  Y  Y    3  5  1  3  3  Largely Rural (rural including hub towns 50–79%)  25  Y    No current HWS  3  4  2  2  2  Urban with City and Town  26  Y  Y    3  1  5  4  4  Urban with Major Conurbation  27    Y    3  1  5  2  3  Urban with Major Conurbation  28  Y  Y    3  5  2  3  3  Mainly Rural (rural including hub towns ≥80%)  29  Y      3  2  4  2  3  Urban with Major Conurbation  30  Y      3  4  2  3  2  Urban with Significant Rural (rural including hub towns 26–49%)  31  Y    No current HWS  3  5  1  2  3  Largely Rural (rural including hub towns 50–79%)  32  Y  Y    3  1  4  3  3  Urban with Major Conurbation  33  Y      3  5  1  .  .  Mainly Rural (rural including hub towns ≥80%)  34  Y    No current HWS  3  3  1  1  1  Urban with Major Conurbation  35  Y  Y    4  4  1  2  2  Urban with Minor Conurbation  36    Y    4  1  5  2  2  Urban with City and Town  37  Y  Y    4  2  4  2  2  Urban with City and Town  38  Y    No current HWS  4  3  2  2  2  Urban with City and Town  39  Y      4  2  3  2  2  Urban with City and Town  40  Y    No current HWS  4  2  4  3  2  Urban with City and Town  41  Y  Y    4  1  5  4  3  Urban with Major Conurbation  42  Y  Y    5  1  5  1  1  Urban with City and Town  43  Y  Y    5  5  2  1  1  Urban with City and Town  44  Y      5  2  1  1  1  Urban with City and Town  45  Y      5  4  1  1  1  Urban with City and Town  46  Y  Y    5  2  3  1  1  Urban with City and Town  47  Y  Y    5  1  5  1  1  Urban with Major Conurbation  48  Y  Y    5  2  4  1  1  Urban with Major Conurbation  49  Y    No current HWS  5  2  3  1  1  Urban with Major Conurbation  50  Y  Y    5  3  3  1  1  Urban with City and Town  51  Y      5  4  2  2  2  Urban with Major Conurbation  Typically, HWSs covered a period of 3–4 years, although some covered longer periods (12/76 strategies reviewed in total) of up to 5, 6 (one) and 10 (one). A minority of updated strategies stated that the update was based on the previous strategy (Table 2: IDs 32, 36 and 41); while another 10 mentioned the existence of a previous strategy. No instance was recorded where an in-depth evaluation of the previous HWS had taken place, and no decisions or priorities included in the updated HWSs appeared to be based upon an evaluation of the previous strategy. Frequency of different types of evidence use A preference for statistical data Of the 76 HWSs reviewed in total (published in 2013/14 and 2015/16), only one was informed by a single type of evidence; others were informed by up to 13 types. HWSs frequently drew upon statistical sources, mainly to demonstrate evidence of need (Fig. 1). As expected, the Joint Strategic Needs Assessment (JSNA(s)) informed the majority of HWSs, particularly among HWSs published in 2013/14 (47/48 HWSs (98%)). By the second wave of HWSs, their role appeared to attenuate slightly, although they remained the most frequently cited source of evidence (23/28 strategies (82%)). JNSAs appear to be fulfilling their intended role in clarifying the health needs and status of the local population. However, JSNAs typically represent a statistical compendium and it is perhaps unsurprising, given that JSNAs formed a bedrock of many HWSs, that no HWS presented evidence from existing qualitative research studies that had been collected and analysed robustly. Alongside statistics from JSNAs, many HWSs used statistical data on local conditions that were unreferenced (27/48 in 2013/14 and 16/28 in 2015/16). Such data typically contained no source, no date, and sometimes were unclear on the scope and population. A typical example was ‘although the trend in life expectancy is upward, there is a 2.5-year gap between males and females across [HWB], with more inequalities in disadvantaged communities’ (HWS 14). Similar patterns were observed with unreferenced national-level statistics used to compare the HWB position with the national picture. Fig. 1 View largeDownload slide Types of evidence used in Health and Wellbeing Strategies (% of strategies with at least one mention). Fig. 1 View largeDownload slide Types of evidence used in Health and Wellbeing Strategies (% of strategies with at least one mention). National statistics came from a number of sources including outcome framework statistics. Furthermore, a number of HWSs specifically included plans to use Public Health Outcomes Framework data to monitor the success and impact of the HWS, even if these were not directly used to directly inform the direction of the HWS (10 HWSs). A typical example included: ‘we will measure our success by monitoring local progress against key indicators or measures from the three recently published national outcomes frameworks for Public Health, The NHS and Adult Social Care’ (ID: 17). While the targets themselves were often left unspecified, many HWSs emphasized the need to monitor progress and how this could be done, even if this ambition was often left unfulfilled (see above). Trusted academic sources Academic research studies were frequently cited sources of evidence, although were often confined to a limited number of ‘trusted’ or accessible sources. Approximately, half of HWSs included at least one source of academic research published within a peer-reviewed journal or published as a report by an academic institution. There was no apparent difference in the use of academic sources between HWSs published in 2013/14 and 2015/16. One of the most commonly used sources was the Marmot Review (Fair Society, Healthy Lives)12 used in 23 HWSs. Along with Dahlgren and Whitehead’s report13 this was frequently used to justify a focus on health inequalities and social determinants of health. As was the case in previous explorations,3 NICE guidance and systematic reviews were rarely cited and there was no evidence that there had been any increase in usage over time. Inclusion of academic ‘evidence’ was not synonymous with presenting evidence that adhered to academic writing conventions. Typically this could include omitting the source and date of the evidence, for instance ‘a recent study by the Wisconsin University showed the rankings of factors which determined the best health outcomes for a population’ (ID: 43). Local research and evidence gathering While HWSs tended to rely more heavily on published statistical data than published qualitative data, there were indications that engagement activities were making a contribution. Almost half of HWSs included reference to specific local research (20/48 published in 2013/14 and 14/28 published in 2015/16); often this included consultation and engagement activities and involved taking a draft set of priorities for consultation; for example, ‘we consulted widely on these proposals, listened to what people said, undertook an initial Equalities Impact Assessment and used this to inform the draft strategy’ (ID: 13). Few HWSs that had conducted consultative exercises presented the findings alongside the HWSs. There was also indicative evidence that consultation exercises were a more frequent occurrence among HWSs published in 2015/16 (9/28 HWSs) than in 2013/14 (10/48 HWSs). This increase could be symptomatic of the trend towards increased local accountability and politicization said to be occurring in public health decision-making.14 Area characteristics, health priorities, patterns of evidence use There was a little evidence of systematic overlap between the characteristics of HWB areas—including their demographic characteristics (e.g. proportion of older people and proportion of black and ethnic minority residents), rurality, life expectancy and level of deprivation (based on the index of multiple deprivation)—and differential patterns of evidence use within the HWS. Similarly, there were no clear overlaps between different health priorities in an area and patterns of evidence use. Area characteristics, patterns of evidence use and spending allocations Funding for public health activities through ring-fenced public health allocations has been substantially reduced since the HSCA.15 Initial allocations principally reflected the degree to which local systems invested historically in preventive services, and the standardized mortality ratio for people aged 75 and under. The latter was intended to capture health inequalities as well as socioeconomic deprivation.15,16 We explored the extent to which the differential use of robust sources of evidence (reflecting academic sources and NICE guidelines) were observed to overlap with high public health spending allocations per person (projected for 2017/18).17 Alongside robust evidence sources, we also explored the extent to which the HWS included evidence from specific local research and evidence and guidance produced by national-level stakeholders (e.g. evidence from PHE, the LGA, among others), as well as area characteristics, and how these overlapped with spending allocations. Because of the nature of the indicators we employed crisp-set QCA, converting each factor into a binary measure. Here, we explore the extent to which evidence use and area characteristics (highest two quintiles of deprivation and highest quintile of male life expectancy) overlapped with being in the highest quintile of public health allocated spending; each quintile was calculated on the basis of national ranking and not the ranking within the sample. Because of their proximity to the outcome, we focussed only on HWSs published in 2015/16 in these analyses (n = 28). Given that allocations have their basis in existing public health commitments as well as deprivation/inequality, it is not unsurprising that having above average levels of deprivation and/or low levels of life expectancy were frequently observed conditions among areas with high levels of public health spending allocations. However, deprived areas with high levels of public health spending allocations were also supported by HWSs informed by robust evidence sources (Table 3). This indicates that areas with the highest public health challenge, and the highest budget allocations, were drawing on robust sources to inform health and wellbeing strategies; these configurations accounted for three-quarters of instances of high spending allocations observed. Areas with pre-existing (prior to HSCA 2012) large public health commitments may therefore have a more established culture of evidence use. Those areas with the highest quintile of public health spending allocations but that were not highly deprived did not draw upon ‘robust’ sources of evidence in the HWS. In deprived areas, therefore, the use of robust evidence sources was a sufficient condition overlapping with the highest quintile of public health spending; for those areas that did not have high levels of deprivation or low life expectancy; use of robust sources was not a sufficient condition. The truth table (Table 3) also showed that a number of areas with relatively high levels of deprivation were not included within the highest quintile of public health spending allocations; crucially these same areas did not include robust sources within their HWSs, indicating a less developed evidence use culture within those very areas likely facing the most substantial public health challenges and the fewest resources to tackle these. Table 3 Truth table and intermediate solution examining public health allocations Truth table (Notes: the truth table here displays logical combinations of conditions observed in the data and the outcome; each row represents a combination of conditions)  Conditions reflecting evidence use  Conditions reflecting local characteristics  Outcome  Number of HWSs with observed combination  Indicators of solution fit  Robust evidence sources included in HWS (1 = yes; 0 = No)  Specific Local Research included in HWS (1 = yes; 0 = No)  Reference to JSNA included in HWS (1 = yes; 0 = No)  Reference to evidence/guidance from national stakeholders in HWS (1 = yes; 0 = No)  Most deprived 40% (Index of Multiple Deprivation) (1 = yes; 0 = No)  Male life expectancy in the lowest quintile (1=yes; 0=No)  Outcome (top quintile of public health allocations per person) (1 = yes; 0 = No)  Number of HWSs in configuration  Consistency score with sub-set relationship  Proportional reduction in inconsistency  1  0  1  1  1  1  1  2  1  1  1  1  1  1  1  1  1  2  1  1  0  0  1  1  0  0  1  1  1  1  0  1  0  1  0  0  1  1  1  1  1  1  0  0  1  1  1  1  1  1  1  1  1  1  1  0  1  1  1  1  0  1  1  1  0  0  0  3  0  0  1  1  1  1  0  0  0  3  0  0  0  0  1  0  1  0  0  2  0  0  0  1  1  0  0  0  0  2  0  0  1  0  1  1  0  0  0  2  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  1  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  1  1  0  1  0  0  0  0  1  1  1  0  0  1  0  0  1  0  0  0  0  0  0  1  0  0  1  0  1  0  0  0  0  1  0  0  1  1  1  1  0  1  0  1  0  0  Truth table (Notes: the truth table here displays logical combinations of conditions observed in the data and the outcome; each row represents a combination of conditions)  Conditions reflecting evidence use  Conditions reflecting local characteristics  Outcome  Number of HWSs with observed combination  Indicators of solution fit  Robust evidence sources included in HWS (1 = yes; 0 = No)  Specific Local Research included in HWS (1 = yes; 0 = No)  Reference to JSNA included in HWS (1 = yes; 0 = No)  Reference to evidence/guidance from national stakeholders in HWS (1 = yes; 0 = No)  Most deprived 40% (Index of Multiple Deprivation) (1 = yes; 0 = No)  Male life expectancy in the lowest quintile (1=yes; 0=No)  Outcome (top quintile of public health allocations per person) (1 = yes; 0 = No)  Number of HWSs in configuration  Consistency score with sub-set relationship  Proportional reduction in inconsistency  1  0  1  1  1  1  1  2  1  1  1  1  1  1  1  1  1  2  1  1  0  0  1  1  0  0  1  1  1  1  0  1  0  1  0  0  1  1  1  1  1  1  0  0  1  1  1  1  1  1  1  1  1  1  1  0  1  1  1  1  0  1  1  1  0  0  0  3  0  0  1  1  1  1  0  0  0  3  0  0  0  0  1  0  1  0  0  2  0  0  0  1  1  0  0  0  0  2  0  0  1  0  1  1  0  0  0  2  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  1  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  1  1  0  1  0  0  0  0  1  1  1  0  0  1  0  0  1  0  0  0  0  0  0  1  0  0  1  0  1  0  0  0  0  1  0  0  1  1  1  1  0  1  0  1  0  0  Configuration  Intermediate solution (Notes: represents simplification of truth table after Boolean minimization)  Consistency score with sub-set relationship  Proportional reduction in inconsistency  Raw coverage  Unique coverage  HWB ID  1  Specific local research*~ JSNA* National stakeholder evidence/guidance  1  1  0.125  0.125  23  2  Robust evidence * Specific local research *Highly deprived area* Low male life expectancy  1  1  0.375  0.125  42,47,50  3  Robust evidence * Specific local research * National stakeholder evidence/guidance * Highly deprived area  1  1  0.375  0.125  41,42,50  4  Robust evidence *JSNA * National stakeholder evidence/guidance * Highly deprived area * Low male life expectancy  1  1  0.5  0.25  42,43,46,50  5  ~Robust evidence * ~ Specific local research * JSNA * National stakeholder evidence/guidance *~ Highly deprived area  1  1  0.125  0.125  27    M1  1  1  1      Configuration  Intermediate solution (Notes: represents simplification of truth table after Boolean minimization)  Consistency score with sub-set relationship  Proportional reduction in inconsistency  Raw coverage  Unique coverage  HWB ID  1  Specific local research*~ JSNA* National stakeholder evidence/guidance  1  1  0.125  0.125  23  2  Robust evidence * Specific local research *Highly deprived area* Low male life expectancy  1  1  0.375  0.125  42,47,50  3  Robust evidence * Specific local research * National stakeholder evidence/guidance * Highly deprived area  1  1  0.375  0.125  41,42,50  4  Robust evidence *JSNA * National stakeholder evidence/guidance * Highly deprived area * Low male life expectancy  1  1  0.5  0.25  42,43,46,50  5  ~Robust evidence * ~ Specific local research * JSNA * National stakeholder evidence/guidance *~ Highly deprived area  1  1  0.125  0.125  27    M1  1  1  1      Key: ~ = absence of condition; * = logical ‘AND’. Discussion and Conclusion Summary and limitations Evidence is clearly intended to be at the core of HWSs,2 and we found that HWSs often drew upon a diverse range of sources to justify priorities and actions. However, in line with previous explorations,3 the use of evidence is often not straightforward. Academic sources are frequently confined to a limited set of ‘trusted’ sources; local and national statistics are repeatedly used in ways that prevent verification, benchmarking or understanding trends over time; and the use of peer-reviewed qualitative research/evidence was found to be absent. Although there were indications that many HWSs were informed by consultative exercises, the full findings were not presented. New HWSs were being published during periods of substantial cuts in public health spending,15 although key public health challenges remain substantial.18 QCA analyses suggested that areas with the largest public health challenges and the largest spending allocations were drawing on robust sources of evidence in their HWSs. In contrast, areas that did not have the largest spending allocation but were nevertheless facing substantial public health challenges were not drawing on robust sources in the HWS. Arguably, the judicious use of evidence becomes all the more important when resources are scarce,10 and the results suggest that an evidence use culture may be underdeveloped in many areas of higher deprivation that do not receive the greatest funding. A number of caveats accompany the results including that double data extraction did not occur for the majority of the 76 HWSs reviewed (except for piloting), that area characteristics were based on broad indicators/quintiles; and that unreported or unreferenced sources may also have contributed to HWS development. This latter consideration is a particular disadvantage of documentary analysis, although HWSs are expected to demonstrate which evidence has been considered.2 Implications The results give a complex picture of evidence use in local public health decision-making and priority setting. While there is a thriving evidence use culture, this evidence tends to be drawn from a narrow pool of sources and is not used in the most transparent way. The exclusion of published qualitative literature is concerning as it suggests that gaining an understanding of the mechanisms driving health inequalities and how interventions to tackle these ‘work’ is not prioritized. A recent review uncovered that research evidence often did not meet local authorities’ requirements with respect to locality, political salience and economic focus.10 The results here suggest that where evidence meeting these needs is unavailable, this is often be supplemented by locally conducted research, the methodological quality of which remains unknown. This finding may partially explain why previous studies find that the word ‘evidence’ in public health decision-making is not always synonymous with drawing upon a robust body of research evidence.19 Nevertheless, it is clear that improving the local salience of academic research evidence is central to enhancing its contribution to local decision-making. This may involve enhancing research capacity within LAs/HWBs, developing (statistical and other) methods of enhancing the generalizability of evidence to local areas, and/or improving knowledge translation practices. With respect to the latter, allied work being conducted by the authors with LA public health teams suggests that continuous dialogue between evidence generators and decision-makers would facilitate more effective use of evidence. It is notable that the composition of HWBs usually does not include representation from knowledge brokers; the results here suggest that this absence may be felt more keenly among HWBs facing substantial public health challenges but who will not receive the greatest resources to meet these. Acknowledgements We would like to acknowledge the helpful input of the editors of the journal and the special issue in shaping this paper. We would also like to acknowledge the input of all the stakeholders in our qualitative research project (currently underway) in helping to make sense of evidence use in public health decision-making. Funding The research was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. References 1 Greaves Z, McCafferty S. Health and wellbeing boards: public health decision making bodies or political pawns? Public Health  2017; 143: 78– 84. Google Scholar CrossRef Search ADS PubMed  2 DH 2013, Statutory Guidance on Joint Strategic Needs Assessments and Joint Health and Wellbeing Strategies, ( London: Department of Health). 3 Beenstock J, Sowden S, Hunter DJ et al.  . Are health and well-being strategies in England fit for purpose? A thematic content analysis. J Public Health (Bangkok)  2014; 37: 461– 9. Google Scholar CrossRef Search ADS   4 Diabetes UK. Are Health and Wellbeing Boards Taking Account of Diabetes? A Review of JSNAs and Health and Wellbeing Strategies . London: Diabetes UK, 2014. 5 Scrutton J. A Place for Parity: Health and Wellbeing Boards and Mental Health . London: Mental Health Partnerships, 2013. 6 Health and wellbeing boards directory [database on the Internet]. The King’s Fund. 2016. Available from: https://www.kingsfund.org.uk/projects/health-and-wellbeing-boards/hwb-map (31 August 2016, date last accessed). 7 StataCorp. Stata Statistical Software: Release 13 . College Station, TX: StataCorp LP, 2013. 8 Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4.0: Software for Research Synthesis . London: Institute of Education, 2010. 9 Clarke A, Taylor-Phillips S, Swan J et al.  . Evidence-based commissioning in the English NHS: who uses which sources of evidence? A survey 2010/2011. BMJ Open  2013; 3: e002714. Google Scholar CrossRef Search ADS PubMed  10 Kneale D, Rojas-García A, Raine R et al.  . The use of evidence in English local public health decision-making. Implement Sci  2017; 12: 53. Google Scholar CrossRef Search ADS PubMed  11 Thomas J, O’Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev  2014; 3: 1– 14. Google Scholar CrossRef Search ADS PubMed  12 Marmot M, Allen J, Goldblatt P et al.  . The Marmot review: Fair society, healthy lives. Strategic review of health inequalities in England post-2010 Crown Copyright, London: The Marmot Review. 2010. 13 Dahlgren G, Whitehead M. Policies and Strategies to Promote Social Equity in Health . Stockholm: Institute for Futures Studies, 1991. 14 Marks L, Hunter D, Scalabrini S et al.  . The return of public health to local government in England: changing the parameters of the public health prioritization debate? Public health  2015; 129: 1194– 203. Google Scholar CrossRef Search ADS PubMed  15 FPH. UK Faculty of Public Health Response to Department of Health Consultation on the Public Health Grant: Proposed Target Allocation Formula for 2016/17 . London: Faculty of Public Health, 2016. 16 London Councils. Public Health Funding Allocations—London Councils . London: London Councils, 2013. 17 DH. Public Health Grants to Local Authorities: 2016 to 2017 . London: Department of Health, 2016. 18 Davies A, Keeble E, Bhatia T et al.  . Public Health and Prevention: Research Summary . London: Quality Watch (Nuffield Foundation and The Health Foundation), 2016. 19 Katikireddi SV, Higgins M, Bond L et al.  . How evidence based is English public health policy? Br Med J  2011; 343: d7310. Google Scholar CrossRef Search ADS   © The Author 2018. Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Public Health Oxford University Press

Exploring the importance of evidence in local health and wellbeing strategies

Loading next page...
 
/lp/ou_press/exploring-the-importance-of-evidence-in-local-health-and-wellbeing-Z1ciX0lkqD
Publisher
Oxford University Press
Copyright
© The Author 2018. Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
1741-3842
eISSN
1741-3850
D.O.I.
10.1093/pubmed/fdx152
Publisher site
See Article on Publisher Site

Abstract

Abstract As evidence generators, we need to respond to the changes in the health delivery landscape if we are to continue to support public health decision-makers to make informed and judicious evidence-based choices. This study employs documentary analysis to (i) explore the extent of research evidence use in public health decision-making; (ii) to analyse occurrences of research evidence use in decision-making and (iii) to ascertain whether patterns of evidence use overlap with other area characteristics. Health and Wellbeing Strategies constitute the main source of documentary evidence. Initial results highlight that local areas are undertaking their own programmes of research that are used to inform specific questions, although the methodological robustness of these studies is unknown. There are also commonalities with previous findings, particularly with regard to the underutilization of qualitative research evidence and evidence on the effectiveness of interventions. Using Qualitative Comparative Analysis, this paper also highlights that underutilization of more academic research evidence appears disproportionally weighted towards areas with some of the most complex needs but that are not receiving the highest level of spending to meet these challenges. These areas in particular may be those where knowledge brokerage activities may have the greatest impacts. Management and policy, Public health, Research Introduction Since the introduction of Health and Social Care Act (HSCA) in April 2013, the structure of the English NHS has changed and public health strategizing and decision-making has transferred to primarily involve Local Authorities (local government; LAs) as well as Health and Wellbeing Boards (HWBs). HWBs are intended to support local authorities to cope with new public health responsibilities1 and along with Clinical Commissioning Groups (CCGs), HWBs are expected to produce a Joint Strategic Needs Assessment (JSNA) that identifies the needs of the population, and to create a Joint Health and Wellbeing Strategy(HWS) in response.2 HWSs should ‘translate JSNA findings into clear outcomes the board wants to achieve, which will inform local commissioning—leading to locally led initiatives that meet those outcomes and address the needs’ (p. 9).2 HWSs are expected to be grounded in local needs and to take into account local viewpoints and perspectives.2 As public documents, JSNAs and HWSs must be published,2 and a small number of other preceding studies have drawn on the content of HWSs to address different research questions.3–5 In the context of this study, HWSs are expected to ‘show what evidence has been considered, and what priorities have been agreed and why’ (p. 13),2 and are likely to give one of the most public-facing insights into evidence use patterns in public health and wellbeing decision-making. Public health decision-making structures now differ markedly across the UK’s devolved nations and this study is motivated by a need to introduce greater granularity in understanding the role of the evidence in English public health decision-making. Previous explorations of HWSs have illuminated differences between conceptualizations of ‘evidence’ among evidence users and generators.3 In this study, we aim to build on these foundations through focussing less on the way in which the ‘term’ evidence is used, but on the use of different ‘types’ of evidence. The timing also allows us, in theory, to explore two sets of HWSs, given that HWBs were mandated to publish their first HWS in 2013/14 and many were expected to publish updated strategies between 2015 and 2017. Finally, our chosen synthesis methods also allow us to explore how the characteristics of HWB areas may overlap with different evidence use patterns. Aims and Methods Our core unit of analysis in this study are data extracted from HWSs. A list of HWBs was obtained,6 and a sample of one-third was drawn randomly using STATA.7 At least one HWS was obtained for each HWB and uploaded into EPPI-Reviewer (systematic review software)8 and data extracted and coded. A pre-specified coding template was developed that included each strategy’s duration; the priority issues identified; and a comprehensive list of different types of evidence that were used/referred to. Types of evidence were distinguished by source, methodology and coverage, and reflected sources and forms found to be frequently utilized in public health decision-making (definitions are provided in Table 1).9,10 Each HWS was read in full by either DK or ARG, and supporting information was entered for each code used. Double extraction took place during a piloting stage to ensure consistency of extraction, with an emphasis on recording instances of evidence use as opposed to frequency of occurrence. Table 1 Definitions of evidence types used Type  Working definition  Office for National Statistics (ONS) Statistics  Estimates of population size or population growth or population characteristics deriving from the census or other sources. All evidence on population size was classed under this code. Example: ‘The Office for National Statistics estimates the current population of xxxxx at 231 900’  Specified/referenced local statistics  Statistics about the local population that are either formally referenced or a source is named. For example ‘our residents’ survey in 2014 showed that…’  Unspecified/unreferenced local statistics  Statistics about the local population that are not referenced in any way (either date or source). For example: ‘Significant parts of the housing stock in xxx are in poor condition, with an estimated xxxx properties non-decent’  National Institute for Health and Care Excellence (NICE) Guidance  NICE Guidance or other publication from NICE. For example ‘this strategy is aligned with NICE Guidelines on Anxiety and Depression’. NICE Guidance is usually informed by systematic reviews on the issue and expert and stakeholder consultation  Public Health England (PHE) Guidance or Strategy  PHE formal guidance or reports. Note instances of drawing upon the Public Health Outcomes Framework (PHOF) were cited as drawing on ‘national statistics’ (below) given that PHOF guidance was originally issued by the Department of Health. Many of the latter instances represented intended usage, as opposed to usage that informed the HWS itself  Local Government Association (LGA)  LGA guidance or research; for example LGA guidance on the conduct and purpose of HWS  King’s Fund  King’s fund guidance or research. For example: ‘These findings were reinforced in research by the King’s Fund which looked at four key lifestyle behaviours’  Local Evaluation/existing service  A named evaluation of a service which may be formally referenced. For example: ‘evaluation of Change 4 Life in xxxx showed that…’. Note where there is no reference to a service and there is no other reference, this should go as unreferenced local statistic  External individual expert—not institutional  This refers to input of a local expert—e.g. the Local Police and Crime Commissioner views x as problematic. Where an institution is named—e.g. Hertfordshire police view x as problematic—the code below was used  External expertise—institutional  This includes institutional or academic expertise that is not otherwise disaggregated—e.g. academics from UCL tell us xxxx  Academic research study  Includes journal articles and reports produced by academic institutions—e.g. UCL produced reports—also are included here  Systematic review  Any systematic review (including meta-analyses)—also put in scoping review here  Joint Strategic Needs Assessment  Joint Strategic Needs Assessment  Specific Locally commissioned research  This includes any form of research that has not been captured elsewhere that is carried out locally (e.g. at LA level or lower geography)—interviews, consultations, surveys with open ended questions, surveys that may not be representative or weighted etc – this research is usually carried out by the LA/HWB or commissioned by the LA/HWB  Specified/referenced national statistics  National statistics that have a source—e.g. DWP figures show household income in the UK declined over the past decade  Unspecified/unreferenced national statistics  National statistics not captured elsewhere—e.g. figures show wellbeing in the UK declined over the past decade  National Policy or Reports from National Agencies (Not NICE/PHE)  This includes policy from DH, DCLG or DfE. For example ‘Mental health (and emotional wellbeing) is everyone’s business: individuals, families, employers, educators and communities all need to play their part. (No Health without Mental Health: A cross government mental health outcomes strategy for people of all ages’. (DH 2011))  Specific commissioning tool  Specific commissioning tools. For example: ‘Early Help delivery tool, including the training and support to schools and lead professionals’  Other Local Policy or Strategy  References to other local strategies and plans that support the commissioning or policy-setting  Unspecified Research/Evidence  Evidence with otherwise no source or context e.g. ‘Evidence also suggests that there will be consequences of the economic downturn and the welfare reforms in relation to mental health such as: impact on individuals due to rise in unemployment causing mental health problems; shrinking economy, offering fewer job opportunities and exacerbating poverty and social exclusion; potential reductions in service provision.’  Other  Other forms of evidence contributing to the HWS not captured elsewhere  Type  Working definition  Office for National Statistics (ONS) Statistics  Estimates of population size or population growth or population characteristics deriving from the census or other sources. All evidence on population size was classed under this code. Example: ‘The Office for National Statistics estimates the current population of xxxxx at 231 900’  Specified/referenced local statistics  Statistics about the local population that are either formally referenced or a source is named. For example ‘our residents’ survey in 2014 showed that…’  Unspecified/unreferenced local statistics  Statistics about the local population that are not referenced in any way (either date or source). For example: ‘Significant parts of the housing stock in xxx are in poor condition, with an estimated xxxx properties non-decent’  National Institute for Health and Care Excellence (NICE) Guidance  NICE Guidance or other publication from NICE. For example ‘this strategy is aligned with NICE Guidelines on Anxiety and Depression’. NICE Guidance is usually informed by systematic reviews on the issue and expert and stakeholder consultation  Public Health England (PHE) Guidance or Strategy  PHE formal guidance or reports. Note instances of drawing upon the Public Health Outcomes Framework (PHOF) were cited as drawing on ‘national statistics’ (below) given that PHOF guidance was originally issued by the Department of Health. Many of the latter instances represented intended usage, as opposed to usage that informed the HWS itself  Local Government Association (LGA)  LGA guidance or research; for example LGA guidance on the conduct and purpose of HWS  King’s Fund  King’s fund guidance or research. For example: ‘These findings were reinforced in research by the King’s Fund which looked at four key lifestyle behaviours’  Local Evaluation/existing service  A named evaluation of a service which may be formally referenced. For example: ‘evaluation of Change 4 Life in xxxx showed that…’. Note where there is no reference to a service and there is no other reference, this should go as unreferenced local statistic  External individual expert—not institutional  This refers to input of a local expert—e.g. the Local Police and Crime Commissioner views x as problematic. Where an institution is named—e.g. Hertfordshire police view x as problematic—the code below was used  External expertise—institutional  This includes institutional or academic expertise that is not otherwise disaggregated—e.g. academics from UCL tell us xxxx  Academic research study  Includes journal articles and reports produced by academic institutions—e.g. UCL produced reports—also are included here  Systematic review  Any systematic review (including meta-analyses)—also put in scoping review here  Joint Strategic Needs Assessment  Joint Strategic Needs Assessment  Specific Locally commissioned research  This includes any form of research that has not been captured elsewhere that is carried out locally (e.g. at LA level or lower geography)—interviews, consultations, surveys with open ended questions, surveys that may not be representative or weighted etc – this research is usually carried out by the LA/HWB or commissioned by the LA/HWB  Specified/referenced national statistics  National statistics that have a source—e.g. DWP figures show household income in the UK declined over the past decade  Unspecified/unreferenced national statistics  National statistics not captured elsewhere—e.g. figures show wellbeing in the UK declined over the past decade  National Policy or Reports from National Agencies (Not NICE/PHE)  This includes policy from DH, DCLG or DfE. For example ‘Mental health (and emotional wellbeing) is everyone’s business: individuals, families, employers, educators and communities all need to play their part. (No Health without Mental Health: A cross government mental health outcomes strategy for people of all ages’. (DH 2011))  Specific commissioning tool  Specific commissioning tools. For example: ‘Early Help delivery tool, including the training and support to schools and lead professionals’  Other Local Policy or Strategy  References to other local strategies and plans that support the commissioning or policy-setting  Unspecified Research/Evidence  Evidence with otherwise no source or context e.g. ‘Evidence also suggests that there will be consequences of the economic downturn and the welfare reforms in relation to mental health such as: impact on individuals due to rise in unemployment causing mental health problems; shrinking economy, offering fewer job opportunities and exacerbating poverty and social exclusion; potential reductions in service provision.’  Other  Other forms of evidence contributing to the HWS not captured elsewhere  Different synthesis methods were employed: (i) using the coding template, we provided a narrative account of the context in which evidence was used, focussing on key themes and their recurrence across strategies; (ii) we undertook further analyses through tabulations and charts; (iii) third, we undertook Qualitative Comparative Analyses (QCA). QCA has its basis in set-theoretic logic, and is well-placed in synthesizing data from a small number of cases to uncover complex configurations of conditions observed to overlap with an outcome.11 QCA was used primarily to understand: (i) the HWB characteristics overlapping with evidence use; as well as (ii) exploring overlaps between evidence use patterns and HWB characteristics with LA public health spending allocations. We followed guidance provided elsewhere11 in creating our QCA solution, and ran a series of diagnostic tests to ensure the quality of our solution (no contradictory configurations were detected and no contradictory simplifying assumptions were made in accounting for logical remainders (combinations of conditions not supported by observed cases)). We do not name HWBs and avoid disclosing their identity in our results; while HWSs are public documents, their inclusion in analyses may not have been foreseen by HWBs and it was not within the scope of this work to contact HWBs for further information or clarifications. Results The health of health and wellbeing strategies From our sample of HWBs, we located a strategy published between 2012 and 2014 and an updated strategy published between 2015 and March 2017 for just under half of areas (25/51; see Table 3). One strategy alone was located for 26 HWBs (23 published 2012–14 and 3 published 2015–17). Almost a quarter of HWBs (23.5%) did not have a current strategy in March 2017, with strategies having either expired with no successor (10/12 HWBs) or having been published in 2013/14 with no expiry date (2/12 HWBs). Given that HWBs are required to publish HWSs to outline their long-term direction, this appears to compromise the significance of HWSs (Table 2). The absence of a current strategy was not patterned by the characteristics of the HWB area (e.g. level of deprivation). Table 2 Descriptive characteristics of Health and Wellbeing board areas and the presence of Health and Wellbeing Strategies ID number  HWS discovered published 2013/14  HWS discovered published 2015/16  HWB HAS current HWS in place  Quintile of index of multiple deprivation (1 = lest deprived)  Quintile proportion of older people aged 65 + (1 = fewest older people)  Quintile proportion of black and minority peope (1 = fewest BME people)  Quintile male life exoectancy (1 = lowest life expectancy)  Quintile female life exoectancy (1 = lowest life expectancy)  Rurality  1  Y    No current HWS  1  3  4  5  5  Urban with Major Conurbation  2  Y  Y    1  5  1  5  5  Largely Rural (rural including hub towns 50–79%)  3  Y      1  5  2  4  5  Urban with Significant Rural (rural including hub towns 26–49%)  4  Y  Y    1  2  5  5  5  Urban with Major Conurbation  5    Y  No current HWS  1  1  5  5  5  Urban with Major Conurbation  6  Y  Y    1  5  1  4  4  Largely Rural (rural including hub towns 50–79%)  7  Y  Y    1  5  2  5  5  Mainly Rural (rural including hub towns ≥80%)  8  Y    No current HWS  1  3  2  5  5  Urban with City and Town  9  Y  Y    1  3  3  3  4  Urban with City and Town  10  Y  Y    2  2  5  5  5  Urban with Major Conurbation  11  Y    No current HWS  2  3  4  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  12  Y    No current HWS  2  3  4  5  4  Urban with Major Conurbation  13  Y  Y    2  5  2  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  14  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  15  Y  Y    2  4  3  4  4  Urban with Major Conurbation  16  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  17  Y  Y    2  1  4  3  3  Urban with City and Town  18  Y      2  .  .  .  .    19  Y  Y    2  4  3  3  3  Urban with Major Conurbation  20  Y    No current HWS  2  3  4  4  3  Urban with Major Conurbation  21  Y  Y    2  3  1  1  2  Urban with City and Town  22  Y      3  3  3  2  2  Urban with Major Conurbation  23  Y  Y    3  1  5  5  5  Urban with Major Conurbation  24  Y  Y    3  5  1  3  3  Largely Rural (rural including hub towns 50–79%)  25  Y    No current HWS  3  4  2  2  2  Urban with City and Town  26  Y  Y    3  1  5  4  4  Urban with Major Conurbation  27    Y    3  1  5  2  3  Urban with Major Conurbation  28  Y  Y    3  5  2  3  3  Mainly Rural (rural including hub towns ≥80%)  29  Y      3  2  4  2  3  Urban with Major Conurbation  30  Y      3  4  2  3  2  Urban with Significant Rural (rural including hub towns 26–49%)  31  Y    No current HWS  3  5  1  2  3  Largely Rural (rural including hub towns 50–79%)  32  Y  Y    3  1  4  3  3  Urban with Major Conurbation  33  Y      3  5  1  .  .  Mainly Rural (rural including hub towns ≥80%)  34  Y    No current HWS  3  3  1  1  1  Urban with Major Conurbation  35  Y  Y    4  4  1  2  2  Urban with Minor Conurbation  36    Y    4  1  5  2  2  Urban with City and Town  37  Y  Y    4  2  4  2  2  Urban with City and Town  38  Y    No current HWS  4  3  2  2  2  Urban with City and Town  39  Y      4  2  3  2  2  Urban with City and Town  40  Y    No current HWS  4  2  4  3  2  Urban with City and Town  41  Y  Y    4  1  5  4  3  Urban with Major Conurbation  42  Y  Y    5  1  5  1  1  Urban with City and Town  43  Y  Y    5  5  2  1  1  Urban with City and Town  44  Y      5  2  1  1  1  Urban with City and Town  45  Y      5  4  1  1  1  Urban with City and Town  46  Y  Y    5  2  3  1  1  Urban with City and Town  47  Y  Y    5  1  5  1  1  Urban with Major Conurbation  48  Y  Y    5  2  4  1  1  Urban with Major Conurbation  49  Y    No current HWS  5  2  3  1  1  Urban with Major Conurbation  50  Y  Y    5  3  3  1  1  Urban with City and Town  51  Y      5  4  2  2  2  Urban with Major Conurbation  ID number  HWS discovered published 2013/14  HWS discovered published 2015/16  HWB HAS current HWS in place  Quintile of index of multiple deprivation (1 = lest deprived)  Quintile proportion of older people aged 65 + (1 = fewest older people)  Quintile proportion of black and minority peope (1 = fewest BME people)  Quintile male life exoectancy (1 = lowest life expectancy)  Quintile female life exoectancy (1 = lowest life expectancy)  Rurality  1  Y    No current HWS  1  3  4  5  5  Urban with Major Conurbation  2  Y  Y    1  5  1  5  5  Largely Rural (rural including hub towns 50–79%)  3  Y      1  5  2  4  5  Urban with Significant Rural (rural including hub towns 26–49%)  4  Y  Y    1  2  5  5  5  Urban with Major Conurbation  5    Y  No current HWS  1  1  5  5  5  Urban with Major Conurbation  6  Y  Y    1  5  1  4  4  Largely Rural (rural including hub towns 50–79%)  7  Y  Y    1  5  2  5  5  Mainly Rural (rural including hub towns ≥80%)  8  Y    No current HWS  1  3  2  5  5  Urban with City and Town  9  Y  Y    1  3  3  3  4  Urban with City and Town  10  Y  Y    2  2  5  5  5  Urban with Major Conurbation  11  Y    No current HWS  2  3  4  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  12  Y    No current HWS  2  3  4  5  4  Urban with Major Conurbation  13  Y  Y    2  5  2  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  14  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  15  Y  Y    2  4  3  4  4  Urban with Major Conurbation  16  Y      2  4  3  4  4  Urban with Significant Rural (rural including hub towns 26–49%)  17  Y  Y    2  1  4  3  3  Urban with City and Town  18  Y      2  .  .  .  .    19  Y  Y    2  4  3  3  3  Urban with Major Conurbation  20  Y    No current HWS  2  3  4  4  3  Urban with Major Conurbation  21  Y  Y    2  3  1  1  2  Urban with City and Town  22  Y      3  3  3  2  2  Urban with Major Conurbation  23  Y  Y    3  1  5  5  5  Urban with Major Conurbation  24  Y  Y    3  5  1  3  3  Largely Rural (rural including hub towns 50–79%)  25  Y    No current HWS  3  4  2  2  2  Urban with City and Town  26  Y  Y    3  1  5  4  4  Urban with Major Conurbation  27    Y    3  1  5  2  3  Urban with Major Conurbation  28  Y  Y    3  5  2  3  3  Mainly Rural (rural including hub towns ≥80%)  29  Y      3  2  4  2  3  Urban with Major Conurbation  30  Y      3  4  2  3  2  Urban with Significant Rural (rural including hub towns 26–49%)  31  Y    No current HWS  3  5  1  2  3  Largely Rural (rural including hub towns 50–79%)  32  Y  Y    3  1  4  3  3  Urban with Major Conurbation  33  Y      3  5  1  .  .  Mainly Rural (rural including hub towns ≥80%)  34  Y    No current HWS  3  3  1  1  1  Urban with Major Conurbation  35  Y  Y    4  4  1  2  2  Urban with Minor Conurbation  36    Y    4  1  5  2  2  Urban with City and Town  37  Y  Y    4  2  4  2  2  Urban with City and Town  38  Y    No current HWS  4  3  2  2  2  Urban with City and Town  39  Y      4  2  3  2  2  Urban with City and Town  40  Y    No current HWS  4  2  4  3  2  Urban with City and Town  41  Y  Y    4  1  5  4  3  Urban with Major Conurbation  42  Y  Y    5  1  5  1  1  Urban with City and Town  43  Y  Y    5  5  2  1  1  Urban with City and Town  44  Y      5  2  1  1  1  Urban with City and Town  45  Y      5  4  1  1  1  Urban with City and Town  46  Y  Y    5  2  3  1  1  Urban with City and Town  47  Y  Y    5  1  5  1  1  Urban with Major Conurbation  48  Y  Y    5  2  4  1  1  Urban with Major Conurbation  49  Y    No current HWS  5  2  3  1  1  Urban with Major Conurbation  50  Y  Y    5  3  3  1  1  Urban with City and Town  51  Y      5  4  2  2  2  Urban with Major Conurbation  Typically, HWSs covered a period of 3–4 years, although some covered longer periods (12/76 strategies reviewed in total) of up to 5, 6 (one) and 10 (one). A minority of updated strategies stated that the update was based on the previous strategy (Table 2: IDs 32, 36 and 41); while another 10 mentioned the existence of a previous strategy. No instance was recorded where an in-depth evaluation of the previous HWS had taken place, and no decisions or priorities included in the updated HWSs appeared to be based upon an evaluation of the previous strategy. Frequency of different types of evidence use A preference for statistical data Of the 76 HWSs reviewed in total (published in 2013/14 and 2015/16), only one was informed by a single type of evidence; others were informed by up to 13 types. HWSs frequently drew upon statistical sources, mainly to demonstrate evidence of need (Fig. 1). As expected, the Joint Strategic Needs Assessment (JSNA(s)) informed the majority of HWSs, particularly among HWSs published in 2013/14 (47/48 HWSs (98%)). By the second wave of HWSs, their role appeared to attenuate slightly, although they remained the most frequently cited source of evidence (23/28 strategies (82%)). JNSAs appear to be fulfilling their intended role in clarifying the health needs and status of the local population. However, JSNAs typically represent a statistical compendium and it is perhaps unsurprising, given that JSNAs formed a bedrock of many HWSs, that no HWS presented evidence from existing qualitative research studies that had been collected and analysed robustly. Alongside statistics from JSNAs, many HWSs used statistical data on local conditions that were unreferenced (27/48 in 2013/14 and 16/28 in 2015/16). Such data typically contained no source, no date, and sometimes were unclear on the scope and population. A typical example was ‘although the trend in life expectancy is upward, there is a 2.5-year gap between males and females across [HWB], with more inequalities in disadvantaged communities’ (HWS 14). Similar patterns were observed with unreferenced national-level statistics used to compare the HWB position with the national picture. Fig. 1 View largeDownload slide Types of evidence used in Health and Wellbeing Strategies (% of strategies with at least one mention). Fig. 1 View largeDownload slide Types of evidence used in Health and Wellbeing Strategies (% of strategies with at least one mention). National statistics came from a number of sources including outcome framework statistics. Furthermore, a number of HWSs specifically included plans to use Public Health Outcomes Framework data to monitor the success and impact of the HWS, even if these were not directly used to directly inform the direction of the HWS (10 HWSs). A typical example included: ‘we will measure our success by monitoring local progress against key indicators or measures from the three recently published national outcomes frameworks for Public Health, The NHS and Adult Social Care’ (ID: 17). While the targets themselves were often left unspecified, many HWSs emphasized the need to monitor progress and how this could be done, even if this ambition was often left unfulfilled (see above). Trusted academic sources Academic research studies were frequently cited sources of evidence, although were often confined to a limited number of ‘trusted’ or accessible sources. Approximately, half of HWSs included at least one source of academic research published within a peer-reviewed journal or published as a report by an academic institution. There was no apparent difference in the use of academic sources between HWSs published in 2013/14 and 2015/16. One of the most commonly used sources was the Marmot Review (Fair Society, Healthy Lives)12 used in 23 HWSs. Along with Dahlgren and Whitehead’s report13 this was frequently used to justify a focus on health inequalities and social determinants of health. As was the case in previous explorations,3 NICE guidance and systematic reviews were rarely cited and there was no evidence that there had been any increase in usage over time. Inclusion of academic ‘evidence’ was not synonymous with presenting evidence that adhered to academic writing conventions. Typically this could include omitting the source and date of the evidence, for instance ‘a recent study by the Wisconsin University showed the rankings of factors which determined the best health outcomes for a population’ (ID: 43). Local research and evidence gathering While HWSs tended to rely more heavily on published statistical data than published qualitative data, there were indications that engagement activities were making a contribution. Almost half of HWSs included reference to specific local research (20/48 published in 2013/14 and 14/28 published in 2015/16); often this included consultation and engagement activities and involved taking a draft set of priorities for consultation; for example, ‘we consulted widely on these proposals, listened to what people said, undertook an initial Equalities Impact Assessment and used this to inform the draft strategy’ (ID: 13). Few HWSs that had conducted consultative exercises presented the findings alongside the HWSs. There was also indicative evidence that consultation exercises were a more frequent occurrence among HWSs published in 2015/16 (9/28 HWSs) than in 2013/14 (10/48 HWSs). This increase could be symptomatic of the trend towards increased local accountability and politicization said to be occurring in public health decision-making.14 Area characteristics, health priorities, patterns of evidence use There was a little evidence of systematic overlap between the characteristics of HWB areas—including their demographic characteristics (e.g. proportion of older people and proportion of black and ethnic minority residents), rurality, life expectancy and level of deprivation (based on the index of multiple deprivation)—and differential patterns of evidence use within the HWS. Similarly, there were no clear overlaps between different health priorities in an area and patterns of evidence use. Area characteristics, patterns of evidence use and spending allocations Funding for public health activities through ring-fenced public health allocations has been substantially reduced since the HSCA.15 Initial allocations principally reflected the degree to which local systems invested historically in preventive services, and the standardized mortality ratio for people aged 75 and under. The latter was intended to capture health inequalities as well as socioeconomic deprivation.15,16 We explored the extent to which the differential use of robust sources of evidence (reflecting academic sources and NICE guidelines) were observed to overlap with high public health spending allocations per person (projected for 2017/18).17 Alongside robust evidence sources, we also explored the extent to which the HWS included evidence from specific local research and evidence and guidance produced by national-level stakeholders (e.g. evidence from PHE, the LGA, among others), as well as area characteristics, and how these overlapped with spending allocations. Because of the nature of the indicators we employed crisp-set QCA, converting each factor into a binary measure. Here, we explore the extent to which evidence use and area characteristics (highest two quintiles of deprivation and highest quintile of male life expectancy) overlapped with being in the highest quintile of public health allocated spending; each quintile was calculated on the basis of national ranking and not the ranking within the sample. Because of their proximity to the outcome, we focussed only on HWSs published in 2015/16 in these analyses (n = 28). Given that allocations have their basis in existing public health commitments as well as deprivation/inequality, it is not unsurprising that having above average levels of deprivation and/or low levels of life expectancy were frequently observed conditions among areas with high levels of public health spending allocations. However, deprived areas with high levels of public health spending allocations were also supported by HWSs informed by robust evidence sources (Table 3). This indicates that areas with the highest public health challenge, and the highest budget allocations, were drawing on robust sources to inform health and wellbeing strategies; these configurations accounted for three-quarters of instances of high spending allocations observed. Areas with pre-existing (prior to HSCA 2012) large public health commitments may therefore have a more established culture of evidence use. Those areas with the highest quintile of public health spending allocations but that were not highly deprived did not draw upon ‘robust’ sources of evidence in the HWS. In deprived areas, therefore, the use of robust evidence sources was a sufficient condition overlapping with the highest quintile of public health spending; for those areas that did not have high levels of deprivation or low life expectancy; use of robust sources was not a sufficient condition. The truth table (Table 3) also showed that a number of areas with relatively high levels of deprivation were not included within the highest quintile of public health spending allocations; crucially these same areas did not include robust sources within their HWSs, indicating a less developed evidence use culture within those very areas likely facing the most substantial public health challenges and the fewest resources to tackle these. Table 3 Truth table and intermediate solution examining public health allocations Truth table (Notes: the truth table here displays logical combinations of conditions observed in the data and the outcome; each row represents a combination of conditions)  Conditions reflecting evidence use  Conditions reflecting local characteristics  Outcome  Number of HWSs with observed combination  Indicators of solution fit  Robust evidence sources included in HWS (1 = yes; 0 = No)  Specific Local Research included in HWS (1 = yes; 0 = No)  Reference to JSNA included in HWS (1 = yes; 0 = No)  Reference to evidence/guidance from national stakeholders in HWS (1 = yes; 0 = No)  Most deprived 40% (Index of Multiple Deprivation) (1 = yes; 0 = No)  Male life expectancy in the lowest quintile (1=yes; 0=No)  Outcome (top quintile of public health allocations per person) (1 = yes; 0 = No)  Number of HWSs in configuration  Consistency score with sub-set relationship  Proportional reduction in inconsistency  1  0  1  1  1  1  1  2  1  1  1  1  1  1  1  1  1  2  1  1  0  0  1  1  0  0  1  1  1  1  0  1  0  1  0  0  1  1  1  1  1  1  0  0  1  1  1  1  1  1  1  1  1  1  1  0  1  1  1  1  0  1  1  1  0  0  0  3  0  0  1  1  1  1  0  0  0  3  0  0  0  0  1  0  1  0  0  2  0  0  0  1  1  0  0  0  0  2  0  0  1  0  1  1  0  0  0  2  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  1  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  1  1  0  1  0  0  0  0  1  1  1  0  0  1  0  0  1  0  0  0  0  0  0  1  0  0  1  0  1  0  0  0  0  1  0  0  1  1  1  1  0  1  0  1  0  0  Truth table (Notes: the truth table here displays logical combinations of conditions observed in the data and the outcome; each row represents a combination of conditions)  Conditions reflecting evidence use  Conditions reflecting local characteristics  Outcome  Number of HWSs with observed combination  Indicators of solution fit  Robust evidence sources included in HWS (1 = yes; 0 = No)  Specific Local Research included in HWS (1 = yes; 0 = No)  Reference to JSNA included in HWS (1 = yes; 0 = No)  Reference to evidence/guidance from national stakeholders in HWS (1 = yes; 0 = No)  Most deprived 40% (Index of Multiple Deprivation) (1 = yes; 0 = No)  Male life expectancy in the lowest quintile (1=yes; 0=No)  Outcome (top quintile of public health allocations per person) (1 = yes; 0 = No)  Number of HWSs in configuration  Consistency score with sub-set relationship  Proportional reduction in inconsistency  1  0  1  1  1  1  1  2  1  1  1  1  1  1  1  1  1  2  1  1  0  0  1  1  0  0  1  1  1  1  0  1  0  1  0  0  1  1  1  1  1  1  0  0  1  1  1  1  1  1  1  1  1  1  1  0  1  1  1  1  0  1  1  1  0  0  0  3  0  0  1  1  1  1  0  0  0  3  0  0  0  0  1  0  1  0  0  2  0  0  0  1  1  0  0  0  0  2  0  0  1  0  1  1  0  0  0  2  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  1  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  0  0  0  1  0  1  1  0  1  0  0  0  0  1  1  1  0  0  1  0  0  1  0  0  0  0  0  0  1  0  0  1  0  1  0  0  0  0  1  0  0  1  1  1  1  0  1  0  1  0  0  Configuration  Intermediate solution (Notes: represents simplification of truth table after Boolean minimization)  Consistency score with sub-set relationship  Proportional reduction in inconsistency  Raw coverage  Unique coverage  HWB ID  1  Specific local research*~ JSNA* National stakeholder evidence/guidance  1  1  0.125  0.125  23  2  Robust evidence * Specific local research *Highly deprived area* Low male life expectancy  1  1  0.375  0.125  42,47,50  3  Robust evidence * Specific local research * National stakeholder evidence/guidance * Highly deprived area  1  1  0.375  0.125  41,42,50  4  Robust evidence *JSNA * National stakeholder evidence/guidance * Highly deprived area * Low male life expectancy  1  1  0.5  0.25  42,43,46,50  5  ~Robust evidence * ~ Specific local research * JSNA * National stakeholder evidence/guidance *~ Highly deprived area  1  1  0.125  0.125  27    M1  1  1  1      Configuration  Intermediate solution (Notes: represents simplification of truth table after Boolean minimization)  Consistency score with sub-set relationship  Proportional reduction in inconsistency  Raw coverage  Unique coverage  HWB ID  1  Specific local research*~ JSNA* National stakeholder evidence/guidance  1  1  0.125  0.125  23  2  Robust evidence * Specific local research *Highly deprived area* Low male life expectancy  1  1  0.375  0.125  42,47,50  3  Robust evidence * Specific local research * National stakeholder evidence/guidance * Highly deprived area  1  1  0.375  0.125  41,42,50  4  Robust evidence *JSNA * National stakeholder evidence/guidance * Highly deprived area * Low male life expectancy  1  1  0.5  0.25  42,43,46,50  5  ~Robust evidence * ~ Specific local research * JSNA * National stakeholder evidence/guidance *~ Highly deprived area  1  1  0.125  0.125  27    M1  1  1  1      Key: ~ = absence of condition; * = logical ‘AND’. Discussion and Conclusion Summary and limitations Evidence is clearly intended to be at the core of HWSs,2 and we found that HWSs often drew upon a diverse range of sources to justify priorities and actions. However, in line with previous explorations,3 the use of evidence is often not straightforward. Academic sources are frequently confined to a limited set of ‘trusted’ sources; local and national statistics are repeatedly used in ways that prevent verification, benchmarking or understanding trends over time; and the use of peer-reviewed qualitative research/evidence was found to be absent. Although there were indications that many HWSs were informed by consultative exercises, the full findings were not presented. New HWSs were being published during periods of substantial cuts in public health spending,15 although key public health challenges remain substantial.18 QCA analyses suggested that areas with the largest public health challenges and the largest spending allocations were drawing on robust sources of evidence in their HWSs. In contrast, areas that did not have the largest spending allocation but were nevertheless facing substantial public health challenges were not drawing on robust sources in the HWS. Arguably, the judicious use of evidence becomes all the more important when resources are scarce,10 and the results suggest that an evidence use culture may be underdeveloped in many areas of higher deprivation that do not receive the greatest funding. A number of caveats accompany the results including that double data extraction did not occur for the majority of the 76 HWSs reviewed (except for piloting), that area characteristics were based on broad indicators/quintiles; and that unreported or unreferenced sources may also have contributed to HWS development. This latter consideration is a particular disadvantage of documentary analysis, although HWSs are expected to demonstrate which evidence has been considered.2 Implications The results give a complex picture of evidence use in local public health decision-making and priority setting. While there is a thriving evidence use culture, this evidence tends to be drawn from a narrow pool of sources and is not used in the most transparent way. The exclusion of published qualitative literature is concerning as it suggests that gaining an understanding of the mechanisms driving health inequalities and how interventions to tackle these ‘work’ is not prioritized. A recent review uncovered that research evidence often did not meet local authorities’ requirements with respect to locality, political salience and economic focus.10 The results here suggest that where evidence meeting these needs is unavailable, this is often be supplemented by locally conducted research, the methodological quality of which remains unknown. This finding may partially explain why previous studies find that the word ‘evidence’ in public health decision-making is not always synonymous with drawing upon a robust body of research evidence.19 Nevertheless, it is clear that improving the local salience of academic research evidence is central to enhancing its contribution to local decision-making. This may involve enhancing research capacity within LAs/HWBs, developing (statistical and other) methods of enhancing the generalizability of evidence to local areas, and/or improving knowledge translation practices. With respect to the latter, allied work being conducted by the authors with LA public health teams suggests that continuous dialogue between evidence generators and decision-makers would facilitate more effective use of evidence. It is notable that the composition of HWBs usually does not include representation from knowledge brokers; the results here suggest that this absence may be felt more keenly among HWBs facing substantial public health challenges but who will not receive the greatest resources to meet these. Acknowledgements We would like to acknowledge the helpful input of the editors of the journal and the special issue in shaping this paper. We would also like to acknowledge the input of all the stakeholders in our qualitative research project (currently underway) in helping to make sense of evidence use in public health decision-making. Funding The research was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. References 1 Greaves Z, McCafferty S. Health and wellbeing boards: public health decision making bodies or political pawns? Public Health  2017; 143: 78– 84. Google Scholar CrossRef Search ADS PubMed  2 DH 2013, Statutory Guidance on Joint Strategic Needs Assessments and Joint Health and Wellbeing Strategies, ( London: Department of Health). 3 Beenstock J, Sowden S, Hunter DJ et al.  . Are health and well-being strategies in England fit for purpose? A thematic content analysis. J Public Health (Bangkok)  2014; 37: 461– 9. Google Scholar CrossRef Search ADS   4 Diabetes UK. Are Health and Wellbeing Boards Taking Account of Diabetes? A Review of JSNAs and Health and Wellbeing Strategies . London: Diabetes UK, 2014. 5 Scrutton J. A Place for Parity: Health and Wellbeing Boards and Mental Health . London: Mental Health Partnerships, 2013. 6 Health and wellbeing boards directory [database on the Internet]. The King’s Fund. 2016. Available from: https://www.kingsfund.org.uk/projects/health-and-wellbeing-boards/hwb-map (31 August 2016, date last accessed). 7 StataCorp. Stata Statistical Software: Release 13 . College Station, TX: StataCorp LP, 2013. 8 Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4.0: Software for Research Synthesis . London: Institute of Education, 2010. 9 Clarke A, Taylor-Phillips S, Swan J et al.  . Evidence-based commissioning in the English NHS: who uses which sources of evidence? A survey 2010/2011. BMJ Open  2013; 3: e002714. Google Scholar CrossRef Search ADS PubMed  10 Kneale D, Rojas-García A, Raine R et al.  . The use of evidence in English local public health decision-making. Implement Sci  2017; 12: 53. Google Scholar CrossRef Search ADS PubMed  11 Thomas J, O’Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev  2014; 3: 1– 14. Google Scholar CrossRef Search ADS PubMed  12 Marmot M, Allen J, Goldblatt P et al.  . The Marmot review: Fair society, healthy lives. Strategic review of health inequalities in England post-2010 Crown Copyright, London: The Marmot Review. 2010. 13 Dahlgren G, Whitehead M. Policies and Strategies to Promote Social Equity in Health . Stockholm: Institute for Futures Studies, 1991. 14 Marks L, Hunter D, Scalabrini S et al.  . The return of public health to local government in England: changing the parameters of the public health prioritization debate? Public health  2015; 129: 1194– 203. Google Scholar CrossRef Search ADS PubMed  15 FPH. UK Faculty of Public Health Response to Department of Health Consultation on the Public Health Grant: Proposed Target Allocation Formula for 2016/17 . London: Faculty of Public Health, 2016. 16 London Councils. Public Health Funding Allocations—London Councils . London: London Councils, 2013. 17 DH. Public Health Grants to Local Authorities: 2016 to 2017 . London: Department of Health, 2016. 18 Davies A, Keeble E, Bhatia T et al.  . Public Health and Prevention: Research Summary . London: Quality Watch (Nuffield Foundation and The Health Foundation), 2016. 19 Katikireddi SV, Higgins M, Bond L et al.  . How evidence based is English public health policy? Br Med J  2011; 343: d7310. Google Scholar CrossRef Search ADS   © The Author 2018. Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Journal of Public HealthOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off