How are evidence and knowledge used in orthopaedic decision-making? Three comparative case studies of different approaches to implementation of clinical guidance in practice

How are evidence and knowledge used in orthopaedic decision-making? Three comparative case... Background: The uptake and use of clinical guidelines is often insufficient to change clinical behaviour and reduce variation in practice. As a consequence of diverse organisational contexts, the simple provision of guidelines cannot ensure fidelity or guarantee their use when making decisions. Implementation research in surgery has focused on understanding what evidence exists for clinical practice decisions but limits understanding to the technical, educational and accessibility issues. This research aims to identify where, when and how evidence and knowledge are used in orthopaedic decision-making and how variation in these factors contributes to different approaches to implementation of clinical guidance in practice. Methods: We used in-depth case studies to examine guideline implementation in real-life surgical practice. We conducted comparative case studies in three English National Health Service hospitals over a 12-month period. Each in-depth case study consisted of a mix of qualitative methods including interviews, observations and document analysis. Data included field notes from observations of day-to-day practice, 64 interviews with NHS surgeons and staff and the collection of 121 supplementary documents. Results: Case studies identified 17 sources of knowledge and evidence which influenced clinical decisions in elective orthopaedic surgery. A comparative analysis across cases revealed that each hospital had distinct approaches to decision-making. Decision-making is described as occurring as a result of how 17 types of knowledge and evidence were privileged and of how they interacted and changed in context. Guideline implementation was contingent and mediated through four distinct contextual levels. Implementation could be assessed for individual surgeons, groups of surgeons or the organisation as a whole, but it could also differ between these levels. Differences in how evidence and knowledge were used contributed to variations in practice from guidelines. (Continued on next page) * Correspondence: A.L.Grove@warwick.ac.uk Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Grove et al. Implementation Science (2018) 13:75 Page 2 of 14 (Continued from previous page) Conclusion: A range of complex and competing sources of evidence and knowledge exists which influence the working practices of healthcare professionals. The dynamic selection, combination and use of each type of knowledge and evidence influence the implementation and use of clinical guidance in practice. Clinical guidelines are a fundamental part of practice, but represent only one type of evidence influencing clinical decisions. In the orthopaedic speciality, other distinct sources of evidence and knowledge are selected and used which impact on how guidelines are implemented. New approaches to guideline implementation need to appreciate and incorporate this diverse range of knowledge and evidence which influences clinical decisions and to take account of the changing contexts in which decisions are made. Keywords: Guidelines, Implementation, Orthopaedic surgery, Evidence-based medicine, Comparative case study Background professional and environmental factors which have been Across the world, policymaking organisations exist to pro- described previously, but also requires a description of the duce clinical guidance and recommendations for health- local practice context and wider healthcare sector in care, which are based on scientific evidence. In the UK, sufficient detail to facilitate more effective guideline these organisations are the National Institute for Health implementation. Therefore, the aim of this research was and Care Excellence (NICE) and the Scottish Intercollegi- to identify where, when and how evidence and knowledge ate Guidelines Network (SIGN) [1, 2]. Together, they pro- are used in healthcare decision-making and how variation vide evidence-based guidance on the most effective ways in these factors contributes to different approaches to to diagnose, treat and prevent poor health for National implementation of clinical guidance in practice. Health Service (NHS) patients. One of the aims of clinical guidance is to reduce variation in practice, and therefore, Over and above clinical guidelines limit inequalities in service delivery [3]. However, previous Achieving effective guideline implementation reflects research has established that healthcare organisations face not only the people involved, but also their professional several challenges to implementation [4–6]. roles, their positions in the organisation and the epi- In 2014, a review of systematic reviews explored factors stemic communities to which they belong [10, 11]. which influence guideline implementation and uptake [7]. Producers and users of evidence and guidelines often sit The findings report a range of factors including those as- on different sides of the social, scientific and clinical sociated with guidelines themselves (e.g., complexity), with boundaries. Therefore, they possess varying types of healthcare professionals (e.g., lack of awareness) and with knowledge which may mean that they privilege evidence the working environment such as limited time, personnel differently. This can make integration across these com- and resources devoted to support guideline adherence [7]. munities challenging [11]. Day-to-day decision-making Limited information was provided to describe the context by healthcare professionals requires the selection of many of guideline implementation in detail or the differences in types of knowledge and evidence, situated within local, healthcare environments which might impact implemen- contextual and social circumstances [12]. For example, tation, such as differences in the processes of healthcare research knowledge coexists with the lived experience of delivery and national policy decisions. Evidence exists patients and the macro healthcare initiatives and incen- describing the barriers to implementation of clinical tives from policymakers and regulatory bodies. guidance and the rates of guideline uptake across a Preserving clinical autonomy and medical judgement by range of diseases [3–5, 8, 9]. For example, lack of time these professional groups is also recognised as important, was identified as the most commonly reported barrier particularly when guidelines challenge traditional practices to implementation [5]. However, too few studies rigor- [13, 14]. In this study, we examine guideline implementa- ously assess the effectiveness of approaches to improve tion in the context of real-life NHS practice. We investigate implementation of clinical guidelines or explicitly de- guideline implementation problems through the application scribe factors which enable their uptake and use [6, 7]. of a range of qualitative methods to explore where, when Rates of guideline uptake provide a proxy for guideline and how evidence and knowledge were used in clinical implementation, but fail to demonstrate the realistic decision-making in elective orthopaedic surgery. uptake and actual use of clinical guidelines in real world healthcare practice. Guideline implementation in orthopaedic surgery There is a need for empirical research which examines a A mixed methods systematic review has identified vari- wide range of contextual characteristics influencing the ous sources of evidence and knowledge which influence uptake and use of clinical guidelines. This includes the decision-making within orthopaedic surgery [15]. The Grove et al. Implementation Science (2018) 13:75 Page 3 of 14 findings revealed several factors which impact upon previous version of the guidance was disseminated to guideline implementation specific to the orthopaedic spe- practitioners. Therefore, our study was ideally timed to cialty. For example, compared to other clinical specialties, investigate the implementation of this updated guidance orthopaedic surgery represents a highly professionalised in the NHS to facilitate more general exploration of area of clinical work where elite communities of practice guidelines. are strongly embedded [16]. There is a tendency towards Three comparative case studies were conducted, using decisive and authoritative patterns of decision-making, qualitative methods with multiple levels of analysis [18]. and surgeons are able to retain substantial autonomy over The case studies were conducted in UK hospital Trusts their work practices to resist external intervention [15]. in the midlands, north and south west of England (see Enhanced implementation of guidelines is an unex- Table 1). A hospital Trust is an organisation that pro- plored area of research in the highly professionalised, vides secondary healthcare services to a locality within intensely networked group that is orthopaedic surgery. the English NHS system. The protocol for the entire This study provides an in-depth understanding of study has been described in detail elsewhere [19]. Each decision-making using clinical guidance. It generates a case study traced the implementation of NICE guidance unique perspective on the challenges of translating in practice to explore the understanding and use of evi- research into clinical practice and moves away from the dence and knowledge in orthopaedic surgery. We selected approach of assessing compliance or auditing guideline cases to represent maximal variation in orthopaedic ser- uptake. A key part of understanding complex problems, vices in England [20]. For example, an orthopaedic depart- such as where, when and how evidence and knowledge ment in a teaching hospital, one in a non-teaching are used in practice, requires examining the values, beliefs hospital and a third in a designated academic orthopaedic and norms of individuals who are responsible for making department where staff members hold hybrid academic/ decisions in context. This complements a broader investi- clinical roles in both the hospital and affiliated University. gation of an organisation’s capacity to support clinical We followed the roadmap for a case study research de- guideline implementation and of other contextual factors veloped by Eisenhardt [21] where each case study started within the healthcare sector which influence the use of as close as possible to the ideal of no theory under consid- knowledge and evidence in practice. eration. This method prevents any pre-selected theoretical perspectives limiting the data collection process [21]. The Methods research was abductive in nature. We were informed We examined the implementation of an example of NICE by previous literature and theory but also by the data guidance within orthopaedic surgery, to identify similar- collected [22]. ities and differences in the way this type of evidence is Data collection in the field allowed for concepts of used in practice. In 2014, NICE released updated guidance interest to develop as the case studies progressed [23]. on hip implants for total hip replacement for end-stage This flexible approach is a key feature of case study de- arthritis (see Additional file 1: Appendix SP1 for an signs, which enabled us to adjust data collection processes example of the guideline recommendations) [17]. At the to further investigate emergent themes and to take advan- time of our study, it had been over a decade since the tage of opportunities as they arose [24, 25]. For example, Table 1 Case study setting and participants Descriptor Case study A Case study B Case study C Setting Orthopaedic trauma centre Small hospital Trust split between Large orthopaedic department two geographical sites. Therefore with specialist trauma centre the orthopaedic services were which received national referrals separated across two hospital for complex hip implant revision buildings surgery University link Teaching hospital with a designated None Teaching hospital academic orthopaedic department located in a university owned building within the NHS hospital Participants A majority of surgeons held joint Surgeons provided general Surgeons in the teaching hospital posts between the NHS and the orthopaedic services to the held contracts with the NHS hospital. same university department. The local population supported A minority of surgeons held honorary staff conducted clinical effectiveness by a designated group of contracts with one of the four universities and cost-effectiveness research, allied health professionals in the region, i.e. they were not from mainly national randomised the same academic department controlled trials of various techniques and treatments within orthopaedic surgery Grove et al. Implementation Science (2018) 13:75 Page 4 of 14 the importance of groups of surgical colleagues acting as to understand and frame intentions to change practice communities of practice (as a potential theme) grew, as within the orthopaedic departments. Analysis of the doc- more data was collected and as case studies progressed in uments enabled us to gain a wider understanding of the series. This enabled us to search for specific instances of context within which decisions were made. surgeons in communities in the later cases and thus We interviewed 64 participants between December formed part of the data collection process. 2014 and December 2015. During the interviews, we Across the three cases, we sampled orthopaedic surgeons sought to understand the approaches and beliefs of par- and Allied Health Professionals (AHPs) who conducted or ticipants regarding knowledge and evidence, in order to facilitated joint replacement surgery, i.e., we sampled pur- reveal the strategies used by professionals when making posively aiming for heterogeneity of professional back- decisions. Questions explored the extent of professionals’ ground, level of training and years in practice. Snowball beliefs regarding NICE and the involvement and impact of sampling enabled us to follow direct recommendations clinical guidance on surgical practice within their hospital. from participants. We aimed to explore guideline im- The open interview format enabled participants to expand plementation from all perspectives, so we also invited on topics of interest freely. We set out to discover what administrators and managers involved in guideline im- professionals considered to be evidence and knowledge in plementation and in the decisions made for patients practice, rather than focusing on any pre-existing defin- undergoing hip replacement surgery. ition which may have restricted the findings. Each inter- view was labelled with location and timing and an Data collection anonymised identification number. We selected a combination of qualitative methods in- A copy of the interview topic guide is presented in the cluding document analysis, observation and interviews. Additional file 1: Appendix SP2. Table 3 displays the One author (AG) was responsible for all data collection different professional groups interviewed (‘C’ clinical, ‘A’ in each of the cases. Data collection remained systematic allied health professionals and ‘M’ managers). To obtain a and transparent, and decisions were recorded in case national perspective, we conducted eight key informant summaries. We continued data collection until no new interviews with stakeholders from NICE, The Royal information was obtained and theoretical saturation College of Surgeons, and Clinical Commissioning Groups. within each case was reached [26]. Within each case, 3 months of observation took place between 1 December Data analysis 2014 and 11 December 2015. Observations consisted of As outlined in the roadmap method, all data were ana- opportunistic shadowing involving, watching clinic and lysed, integrated and triangulated within case before teaching sessions and attendance at planned operating comparative case analysis was undertaken across the sessions, particularly pre-theatre preparation time. Obser- cases. The three data sources were processed into text vations enabled informal discussions with surgeons and format to allow for thematic analysis through data famil- clinical staff and provided an opportunity to describe iarisation, coding and development of categories from actions and decisions in real time. Each observation was codes [28]. The first stage of data analysis was conducted recorded in a field journal using a predetermined by one author (AG). Second round coding was per- template. formed jointly by all authors during two data analysis Document analysis involved collection of key organisa- sessions. Coding differences were reconciled through tional documents such as clinical pathways describing discussion by all authors and refinement of first- and structured multidisciplinary plan of care and hospital second-order codes was performed to generate categor- protocols (see Table 2)[27]. The documents helped us ies and themes. The three types of qualitative data were integrated using the Pillar Integration Process [29]. This is a matrix integration technique for mixing data which Table 2 Document type and quantity by case study site has been collected using different methods. Document type Case A Case B Case C Clinical pathways 5 3 6 Table 3 Participant numbers detailed by case study site and by Protocols 17 2 4 professional group Meeting notes 7 5 11 Professional group Case A Case B Case C Key informant interviews Strategy documents 2 1 0 Clinical (C) 12 10 8 4 Quarterly and annual reports 14 18 17 AHP (A) 4 5 6 2 Internal presentations 2 5 2 Managers (M) 2 4 5 2 Sub-total 47 34 40 Sub-total 18 19 19 8 Total 121 Total 64 Grove et al. Implementation Science (2018) 13:75 Page 5 of 14 We compared data across data collection methods. Types of knowledge and evidence which influence the Triangulation between data collection methods facili- implementation of clinical guidance in elective tated the validation of cases as we searched for orthopaedic surgery convergence among the multiple data sources. We tri- Case studies revealed that a significant number and a di- angulated data from the three data sources and inter- verse range of knowledge and evidence sources were preted them together to find common themes by used in decisions made by orthopaedic surgeons regard- eliminating overlapping areas and identifying areas of con- ing hip replacement surgery. We characterised these vergence [26]. We noted, for example, if a ‘guideline sources into micro, meso or macro levels of influence, as implementation process’ document in a hospital did displayed in Fig. 2. The sources of evidence and know- not match with the data collected in the observations ledge have been categorised this way to demonstrate the and interviews, this perhaps demonstrates that partici- structural level at which they were enacted in practice pants were not aware of this document or that it was [31]. The multi-level approach to synthesis helped to not an important factor into decision-making. Each recognise the interdependence between the various case was written up to provide narrative descriptions levels. Micro knowledge and evidence that tended to of the current situation at each hospital. This process influence individual decision makers, meso evidence and was central to generating familiarity and insight [30]. knowledge sources appeared to act at the level of the It enabled us to see patterns in each case as they organisation, whereas macro knowledge and evidence emerged and accelerated our cross case comparisons existed in the higher domain of the wider healthcare [21]. The goal of the cross case comparison was to environment. Narrative descriptions and an example of search for further patterns in the data and to explore each source of evidence and knowledge are provided how these were represented or played out differently in Table 4. in the three cases. Evidence in the form of NICE guidance, i.e., formal co- dified knowledge, was just one of the 17 types of know- Results ledge and evidence identified in our study. Therefore, The overarching themes displayed in Fig. 1 represent the additional 16 sources of evidence, such as the struc- broader narratives to describe the structural levels ture and location of the hospital or the opinions of which influenced guideline implementation in elective leaders and professional societies, influenced the uptake orthopaedic practice. The evidence and knowledge of and use of clinical guidelines in orthopaedic surgery. individual surgeons, groups of healthcare professionals, A determining factor of guideline implementation healthcare organisations and the regulatory environ- was how these knowledge and evidence types were ment interacted to produce the context for guideline amalgamated together in the different contexts of prac- implementation. tice. The amalgamation process was flexible, adaptable Fig. 1 Visual representation of the four thematic findings which describe the influence of evidence and knowledge on decision-making Grove et al. Implementation Science (2018) 13:75 Page 6 of 14 Fig. 2 Summary of the of knowledge and evidence identified in case studies of guideline implementation in orthopaedic practice and on-going, and therefore, the dominant source of Case A evidence and knowledge in each case would often Case A was an academic centre located within a trauma change. What was important to the decision-maker at and orthopaedic department. More than the other two onepoint in time wasnot always thepresenceorcon- cases, case A appeared to have a positive view of the for- tent of evidence-based guidelines (macro). It could mal codified knowledge contained in clinical guidelines equally be any other type of evidence or knowledge, and what guidelines set out to achieve in the healthcare interacting with, for example, a surgeon’s training and sector. The surgeons working in case A took a popula- formal education (micro) or the contingencies of prac- tion perspective on clinical decisions. The culture, norms tice such as the structure of the hospital (meso). and political influence of the sector acted on case A in a positive fashion, as surgeons working here valued the Distinct approaches to decision-making processes and aims of generating clinical guidelines and Comparison across cases was essential to look beyond the goals of NICE as an organisation. Surgeons were ac- the initial impressions and see the findings through customed to answering questions using a larger popula- multiple lenses. We framed our cases according to tion frame of reference and suggested that they “may their general approach to decision-making and how subconsciously be following NICE guidance” (Junior sur- guidelines were implemented in each hospital. The geon) as it was indoctrinated in the organisational know- differences in decision-making contributed to vari- ledge and processes. One surgeon states that “NICE ation from the codified evidence contained in guide- guidelines help you look at cost effectiveness and evi- lines across all three cases. The analysis revealed that dence a lot more than you would think about in normal the individuals, groups and organisations in each case daily practice as an individual orthopaedic surgeon” had a distinct approach to decision-making. The approach (Consultant surgeon). was dependent on how the 17 types of knowledge and evi- Case A had the most advanced and formalised guide- dence interacted, changed and were used in orthopaedic line implementation processes i.e., their managerial practice. Guideline implementation was contingent knowledge. This was reflected in case A’s extensive and mediated through the distinct contextual environ- protocol documentation compared to the other two ments, which were subject to forces of the regulatory cases (17 versus 4 and 2). Each protocol was linked to a environment. piece of clinical guidance or an internal evidence Grove et al. Implementation Science (2018) 13:75 Page 7 of 14 Table 4 Narrative descriptions and an example of each source of evidence and knowledge depicted in Fig. 2 Evidence and knowledge types Narrative description Example from the data Macro External evidence created by healthcare The wider delivery of healthcare in England An inspection report from the regulators, e.g. CQC and GMC is governed by the UK Health and Social Care Quality Commission Care Regulators such as the Care Quality Commission. In orthopaedics, surgeons have to be registered with the General Medical Council and with their Royal College. Regulators are responsible for ensuring that surgeons are included in an up-to-date registry of qualified doctors and practice according to established standards The media and the influence of The mass media (or press) is a diversified An article in a newspaper describing ‘the press’ collection of resources who reach a large ‘good’ or ‘bad’ hip implants audience via mass communication The opinion of leaders and An opinion leader was an eminent individual An opinion leader could be a principal professional societies who had the ability to influence the opinion investigator of a large clinical trial in of the orthopaedic community on a subject orthopaedics. The professional societies matter for which they well known. The were the British Orthopaedic Association professional societies were larger organisations and the Hip Society who represented the groups and sub-groups of surgeons Formal codified knowledge Evidence or knowledge that is written A NICE guideline or article published in down can be shared and is easy to access a journal and available to the public Culture, norms and political influence The standards and accepted way of practicing The hierarchical structure of the healthcare of the sector in the UK healthcare context. Including the system. Political factors included strategies public delivery of services and formal and enforced by government and the medico- informal methods in which healthcare is legal challenges to practice organised in the NHS Meso Managerial knowledge Each hospitals’ business organisational NHS hospital resource issues such as time, processes which underpin day to day cost and safety or quality of services routines and capabilities of the Trust Organisational knowledge An extension of managerial knowledge A hospitals’ internal processes which are which has a wider structural emphasis. It not written down. Anecdotally referred is embedded in the processes of healthcare to as “the way we do things around here” organisations and influences the behaviour of staff The structure and location of the hospital The physical location of the hospital buildings The number of elective orthopaedic and departments and the structure of the theatres available to use hospital wards Evidence from implant manufacturing Information that came directly from Leaflets about a hip implant from a companies manufacturer’s representatives located in t manufacturer’s representative he hospital or indirectly through marketing Socialisation and association with colleagues Knowledge that came from the inside and Evidence of the outcome of a surgery spread within the defined clinical group, in from a colleague or knowledge that a this case the orthopaedic community mentor had passed on Micro Informal experiential knowledge Tacit knowledge that surgeons ‘know’ Represents a surgeon’s lifetime’s work, regarding how to behave and perform as and in turn their identity as a surgeon an orthopaedic surgeon Informal experiential knowledge built The tacit knowledge that surgeons ‘know’ Knowing which colleague to refer a up over time which has built up over time working in the difficult case to when the surgeon specific hospital but which can be difficult does not have the specific expertise to describe or experience Evidence from the professional hierarchy The layered social structure within the Described as the ‘clinical pecking hospital which conceptualised the order’ with the consultant surgeon superior and inferior relationships at the top between clinical staff Grove et al. Implementation Science (2018) 13:75 Page 8 of 14 Table 4 Narrative descriptions and an example of each source of evidence and knowledge depicted in Fig. 2 (Continued) Evidence and knowledge types Narrative description Example from the data Training and formal education The training and formal education of A Master’s degree in Evidence-Based healthcare professionals which are Medicine recognised through standard academic qualifications Apprenticeship style training and Personal training which occurs during Training gained through fellowship informal education each working day with senior colleagues programmes and practice-based learning Individual patient and surgeon factors Characteristics of the patient or surgeon Patients age or a surgeons years in practice that influenced clinical practice decisions Evidence linked to the innate ‘feel’ A description of the surgeon’s judgement, A surgeon not knowing exactly what of surgery skill, craft and instinct will occur during an operation until they started the surgery and can see and feel the operation takes place summary which had been produced by medical librar- and the likely outcome of an orthopaedic intervention ians. The implementation process was described by the described in NICE guidelines were only part of the Chair of the Guideline Committee: knowledge picture which had to fit into the ever chan- ging context of practice. However, external evidence cre- We started monitoring NICE implementation in the ated by regulators and managerial and organisational trust in 2011/12 when we set up the NICE knowledge could limit the behaviour and decisions of implementation group…when NICE issues their surgeons. For example, surgeons working here were guidance, I forward a list to the NICE administrator restricted in the hip replacement implants, they could who would send it out to lead clinician in select based on cost and procurement contracts (meso) orthopaedics…If it’s a clinical guideline, or quality and thresholds established by the Orthopaedic Device standard…there’s recommendations in there, and a Evaluation Panel in the UK (macro). form of a baseline assessment with the recommendations in…they (the clinician) have to Case B indicate if we’re compliant or not. (Hybrid surgeon) Case B was a small hospital Trust split between two geographical sites. Case B demonstrated an ‘“it depends” The surgeons in case A demonstrated a distinctly approach to decision-making produced by the binary different trend in their confidence and appreciation of characteristics of the Trust’; this was bought about by clinical guidance (individual surgeon factors). We con- the structure and location of the hospital. The binary sider that this was due to their departmental focus on characteristics reflect the two distinct hospital locations research (apprenticeship style training) and academic in case B. By name, the hospitals were one Trust; how- output developed through training and formal education ever, the day-to-day operations and decision-making in Evidence Based Medicine (EBM), and beliefs regard- practices were separated and distinct. Each hospital loca- ing the importance of NICE and the EBM approach. A tion in case B had their own way of doing things (organ- quote from a surgeon illustrates this: isational knowledge) and staff acted protectively to maintain them. The ‘it depends approach’ signifies the “We do try as much as possible to follow the basis of participants views that what happened in practice evidence...So the typical patients would be where we depended on which of the two case B hospital sites they would need to use evidence. There is not much of a were located at when the decision was made. dilemma about someone with end stage osteoarthritis In case B, external evidence created by regulators was as the guidance shows the pathway that they should not regarded as important. Formal codified evidence in take.” (Consultant surgeon) NICE guidelines was not valued by the clinical staff. Guidelines were often considered in a negative light and The distinct approach to decision-making in case A considered to be the responsibility of hospital admin- was ‘pragmatic EBM decision-making’, where the trad- istrators and managers (i.e., managerial knowledge). itional approach to evidence-based decision-making rec- Described by a hospital board member below: ommended in clinical guidelines was suitable, but not sufficient for their practice. Surgeons in case A acknow- “I think NICE guidance is very much just seen as ledge traditional EBM and focused on pragmatic EBM. another layer of administration for clinicians. If no This practice-based approach to EBM considered the one’s looking at whether you’re following NICE important point that knowing what to do, how to do it, guidance or not they just sit on a shelf, unless you Grove et al. Implementation Science (2018) 13:75 Page 9 of 14 have a very active team of clinicians who take this on elsewhere in the Trust. They were also resistant to pressure board. But that’s not a consistent.” (Hospital board from external policymakers and evidence from regulators. member) This divide across professional boundaries compounded by the professional hierarchy and cultural norms of the sector The negative value attached to NICE and guidelines made guideline implementation challenging. Nevertheless, was echoed by the surgeon in the quote below, in which knowledge and evidence generated through managerial they demonstrate the importance of socialisation and as- and organisational knowledge was present in the wider sociation with colleagues when making clinical decisions: hospital organisation: “NICE are not being proactive enough, I would say, in We have standard operating procedures for NICE, terms of making recommendations on prostheses and which encourage [surgeons] to write their own action they could do a lot more. There is very little sort of plan (Trauma & Orthopaedic Manager) robust evidence to guide practice so you rely on other peoples’ anecdotal experience and normal practice to However, observations of practice revealed that man- help guide what, what works and what doesn’t” agerial and organisational knowledge enacted through (Consultant surgeon) processes attached to NICE guidelines and governance belonged in the managerial and administrative domain, Within the organisational processes, guidelines were not within surgical staff. Surgeons reported having “never not considered an important part of practice in case B. seen” the organisation’s NICE process (Consultant Compliance to NICE guidelines as evidence of external surgeon). This is reflected in the observation note below: regulation did not appear to be valued and hence imple- mentation was haphazard, as described by a hospital NICE was rarely noted as an influential factor in the administrator: day-to-day activities of surgeons. Surgeons I spoke to were unaware their hospital had a NICE process, they I send (guidance) out to the General Manager and would respond “do we have one?”“I’ve never seen it”. then they will send it to the most appropriate person. (Observation note) We used to meet to discuss if we were compliant… but now we send a questionnaire out…so they have to Clinical practice decisions were made using knowledge do is tick ‘Yes’ and ‘No’ but they do not always and evidence gained through socialisation and associ- respond. (Administrator) ation with colleagues. Surgeons possessed resilient ex- periential knowledge built up over time as the majority What mattered for implementation in case B was the had been working in this hospital for their entire careers dynamics of organisational change and leadership enacted and were relatively separate from ‘outside’ knowledge. through evidence from the professional hierarchy in the Formal codified evidence in clinical guidelines had to two orthopaedic departments within the Trust. The indi- compete with the complex social systems that existed in vidual surgeon factors and the differing characteristics and the hospital. For example, case C was a referral centre processes of the two groups of surgeons headed up by which specialised in performing complex hip replace- opinion leaders meant that guideline implementation ment revision surgery. In this context, clinical guidelines across the organisation was difficult. The distinct groups appeared to be less important because the approaches, struggled to work together and share organisational know- techniques and implants needed to perform hip revisions ledge because the specific contexts that clinicians were were specialist (the innate feel of surgery) and therefore socialised into differed. For example, evidence from im- not included in guideline recommendations. One sur- plant manufactures carried greater weight in one location geon noted “NICE, is irrelevant. They don’t tell me compared to the other. The socialisation and association anything I NEED to know” (Consultant surgeon). The with colleagues reflected the behaviour and norms that specialist surgeons working in case C referred to a stand- guided or regulated the action and decision-making of ard hip replacement operation as “boring” and, hence the individuals at case B. work of other surgeons “on the treadmill” who are outside of their community or social system (Consultant surgeon). Case C In this sense, guidelines were not valued or applicable to Finally, case C was an orthopaedic department located in a their specialist work. The informal experimental know- teaching hospital. In case C, ‘socialised decision-making ledge and informal experiential evidence built up over time was prominent and evidence was discretionary’.The ortho- appeared to take precedence in clinical decision-making paedic department in case C was closed to the influence of processes and therefore restricted guideline implementa- administrators and managerial knowledge emerging from tion of guidelines for this group of surgeons. These Grove et al. Implementation Science (2018) 13:75 Page 10 of 14 dominate types of knowledge and evidence were grounded to guideline implementation in orthopaedic surgery. in the experience of surgical work in practice and legacy These reflect the relationship between guideline imple- knowledge about the organisational functions which sur- mentation and knowledge and evidence that is actually geons developed as a consequence of working there for a used in orthopaedic practice. Orthopaedic surgeons in long period of time. our study held ambivalent or negative attitudes to- wards clinical guidelines. They did not privilege this Discussion formal codified evidence because it originated outside In orthopaedic surgery, clinical guidelines are an import- of orthopaedics and did not contain the micro sources ant part of practice as they not only help guarantee of knowledge and evidence (e.g., experience, training, safety and encourage quality improvement but also en- individual characteristics, and the innate feel of sur- sure that NHS resources are used appropriately [32]. gery)thatwere consideredmoreimportant forsur- The aim of this study was to identify where, when and geons’ decision-making. However, the culture and how evidence and knowledge are used in healthcare norms of EBM, identified in case A, demonstrate that decision-making in orthopaedic surgery and how vari- it was possible to positively influence guideline imple- ation in evidence and knowledge contributes to differ- mentation. Consideration of the power of professional ences in the implementation of clinical guidance in hierarchies is vital, as surgeons working at the top of practice. Previous scholars have counted and categorised the hierarchy, such as clinical leaders can restrict or evidence using broader taxonomies, which describe diminish the influence of mangers and policymakers. knowledge as individual, group, tacit or explicit [33–35]. Organisational constraints linked to financial restric- Others emphasise the role of the person in the activity tions, regulation and procurement-influenced imple- to distinguish between action, doing and practice, and mentation and a lack of focus on clinical guidelines in knowledge facts and processes [36]. It is important to orthopaedic practice. The presence of organisational highlight that what is considered evidence and know- processes and protocols could not ensure what guide- ledge is highly contested and influenced by the environ- lines were valued and used. What was more important ment in which it is used [10]. is what evidence and knowledge transferred from surgical In fulfilling our aim, we discovered 17 different types colleagues and professional societies. of knowledge and evidence which were used in ortho- It is likely that many of the 17 types of knowledge and paedic surgery. During our case comparison, it became evidence identified in this research would not have been clear that the dynamic selection, combination and use of discovered without the structured, comparative case each type of knowledge and evidence influenced the im- study approach used in our study. One of the aims of plementation and use of clinical guidance in practice. At our research was to go beyond reports in the previous the time of study, none of the cases could definitively guideline implementation literature [3–9, 37]. It is sig- provide evidence to demonstrate that they were acces- nificant that a large number and diverse range of know- sing, using and monitoring guideline recommendations. ledge and evidence types acting across the entire domain We examined implementation of guidelines by individ- of healthcare emerged from the three case studies of ual surgeons, groups of surgeons and the Trusts as a NHS practice. Previous research has identified a consid- whole. Interestingly, implementation could differ be- erable number of barriers and facilitators to the imple- tween these levels. mentation of clinical guidelines but often they are too In the context of orthopaedic surgery, the process of generic or limited in scope, for example, the difficultly of privileging different types of knowledge and evidence engaging individual clinicians or problems with process in the context of surgery resulted in three distinct ap- and resource issues within hospitals [7, 13, 14]. proaches to decision-making in orthopaedic surgery. We have demonstrated that guideline implementation These include ‘pragmatic EBM decision-making’ (case A), and subsequent evidence-based practice were not always where NICE guidelines failed to deliver all the knowledge possible or preferable in the three cases. Comparative and evidence needed to make clinical decisions when case study analysis revealed dynamic contextual differ- organisational context restricted surgeon choice. An ‘“it ences and variation in practices and processes between depends” approach produced by the binary characteristics’ the three hospitals. For example, some surgeons had of case B linked to the professional hierarchy of surgeons strict limits placed on the orthopaedic implants they working in separate geographical locations and ‘socialised could order within their hospital. This restricted their decision-making’ where evidence was discretionary due to implant decisions. However, this varied across the cases the strong influence of socialisation and informal know- and differences were found in implant selection practices ledge sharing between surgeons in case C. and processes. This variation had a direct impact on the The similarities and differences between the three uptake and use of evidence in practice and demonstrates approaches generate key contextual dimensions specific the problems of effectively implementing standardised Grove et al. Implementation Science (2018) 13:75 Page 11 of 14 guidance in surgery. One of the aims of clinical guidance Our study has limitations. The direct observation of is to reduce unjustified variation in practice, but we healthcare professionals in their practice may have found that this was not achievable or appropriate in all driven a change to ‘good’ or ‘better’ behaviour by partici- contexts. pants; a phenomenon known as observer bias [39]. To Our findings encourage more research into how to en- ensure the quality and rigour of our data, we extended gage individuals and groups of healthcare professionals our access and observation as much as possible, whilst to consider the content of clinical guidelines and how it also conducting crosschecks and validation during inter- might add to their decision-making processes. The over- views and between different individuals and professional arching view that guidelines are the responsibility of groups. We sampled three hospitals from the population managers and administrators demonstrates a lack of of 135 hospital Trusts in England which deliver hip re- ownership of the guideline in general in orthopaedics. placement services [40]. Although the sample was small, Also, in the context of orthopaedics, our findings pro- we aimed to achieve a broad representation of the types vide insight into approaches to knowledge mobilisation of elective orthopaedic services available in England and targeted at communities of practice as an area for inves- aimed to produce in-depth rather than a breadth in our tigation and improvement. Improvements in the uptake case study design and data. and use of NICE guidance in orthopaedic practice will require the development, presentation and dissemination Policy and practice implications of evidence-based guidelines in surgery to be better Over the last 20 years, there have been significant tailored to the orthopaedic community. changes in the way policy-making organisations such as NICE create and disseminate guidance to improve health Strengths and limitations and social care. What appears to have remained constant The key strength of our research is the use of case studies is the way in which codified knowledge in guidance is to examine the context of guideline implementation. In produced with an assumption that a linear ‘push’ fashion achieving our aim, we were able to discover and explain will ensure that it is received and acted on by clinicians the gap in implementation of clinical guidelines in ortho- working in healthcare organisations. The findings of our paedics. Uptake and use of guidelines, even when study reconfirm and extend our knowledge of the limits grounded on the findings of empirical research including of this approach for improving the use of guidance and gold standard randomised controlled trials, were not guar- for reducing variation in practice for orthopaedics. anteed in practice. This suggests that well-developed In this study, we have raised the issue of whether guidelines are necessary but not sufficient to achieve the NICE guidelines are ever likely to be appropriate for the goals of policymaking organisations such as NICE and field of orthopaedics. This is due to the wide range of Scottish Intercollegiate Guidelines Network (SIGN). Case knowledge and evidence identified as influential to study methods allowed us to describe the decision-making decision-making, coupled with the differences in guide- context in detail. This method has provided a greater line implementation across the structural levels. How- depth of description of the evidence and knowledge ever, we do not consider that the evidence contained in sources than have been outlined in previous literature, for guidelines is inappropriate. Instead, the ways in which example, expanding the tacit-explicit-group-individual knowledge and evidence were privileged differently by categorisations. The way in which evidence and know- individual practitioners, groups of surgeons and organi- ledge interacted with context produced variation in the sations meant that guidelines were rarely accessed as a extent to which guidelines are implemented and used in beneficial evidence source. Surgeons were not concerned orthopaedics. about what guidelines recommended. What was import- A second strength of our research is the use of ant was their definition of knowledge and evidence and multiple data sources (interviews, observations and how this interacted with understandings of knowledge document analysis) to study the same phenomena. The and evidence in their group and wider organisation. In combination of methods facilitated us to overcome the this study, ‘one size’ guidance could never ‘fit all’ the sur- weaknesses that emanate from selecting a single method geons’ requirements and therefore, the guidance had to study the complex process and practice of guideline limited value in their specific circumstances. implementation [38]. We triangulated data to enhance Nevertheless, evidence in guidelines represents best prac- the credibility of our analysis and findings. When data tice, and NICE and SIGN must produce recommendations from one source substantiates a pattern from another, for healthcare. We have provided evidence to suggest that the findings are stronger and better substantiated [26]. the current modes of transfer and implementation are inef- Comparison across our three comparative cases enabled fective. Changes could be made to the process of guideline us to generate a more sophisticated understanding of creation, dissemination or even regulation to move towards the data we collected [22, 23]. effective knowledge mobilisation. For example, a more Grove et al. Implementation Science (2018) 13:75 Page 12 of 14 inclusive process of involving clinicians in guideline devel- all sides of knowledge boundaries to understand and accept opment would be welcomed. The current system relies on why certain options are chosen and actions are taken, espe- clinicians being aware of guideline updates, rather than cially if they vary from guideline recommendations. being enabled to actively volunteer their contributions. Tar- geting the orthopaedic community through professional Conclusion meetings, networks and clinical leaders would communi- The research aimed to explore guideline implementation cate the need for involvement and increase discussion through the application of comparative case studies about new guidelines in contrast to the current process of which investigate the use of NICE clinical guidelines in one-way dissemination by policymakers. decisions made in elective orthopaedic surgery in the Regulation was a valuable mechanism as it achieved a NHS. The results of our study highlight the range of desired outcome for achieving targets and controlling the complex and competing sources of evidence and know- behaviour of the surgeons. However, moving towards ledge which influence the work practices of healthcare regulation as the norm did not appear to be a desirable professionals. Case study analysis revealed three distinct option for most of the professional groups in this study. styles of orthopaedic practice which represent the ways Knowledge and evidence from external regulators did not in which 17 types of knowledge and evidence were used hold the same positive status achieved by the knowledge during decision-making. The way in which evidence and and evidence emanating from colleague and professional knowledge were selected and used impacted on how networks. Restricting the discretion and authority of clin- guidelines were implemented in the orthopaedic special- ical professionals by increasing regulatory power would ity. Findings from the case comparison reflect the com- not be recommended. Instead, interventions which take plexity of evidence-based decision-making in the highly advantage of the positive knowledge mobilisation between professionalised organisationally regulated context of orthopaedic colleagues are encouraged, as they may im- surgery. Our results could be used to guide the develop- prove the sharing of evidence-based practice. ment of implementation interventions that are grounded Improvements need to be made to how healthcare in the findings of this study. New approaches to imple- professionals working in hospitals see and think about mentation need to appreciate and incorporate the di- evidence from guidelines in combination with other verse range of knowledge and evidence which influences knowledge and evidence sources. Improvement interven- clinical decisions in orthopaedics and to take account of tions are required to help users of guidance identify the changing contextual situations in which decisions ‘where they are at’ in their decision-making processes. need to be made. Clinical guidelines were often unable to provide a solu- tion to a decision problem; therefore, it is important to understand what other types of evidence and knowledge Additional file are available or used by others. Practitioners need to acknowledge the difference between certain types of Additional file 1: An example of a NICE guidance recommendation. knowledge as positive or negative to patient care. Where Technology appraisal guidance [TA304] [17]. Total hip replacement and resurfacing arthroplasty for end-stage arthritis of the hip. Interview topic possible, those working in healthcare should focus on guide. (DOCX 15 kb) reducing undesirable types of evidence and knowledge present in their organisation. This could be an area for improvement work. However, surgeons in this study Abbreviations AHPs: Allied Health Professionals (including Occupational Therapists, were often unaware of or ambivalent about the conse- Operating Department Practitioners, Physiotherapists, Nurses and Healthcare quence of their decisions because processes were not Assistants); EBM: Evidence-based medicine; NHS: National Health Service; open, transparent or subject to feedback loops. SIGN: Scottish Intercollegiate Guidelines Network Healthcare practitioners could take a more transparent approach in understanding the evidence that is driving Acknowledgements their decisions and how guidelines may fit into the pic- We thank the three hospital departments who agreed to participate in the research project, and all the NHS staff who took part in the interviews and ture. This will facilitate practitioners in deciding whether assisted with observations. guideline recommendations are appropriate in their con- text. If not, other knowledge sources such as clinical ex- Funding perience could take precedence and be shared, explained Amy Grove was supported by a National Institute for Health Research and understood, rather than frowned upon by managers Doctoral Fellowship programme project number 2013-06-064. Aileen Clarke and administrators and recorded as an organisational and Graeme Currie were partly supported by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care risk. Practice-based knowledge was rarely shared be- West Midlands at University Hospitals Birmingham NHS Foundation Trust. tween the professional groups in this study. Encouraging The views expressed are those of the authors and not necessarily those of open decision-making processes might enable those on the NHS, the NIHR or the Department of Health and Social Care. Grove et al. Implementation Science (2018) 13:75 Page 13 of 14 Availability of data and materials 9. Platt C, Larcombe J, Dudley J, McNulty C, Banerjee J, Gyoffry G, Pike K, The datasets generated and analysed during the current study are not Jadresic L. Implementation of NICE guidance on urinary tract infections in publicly available due to restrictions of ethical approvals obtained for this children in primary and secondary care. Acta Paediatr. 2015;104:630–7. study. 10. Gkeredakis E, Swan J, Powell J, Nicolini D, Scarbrough H, Roginski C, Taylor- Phillips S, Clarke A. Mind the gap: understanding utilisation of evidence and Authors’ contributions policy in health care management practice. J Health Organ Manag. 2011;25: All authors (AG, AC, GC) have made substantial contributions to the design 298–314. of the study. AG collected the data, and all authors analysed and interpreted 11. Currie G, El Enany N, Lockett A. Intra-professional dynamics in translational health the data. All authors have been involved in drafting the manuscript and research. The perspective of social scientists. Soc Sci Med. 2014;114:81–8. have given final approval of the version to be published. All authors agree to 12. Davies HTO, Powell AE, Nutley SM. Mobilising knowledge to improve UK be accountable for all aspects of the work in ensuring that questions related health care: learning from other countries and other sectors—a to the accuracy or integrity of any part of the work are appropriately multimethod mapping study. Health Serv Deliv Res. 2015;3:27. investigated and resolved. 13. Denny K. Evidence-based medicine and medical authority. J Med Humanit. 1999;20:247–63. Ethics approval and consent to participate 14. Timmermans S. From autonomy to accountability: the role of clinical practice The study was approved by the Biomedical and Scientific Research Ethics guidelines in professional power. Perspect Biol Med. 2005;48:490–501. Committee of The University of Warwick, England (approved June 2nd 2014; 15. Grove A, Johnson R, Clarke A, Currie G. Evidence and the drivers of variation reference number REGO-2014-645) and the Research and Development in orthopaedic surgical work: a mixed method systematic review. Health departments of each of the three hospital sites (case A approved on 30 June, Syst Policy Res. 2016;3:1. 2014, case B approved on 23 October, 2014, case C approved on 21 August 16. Ferlie W, Wood M, Fitzgerald L. Some limits to evidence-based medicine: a 2014). In line with the ethical approval, each participant will be asked to sign case study from elective orthopaedics. Qual Health Care. 1999;9:99–107. an informed consent form as will be provided with information outlining the 17. NICE Technology Appraisal (TA304). Total hip replacement and resurfacing purpose of the study and their rights to withdraw. arthroplasty for end-stage arthritis of the hip. NICE. 2014. https://www.nice. org.uk/guidance/ta304. Accessed 30 Nov 2017. Consent for publication 18. Stake R. The art of case study research. London: Sage; 1995. Informed consent was obtained to include anonymised data in any 19. Grove A, Clarke A, Currie G. The barriers and facilitators to the publications. Confidentiality is protected as the data is anonymised. implementation of clinical guidance in elective orthopaedic surgery: a qualitative study protocol. Implement Sci. 2015;2:81. Competing interests 20. Sandelowski M. The problem of rigor in qualitative research. Adv Nurs Sci. The authors declare that they have no competing interests. 1986;8:27–37. 21. Eisenhardt KM. Building theories from case study research. Acad Manag Rev. 1989;14:532–50. Publisher’sNote 22. Mantere S, Ketokivi M. Reasoning in organization science. Acad Manag Rev. Springer Nature remains neutral with regard to jurisdictional claims in 2013;38:70–89. published maps and institutional affiliations. 23. Parlett M, Hamilton D. Evaluation as illumination: a new approach to the study of innovatory programs. In: Glass G, editor. Evaluation studies review Author details 1 annual, vol. 1. Beverly Hills: Sage; 1976. Division of Health Sciences, Warwick Medical School, University of Warwick, 2 24. George AL, Bennett A. Case studies and theory development in the social Coventry, UK. Entrepreneurship & Innovation, Organising Healthcare sciences. Cambridge: MIT Press; 2005. Research Network, Warwick Business School, University of Warwick, Coventry, 25. Yin R. Case study research. Beverly Hills: Sage Publications; 1984. UK. 26. Creswell J, Clark V. Designing and conducting mixed methods research. London: Sage Publications; 2010. Received: 18 January 2018 Accepted: 23 May 2018 27. Kinsman L, Rotter T, James E, Snow P, Willis J. What is a clinical pathway? Development of a definition to inform the debate. BMC Med. 2010;8:131. 28. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. References 2006;3:77–101. 1. NICE. About. 2017. http://www.nice.org.uk/about. Accessed on 6 Dec 2017. 29. Johnson R, Grove A, Clarke A. Pillar integration process: a technique to 2. SIGN. Who we are. 2017. http://www.sign.ac.uk/who-we-are.html. Accessed integrate data in mixed methods research. J Mixed Methods Res. 2017:1–20. on 6 Dec 2017. https://doi.org/10.1177/1558689817743108. 3. Lowson K, Jenks M, Filby A, Carr L, Campbell B, Powell J. Examining the 30. Pettigrew AM. Longitudinal field research on change: theory and practice. implementation of NICE guidance: cross-sectional survey of the use of Organ Sci. 1990;1:267–92. NICE interventional procedures guidance by NHS Trusts. Implement Sci. 31. Pope C, Robert G, Bate P, Le May A, Gabbay J. Lost in translation: a multi- 2015;10:93. level case study of the metamorphosis of meanings and action in public 4. Sheldon T, Cullum N, Dawson D, Lankshear L, Lowson K, Watt I, West P, sector organizational innovation. Public Adm. 2006;84:59–79. Wright D, Wright J. What’s the evidence that NICE guidance has been 32. NICE. The NICE way. Lessons learned from the National Institute for Health implemented? Results from a national evaluation using time series analysis, and Care Excellence. 2014. https://www.nice.org.uk/news/article/using-nice- audit of patients’ notes, and interviews. BMJ. 2004;329:999. s-approach-to-base-policy-decisions-on-evidence-could-help-save-billions. 5. Weng Y-H, Kuo K, Yang C-Y, Lo H-L, Chen C, Chiu Y-W. Implementation of Accessed 30 Nov 2017. evidence-based practice across medical, nursing, pharmacological and allied 33. Cook S, Brown J. Bridging epistemologies: the generative dance between healthcare professionals: a questionnaire survey in nationwide hospital organizational knowledge and organizational knowing. Organ Sci. 1999;10: settings. Impact Sci Soc. 2013;8:112. 381–400. 6. Francke A, Smit M, de Veer A, Mistiaen P. Factors influencing the 34. Nonaka I. A dynamic theory of organizational knowledge creation. Organ implementation of clinical guidelines for health care professionals: a Sci. 1994;5:14–37. systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38. 7. Fitzgerald A, Lethaby A, Cikalo M, Glanville J, Wood H. Review of systematic 35. Lam A. Tacit knowledge, organizational learning and societal institutions: an reviews exploring the implementation/uptake od guidelines. York Health integrated framework. Organ Stud. 2000;21:487–513. Economics Consortium. 2014. https://www.nice.org.uk/guidance/ph56/ 36. Orlikowski W. Knowing in practice: enacting a collective capability in evidence/evidence-review-2-431762366. Accessed 8 Jan 2018. distributed organizing. Organ Sci. 2002;13:249–73. 8. Royal College of Physicians. Implementing NICE public health guidance for 37. Berry K, Haddock G. The implementation of the NICE guidelines for the workplace: a National Organisational Audit of NHS Trusts in England. schizophrenia: barriers to the implementation of psychological 2015. https://www.rcplondon.ac.uk/projects/outputs/implementing-nice- interventions and recommendations for the future. Psychol Psychother. public-health-guidance-workplace-2013-round-2. Accessed 8 Jan 2018. 2008;81:419–36. Grove et al. Implementation Science (2018) 13:75 Page 14 of 14 38. Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton S. Integrative approaches to qualitative and quantitative evidence. London: NHS Health Development Agency; 2004. 39. Snyder ML, Frankel A. Observer bias: a stringent test of behavior engulfing the field. J Pers Soc Psychol. 1976;34:857. 40. NHS Confederation. NHS statistics, facts and figures. 2017. http://www. nhsconfed.org/resources/key-statistics-on-the-nhs. Accessed on 6 Dec 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Implementation Science Springer Journals

How are evidence and knowledge used in orthopaedic decision-making? Three comparative case studies of different approaches to implementation of clinical guidance in practice

Free
14 pages

Loading next page...
 
/lp/springer_journal/how-are-evidence-and-knowledge-used-in-orthopaedic-decision-making-qMpDnbS57j
Publisher
Springer Journals
Copyright
Copyright © 2018 by The Author(s).
Subject
Medicine & Public Health; Health Promotion and Disease Prevention; Health Administration; Health Informatics; Public Health
eISSN
1748-5908
D.O.I.
10.1186/s13012-018-0771-4
Publisher site
See Article on Publisher Site

Abstract

Background: The uptake and use of clinical guidelines is often insufficient to change clinical behaviour and reduce variation in practice. As a consequence of diverse organisational contexts, the simple provision of guidelines cannot ensure fidelity or guarantee their use when making decisions. Implementation research in surgery has focused on understanding what evidence exists for clinical practice decisions but limits understanding to the technical, educational and accessibility issues. This research aims to identify where, when and how evidence and knowledge are used in orthopaedic decision-making and how variation in these factors contributes to different approaches to implementation of clinical guidance in practice. Methods: We used in-depth case studies to examine guideline implementation in real-life surgical practice. We conducted comparative case studies in three English National Health Service hospitals over a 12-month period. Each in-depth case study consisted of a mix of qualitative methods including interviews, observations and document analysis. Data included field notes from observations of day-to-day practice, 64 interviews with NHS surgeons and staff and the collection of 121 supplementary documents. Results: Case studies identified 17 sources of knowledge and evidence which influenced clinical decisions in elective orthopaedic surgery. A comparative analysis across cases revealed that each hospital had distinct approaches to decision-making. Decision-making is described as occurring as a result of how 17 types of knowledge and evidence were privileged and of how they interacted and changed in context. Guideline implementation was contingent and mediated through four distinct contextual levels. Implementation could be assessed for individual surgeons, groups of surgeons or the organisation as a whole, but it could also differ between these levels. Differences in how evidence and knowledge were used contributed to variations in practice from guidelines. (Continued on next page) * Correspondence: A.L.Grove@warwick.ac.uk Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Grove et al. Implementation Science (2018) 13:75 Page 2 of 14 (Continued from previous page) Conclusion: A range of complex and competing sources of evidence and knowledge exists which influence the working practices of healthcare professionals. The dynamic selection, combination and use of each type of knowledge and evidence influence the implementation and use of clinical guidance in practice. Clinical guidelines are a fundamental part of practice, but represent only one type of evidence influencing clinical decisions. In the orthopaedic speciality, other distinct sources of evidence and knowledge are selected and used which impact on how guidelines are implemented. New approaches to guideline implementation need to appreciate and incorporate this diverse range of knowledge and evidence which influences clinical decisions and to take account of the changing contexts in which decisions are made. Keywords: Guidelines, Implementation, Orthopaedic surgery, Evidence-based medicine, Comparative case study Background professional and environmental factors which have been Across the world, policymaking organisations exist to pro- described previously, but also requires a description of the duce clinical guidance and recommendations for health- local practice context and wider healthcare sector in care, which are based on scientific evidence. In the UK, sufficient detail to facilitate more effective guideline these organisations are the National Institute for Health implementation. Therefore, the aim of this research was and Care Excellence (NICE) and the Scottish Intercollegi- to identify where, when and how evidence and knowledge ate Guidelines Network (SIGN) [1, 2]. Together, they pro- are used in healthcare decision-making and how variation vide evidence-based guidance on the most effective ways in these factors contributes to different approaches to to diagnose, treat and prevent poor health for National implementation of clinical guidance in practice. Health Service (NHS) patients. One of the aims of clinical guidance is to reduce variation in practice, and therefore, Over and above clinical guidelines limit inequalities in service delivery [3]. However, previous Achieving effective guideline implementation reflects research has established that healthcare organisations face not only the people involved, but also their professional several challenges to implementation [4–6]. roles, their positions in the organisation and the epi- In 2014, a review of systematic reviews explored factors stemic communities to which they belong [10, 11]. which influence guideline implementation and uptake [7]. Producers and users of evidence and guidelines often sit The findings report a range of factors including those as- on different sides of the social, scientific and clinical sociated with guidelines themselves (e.g., complexity), with boundaries. Therefore, they possess varying types of healthcare professionals (e.g., lack of awareness) and with knowledge which may mean that they privilege evidence the working environment such as limited time, personnel differently. This can make integration across these com- and resources devoted to support guideline adherence [7]. munities challenging [11]. Day-to-day decision-making Limited information was provided to describe the context by healthcare professionals requires the selection of many of guideline implementation in detail or the differences in types of knowledge and evidence, situated within local, healthcare environments which might impact implemen- contextual and social circumstances [12]. For example, tation, such as differences in the processes of healthcare research knowledge coexists with the lived experience of delivery and national policy decisions. Evidence exists patients and the macro healthcare initiatives and incen- describing the barriers to implementation of clinical tives from policymakers and regulatory bodies. guidance and the rates of guideline uptake across a Preserving clinical autonomy and medical judgement by range of diseases [3–5, 8, 9]. For example, lack of time these professional groups is also recognised as important, was identified as the most commonly reported barrier particularly when guidelines challenge traditional practices to implementation [5]. However, too few studies rigor- [13, 14]. In this study, we examine guideline implementa- ously assess the effectiveness of approaches to improve tion in the context of real-life NHS practice. We investigate implementation of clinical guidelines or explicitly de- guideline implementation problems through the application scribe factors which enable their uptake and use [6, 7]. of a range of qualitative methods to explore where, when Rates of guideline uptake provide a proxy for guideline and how evidence and knowledge were used in clinical implementation, but fail to demonstrate the realistic decision-making in elective orthopaedic surgery. uptake and actual use of clinical guidelines in real world healthcare practice. Guideline implementation in orthopaedic surgery There is a need for empirical research which examines a A mixed methods systematic review has identified vari- wide range of contextual characteristics influencing the ous sources of evidence and knowledge which influence uptake and use of clinical guidelines. This includes the decision-making within orthopaedic surgery [15]. The Grove et al. Implementation Science (2018) 13:75 Page 3 of 14 findings revealed several factors which impact upon previous version of the guidance was disseminated to guideline implementation specific to the orthopaedic spe- practitioners. Therefore, our study was ideally timed to cialty. For example, compared to other clinical specialties, investigate the implementation of this updated guidance orthopaedic surgery represents a highly professionalised in the NHS to facilitate more general exploration of area of clinical work where elite communities of practice guidelines. are strongly embedded [16]. There is a tendency towards Three comparative case studies were conducted, using decisive and authoritative patterns of decision-making, qualitative methods with multiple levels of analysis [18]. and surgeons are able to retain substantial autonomy over The case studies were conducted in UK hospital Trusts their work practices to resist external intervention [15]. in the midlands, north and south west of England (see Enhanced implementation of guidelines is an unex- Table 1). A hospital Trust is an organisation that pro- plored area of research in the highly professionalised, vides secondary healthcare services to a locality within intensely networked group that is orthopaedic surgery. the English NHS system. The protocol for the entire This study provides an in-depth understanding of study has been described in detail elsewhere [19]. Each decision-making using clinical guidance. It generates a case study traced the implementation of NICE guidance unique perspective on the challenges of translating in practice to explore the understanding and use of evi- research into clinical practice and moves away from the dence and knowledge in orthopaedic surgery. We selected approach of assessing compliance or auditing guideline cases to represent maximal variation in orthopaedic ser- uptake. A key part of understanding complex problems, vices in England [20]. For example, an orthopaedic depart- such as where, when and how evidence and knowledge ment in a teaching hospital, one in a non-teaching are used in practice, requires examining the values, beliefs hospital and a third in a designated academic orthopaedic and norms of individuals who are responsible for making department where staff members hold hybrid academic/ decisions in context. This complements a broader investi- clinical roles in both the hospital and affiliated University. gation of an organisation’s capacity to support clinical We followed the roadmap for a case study research de- guideline implementation and of other contextual factors veloped by Eisenhardt [21] where each case study started within the healthcare sector which influence the use of as close as possible to the ideal of no theory under consid- knowledge and evidence in practice. eration. This method prevents any pre-selected theoretical perspectives limiting the data collection process [21]. The Methods research was abductive in nature. We were informed We examined the implementation of an example of NICE by previous literature and theory but also by the data guidance within orthopaedic surgery, to identify similar- collected [22]. ities and differences in the way this type of evidence is Data collection in the field allowed for concepts of used in practice. In 2014, NICE released updated guidance interest to develop as the case studies progressed [23]. on hip implants for total hip replacement for end-stage This flexible approach is a key feature of case study de- arthritis (see Additional file 1: Appendix SP1 for an signs, which enabled us to adjust data collection processes example of the guideline recommendations) [17]. At the to further investigate emergent themes and to take advan- time of our study, it had been over a decade since the tage of opportunities as they arose [24, 25]. For example, Table 1 Case study setting and participants Descriptor Case study A Case study B Case study C Setting Orthopaedic trauma centre Small hospital Trust split between Large orthopaedic department two geographical sites. Therefore with specialist trauma centre the orthopaedic services were which received national referrals separated across two hospital for complex hip implant revision buildings surgery University link Teaching hospital with a designated None Teaching hospital academic orthopaedic department located in a university owned building within the NHS hospital Participants A majority of surgeons held joint Surgeons provided general Surgeons in the teaching hospital posts between the NHS and the orthopaedic services to the held contracts with the NHS hospital. same university department. The local population supported A minority of surgeons held honorary staff conducted clinical effectiveness by a designated group of contracts with one of the four universities and cost-effectiveness research, allied health professionals in the region, i.e. they were not from mainly national randomised the same academic department controlled trials of various techniques and treatments within orthopaedic surgery Grove et al. Implementation Science (2018) 13:75 Page 4 of 14 the importance of groups of surgical colleagues acting as to understand and frame intentions to change practice communities of practice (as a potential theme) grew, as within the orthopaedic departments. Analysis of the doc- more data was collected and as case studies progressed in uments enabled us to gain a wider understanding of the series. This enabled us to search for specific instances of context within which decisions were made. surgeons in communities in the later cases and thus We interviewed 64 participants between December formed part of the data collection process. 2014 and December 2015. During the interviews, we Across the three cases, we sampled orthopaedic surgeons sought to understand the approaches and beliefs of par- and Allied Health Professionals (AHPs) who conducted or ticipants regarding knowledge and evidence, in order to facilitated joint replacement surgery, i.e., we sampled pur- reveal the strategies used by professionals when making posively aiming for heterogeneity of professional back- decisions. Questions explored the extent of professionals’ ground, level of training and years in practice. Snowball beliefs regarding NICE and the involvement and impact of sampling enabled us to follow direct recommendations clinical guidance on surgical practice within their hospital. from participants. We aimed to explore guideline im- The open interview format enabled participants to expand plementation from all perspectives, so we also invited on topics of interest freely. We set out to discover what administrators and managers involved in guideline im- professionals considered to be evidence and knowledge in plementation and in the decisions made for patients practice, rather than focusing on any pre-existing defin- undergoing hip replacement surgery. ition which may have restricted the findings. Each inter- view was labelled with location and timing and an Data collection anonymised identification number. We selected a combination of qualitative methods in- A copy of the interview topic guide is presented in the cluding document analysis, observation and interviews. Additional file 1: Appendix SP2. Table 3 displays the One author (AG) was responsible for all data collection different professional groups interviewed (‘C’ clinical, ‘A’ in each of the cases. Data collection remained systematic allied health professionals and ‘M’ managers). To obtain a and transparent, and decisions were recorded in case national perspective, we conducted eight key informant summaries. We continued data collection until no new interviews with stakeholders from NICE, The Royal information was obtained and theoretical saturation College of Surgeons, and Clinical Commissioning Groups. within each case was reached [26]. Within each case, 3 months of observation took place between 1 December Data analysis 2014 and 11 December 2015. Observations consisted of As outlined in the roadmap method, all data were ana- opportunistic shadowing involving, watching clinic and lysed, integrated and triangulated within case before teaching sessions and attendance at planned operating comparative case analysis was undertaken across the sessions, particularly pre-theatre preparation time. Obser- cases. The three data sources were processed into text vations enabled informal discussions with surgeons and format to allow for thematic analysis through data famil- clinical staff and provided an opportunity to describe iarisation, coding and development of categories from actions and decisions in real time. Each observation was codes [28]. The first stage of data analysis was conducted recorded in a field journal using a predetermined by one author (AG). Second round coding was per- template. formed jointly by all authors during two data analysis Document analysis involved collection of key organisa- sessions. Coding differences were reconciled through tional documents such as clinical pathways describing discussion by all authors and refinement of first- and structured multidisciplinary plan of care and hospital second-order codes was performed to generate categor- protocols (see Table 2)[27]. The documents helped us ies and themes. The three types of qualitative data were integrated using the Pillar Integration Process [29]. This is a matrix integration technique for mixing data which Table 2 Document type and quantity by case study site has been collected using different methods. Document type Case A Case B Case C Clinical pathways 5 3 6 Table 3 Participant numbers detailed by case study site and by Protocols 17 2 4 professional group Meeting notes 7 5 11 Professional group Case A Case B Case C Key informant interviews Strategy documents 2 1 0 Clinical (C) 12 10 8 4 Quarterly and annual reports 14 18 17 AHP (A) 4 5 6 2 Internal presentations 2 5 2 Managers (M) 2 4 5 2 Sub-total 47 34 40 Sub-total 18 19 19 8 Total 121 Total 64 Grove et al. Implementation Science (2018) 13:75 Page 5 of 14 We compared data across data collection methods. Types of knowledge and evidence which influence the Triangulation between data collection methods facili- implementation of clinical guidance in elective tated the validation of cases as we searched for orthopaedic surgery convergence among the multiple data sources. We tri- Case studies revealed that a significant number and a di- angulated data from the three data sources and inter- verse range of knowledge and evidence sources were preted them together to find common themes by used in decisions made by orthopaedic surgeons regard- eliminating overlapping areas and identifying areas of con- ing hip replacement surgery. We characterised these vergence [26]. We noted, for example, if a ‘guideline sources into micro, meso or macro levels of influence, as implementation process’ document in a hospital did displayed in Fig. 2. The sources of evidence and know- not match with the data collected in the observations ledge have been categorised this way to demonstrate the and interviews, this perhaps demonstrates that partici- structural level at which they were enacted in practice pants were not aware of this document or that it was [31]. The multi-level approach to synthesis helped to not an important factor into decision-making. Each recognise the interdependence between the various case was written up to provide narrative descriptions levels. Micro knowledge and evidence that tended to of the current situation at each hospital. This process influence individual decision makers, meso evidence and was central to generating familiarity and insight [30]. knowledge sources appeared to act at the level of the It enabled us to see patterns in each case as they organisation, whereas macro knowledge and evidence emerged and accelerated our cross case comparisons existed in the higher domain of the wider healthcare [21]. The goal of the cross case comparison was to environment. Narrative descriptions and an example of search for further patterns in the data and to explore each source of evidence and knowledge are provided how these were represented or played out differently in Table 4. in the three cases. Evidence in the form of NICE guidance, i.e., formal co- dified knowledge, was just one of the 17 types of know- Results ledge and evidence identified in our study. Therefore, The overarching themes displayed in Fig. 1 represent the additional 16 sources of evidence, such as the struc- broader narratives to describe the structural levels ture and location of the hospital or the opinions of which influenced guideline implementation in elective leaders and professional societies, influenced the uptake orthopaedic practice. The evidence and knowledge of and use of clinical guidelines in orthopaedic surgery. individual surgeons, groups of healthcare professionals, A determining factor of guideline implementation healthcare organisations and the regulatory environ- was how these knowledge and evidence types were ment interacted to produce the context for guideline amalgamated together in the different contexts of prac- implementation. tice. The amalgamation process was flexible, adaptable Fig. 1 Visual representation of the four thematic findings which describe the influence of evidence and knowledge on decision-making Grove et al. Implementation Science (2018) 13:75 Page 6 of 14 Fig. 2 Summary of the of knowledge and evidence identified in case studies of guideline implementation in orthopaedic practice and on-going, and therefore, the dominant source of Case A evidence and knowledge in each case would often Case A was an academic centre located within a trauma change. What was important to the decision-maker at and orthopaedic department. More than the other two onepoint in time wasnot always thepresenceorcon- cases, case A appeared to have a positive view of the for- tent of evidence-based guidelines (macro). It could mal codified knowledge contained in clinical guidelines equally be any other type of evidence or knowledge, and what guidelines set out to achieve in the healthcare interacting with, for example, a surgeon’s training and sector. The surgeons working in case A took a popula- formal education (micro) or the contingencies of prac- tion perspective on clinical decisions. The culture, norms tice such as the structure of the hospital (meso). and political influence of the sector acted on case A in a positive fashion, as surgeons working here valued the Distinct approaches to decision-making processes and aims of generating clinical guidelines and Comparison across cases was essential to look beyond the goals of NICE as an organisation. Surgeons were ac- the initial impressions and see the findings through customed to answering questions using a larger popula- multiple lenses. We framed our cases according to tion frame of reference and suggested that they “may their general approach to decision-making and how subconsciously be following NICE guidance” (Junior sur- guidelines were implemented in each hospital. The geon) as it was indoctrinated in the organisational know- differences in decision-making contributed to vari- ledge and processes. One surgeon states that “NICE ation from the codified evidence contained in guide- guidelines help you look at cost effectiveness and evi- lines across all three cases. The analysis revealed that dence a lot more than you would think about in normal the individuals, groups and organisations in each case daily practice as an individual orthopaedic surgeon” had a distinct approach to decision-making. The approach (Consultant surgeon). was dependent on how the 17 types of knowledge and evi- Case A had the most advanced and formalised guide- dence interacted, changed and were used in orthopaedic line implementation processes i.e., their managerial practice. Guideline implementation was contingent knowledge. This was reflected in case A’s extensive and mediated through the distinct contextual environ- protocol documentation compared to the other two ments, which were subject to forces of the regulatory cases (17 versus 4 and 2). Each protocol was linked to a environment. piece of clinical guidance or an internal evidence Grove et al. Implementation Science (2018) 13:75 Page 7 of 14 Table 4 Narrative descriptions and an example of each source of evidence and knowledge depicted in Fig. 2 Evidence and knowledge types Narrative description Example from the data Macro External evidence created by healthcare The wider delivery of healthcare in England An inspection report from the regulators, e.g. CQC and GMC is governed by the UK Health and Social Care Quality Commission Care Regulators such as the Care Quality Commission. In orthopaedics, surgeons have to be registered with the General Medical Council and with their Royal College. Regulators are responsible for ensuring that surgeons are included in an up-to-date registry of qualified doctors and practice according to established standards The media and the influence of The mass media (or press) is a diversified An article in a newspaper describing ‘the press’ collection of resources who reach a large ‘good’ or ‘bad’ hip implants audience via mass communication The opinion of leaders and An opinion leader was an eminent individual An opinion leader could be a principal professional societies who had the ability to influence the opinion investigator of a large clinical trial in of the orthopaedic community on a subject orthopaedics. The professional societies matter for which they well known. The were the British Orthopaedic Association professional societies were larger organisations and the Hip Society who represented the groups and sub-groups of surgeons Formal codified knowledge Evidence or knowledge that is written A NICE guideline or article published in down can be shared and is easy to access a journal and available to the public Culture, norms and political influence The standards and accepted way of practicing The hierarchical structure of the healthcare of the sector in the UK healthcare context. Including the system. Political factors included strategies public delivery of services and formal and enforced by government and the medico- informal methods in which healthcare is legal challenges to practice organised in the NHS Meso Managerial knowledge Each hospitals’ business organisational NHS hospital resource issues such as time, processes which underpin day to day cost and safety or quality of services routines and capabilities of the Trust Organisational knowledge An extension of managerial knowledge A hospitals’ internal processes which are which has a wider structural emphasis. It not written down. Anecdotally referred is embedded in the processes of healthcare to as “the way we do things around here” organisations and influences the behaviour of staff The structure and location of the hospital The physical location of the hospital buildings The number of elective orthopaedic and departments and the structure of the theatres available to use hospital wards Evidence from implant manufacturing Information that came directly from Leaflets about a hip implant from a companies manufacturer’s representatives located in t manufacturer’s representative he hospital or indirectly through marketing Socialisation and association with colleagues Knowledge that came from the inside and Evidence of the outcome of a surgery spread within the defined clinical group, in from a colleague or knowledge that a this case the orthopaedic community mentor had passed on Micro Informal experiential knowledge Tacit knowledge that surgeons ‘know’ Represents a surgeon’s lifetime’s work, regarding how to behave and perform as and in turn their identity as a surgeon an orthopaedic surgeon Informal experiential knowledge built The tacit knowledge that surgeons ‘know’ Knowing which colleague to refer a up over time which has built up over time working in the difficult case to when the surgeon specific hospital but which can be difficult does not have the specific expertise to describe or experience Evidence from the professional hierarchy The layered social structure within the Described as the ‘clinical pecking hospital which conceptualised the order’ with the consultant surgeon superior and inferior relationships at the top between clinical staff Grove et al. Implementation Science (2018) 13:75 Page 8 of 14 Table 4 Narrative descriptions and an example of each source of evidence and knowledge depicted in Fig. 2 (Continued) Evidence and knowledge types Narrative description Example from the data Training and formal education The training and formal education of A Master’s degree in Evidence-Based healthcare professionals which are Medicine recognised through standard academic qualifications Apprenticeship style training and Personal training which occurs during Training gained through fellowship informal education each working day with senior colleagues programmes and practice-based learning Individual patient and surgeon factors Characteristics of the patient or surgeon Patients age or a surgeons years in practice that influenced clinical practice decisions Evidence linked to the innate ‘feel’ A description of the surgeon’s judgement, A surgeon not knowing exactly what of surgery skill, craft and instinct will occur during an operation until they started the surgery and can see and feel the operation takes place summary which had been produced by medical librar- and the likely outcome of an orthopaedic intervention ians. The implementation process was described by the described in NICE guidelines were only part of the Chair of the Guideline Committee: knowledge picture which had to fit into the ever chan- ging context of practice. However, external evidence cre- We started monitoring NICE implementation in the ated by regulators and managerial and organisational trust in 2011/12 when we set up the NICE knowledge could limit the behaviour and decisions of implementation group…when NICE issues their surgeons. For example, surgeons working here were guidance, I forward a list to the NICE administrator restricted in the hip replacement implants, they could who would send it out to lead clinician in select based on cost and procurement contracts (meso) orthopaedics…If it’s a clinical guideline, or quality and thresholds established by the Orthopaedic Device standard…there’s recommendations in there, and a Evaluation Panel in the UK (macro). form of a baseline assessment with the recommendations in…they (the clinician) have to Case B indicate if we’re compliant or not. (Hybrid surgeon) Case B was a small hospital Trust split between two geographical sites. Case B demonstrated an ‘“it depends” The surgeons in case A demonstrated a distinctly approach to decision-making produced by the binary different trend in their confidence and appreciation of characteristics of the Trust’; this was bought about by clinical guidance (individual surgeon factors). We con- the structure and location of the hospital. The binary sider that this was due to their departmental focus on characteristics reflect the two distinct hospital locations research (apprenticeship style training) and academic in case B. By name, the hospitals were one Trust; how- output developed through training and formal education ever, the day-to-day operations and decision-making in Evidence Based Medicine (EBM), and beliefs regard- practices were separated and distinct. Each hospital loca- ing the importance of NICE and the EBM approach. A tion in case B had their own way of doing things (organ- quote from a surgeon illustrates this: isational knowledge) and staff acted protectively to maintain them. The ‘it depends approach’ signifies the “We do try as much as possible to follow the basis of participants views that what happened in practice evidence...So the typical patients would be where we depended on which of the two case B hospital sites they would need to use evidence. There is not much of a were located at when the decision was made. dilemma about someone with end stage osteoarthritis In case B, external evidence created by regulators was as the guidance shows the pathway that they should not regarded as important. Formal codified evidence in take.” (Consultant surgeon) NICE guidelines was not valued by the clinical staff. Guidelines were often considered in a negative light and The distinct approach to decision-making in case A considered to be the responsibility of hospital admin- was ‘pragmatic EBM decision-making’, where the trad- istrators and managers (i.e., managerial knowledge). itional approach to evidence-based decision-making rec- Described by a hospital board member below: ommended in clinical guidelines was suitable, but not sufficient for their practice. Surgeons in case A acknow- “I think NICE guidance is very much just seen as ledge traditional EBM and focused on pragmatic EBM. another layer of administration for clinicians. If no This practice-based approach to EBM considered the one’s looking at whether you’re following NICE important point that knowing what to do, how to do it, guidance or not they just sit on a shelf, unless you Grove et al. Implementation Science (2018) 13:75 Page 9 of 14 have a very active team of clinicians who take this on elsewhere in the Trust. They were also resistant to pressure board. But that’s not a consistent.” (Hospital board from external policymakers and evidence from regulators. member) This divide across professional boundaries compounded by the professional hierarchy and cultural norms of the sector The negative value attached to NICE and guidelines made guideline implementation challenging. Nevertheless, was echoed by the surgeon in the quote below, in which knowledge and evidence generated through managerial they demonstrate the importance of socialisation and as- and organisational knowledge was present in the wider sociation with colleagues when making clinical decisions: hospital organisation: “NICE are not being proactive enough, I would say, in We have standard operating procedures for NICE, terms of making recommendations on prostheses and which encourage [surgeons] to write their own action they could do a lot more. There is very little sort of plan (Trauma & Orthopaedic Manager) robust evidence to guide practice so you rely on other peoples’ anecdotal experience and normal practice to However, observations of practice revealed that man- help guide what, what works and what doesn’t” agerial and organisational knowledge enacted through (Consultant surgeon) processes attached to NICE guidelines and governance belonged in the managerial and administrative domain, Within the organisational processes, guidelines were not within surgical staff. Surgeons reported having “never not considered an important part of practice in case B. seen” the organisation’s NICE process (Consultant Compliance to NICE guidelines as evidence of external surgeon). This is reflected in the observation note below: regulation did not appear to be valued and hence imple- mentation was haphazard, as described by a hospital NICE was rarely noted as an influential factor in the administrator: day-to-day activities of surgeons. Surgeons I spoke to were unaware their hospital had a NICE process, they I send (guidance) out to the General Manager and would respond “do we have one?”“I’ve never seen it”. then they will send it to the most appropriate person. (Observation note) We used to meet to discuss if we were compliant… but now we send a questionnaire out…so they have to Clinical practice decisions were made using knowledge do is tick ‘Yes’ and ‘No’ but they do not always and evidence gained through socialisation and associ- respond. (Administrator) ation with colleagues. Surgeons possessed resilient ex- periential knowledge built up over time as the majority What mattered for implementation in case B was the had been working in this hospital for their entire careers dynamics of organisational change and leadership enacted and were relatively separate from ‘outside’ knowledge. through evidence from the professional hierarchy in the Formal codified evidence in clinical guidelines had to two orthopaedic departments within the Trust. The indi- compete with the complex social systems that existed in vidual surgeon factors and the differing characteristics and the hospital. For example, case C was a referral centre processes of the two groups of surgeons headed up by which specialised in performing complex hip replace- opinion leaders meant that guideline implementation ment revision surgery. In this context, clinical guidelines across the organisation was difficult. The distinct groups appeared to be less important because the approaches, struggled to work together and share organisational know- techniques and implants needed to perform hip revisions ledge because the specific contexts that clinicians were were specialist (the innate feel of surgery) and therefore socialised into differed. For example, evidence from im- not included in guideline recommendations. One sur- plant manufactures carried greater weight in one location geon noted “NICE, is irrelevant. They don’t tell me compared to the other. The socialisation and association anything I NEED to know” (Consultant surgeon). The with colleagues reflected the behaviour and norms that specialist surgeons working in case C referred to a stand- guided or regulated the action and decision-making of ard hip replacement operation as “boring” and, hence the individuals at case B. work of other surgeons “on the treadmill” who are outside of their community or social system (Consultant surgeon). Case C In this sense, guidelines were not valued or applicable to Finally, case C was an orthopaedic department located in a their specialist work. The informal experimental know- teaching hospital. In case C, ‘socialised decision-making ledge and informal experiential evidence built up over time was prominent and evidence was discretionary’.The ortho- appeared to take precedence in clinical decision-making paedic department in case C was closed to the influence of processes and therefore restricted guideline implementa- administrators and managerial knowledge emerging from tion of guidelines for this group of surgeons. These Grove et al. Implementation Science (2018) 13:75 Page 10 of 14 dominate types of knowledge and evidence were grounded to guideline implementation in orthopaedic surgery. in the experience of surgical work in practice and legacy These reflect the relationship between guideline imple- knowledge about the organisational functions which sur- mentation and knowledge and evidence that is actually geons developed as a consequence of working there for a used in orthopaedic practice. Orthopaedic surgeons in long period of time. our study held ambivalent or negative attitudes to- wards clinical guidelines. They did not privilege this Discussion formal codified evidence because it originated outside In orthopaedic surgery, clinical guidelines are an import- of orthopaedics and did not contain the micro sources ant part of practice as they not only help guarantee of knowledge and evidence (e.g., experience, training, safety and encourage quality improvement but also en- individual characteristics, and the innate feel of sur- sure that NHS resources are used appropriately [32]. gery)thatwere consideredmoreimportant forsur- The aim of this study was to identify where, when and geons’ decision-making. However, the culture and how evidence and knowledge are used in healthcare norms of EBM, identified in case A, demonstrate that decision-making in orthopaedic surgery and how vari- it was possible to positively influence guideline imple- ation in evidence and knowledge contributes to differ- mentation. Consideration of the power of professional ences in the implementation of clinical guidance in hierarchies is vital, as surgeons working at the top of practice. Previous scholars have counted and categorised the hierarchy, such as clinical leaders can restrict or evidence using broader taxonomies, which describe diminish the influence of mangers and policymakers. knowledge as individual, group, tacit or explicit [33–35]. Organisational constraints linked to financial restric- Others emphasise the role of the person in the activity tions, regulation and procurement-influenced imple- to distinguish between action, doing and practice, and mentation and a lack of focus on clinical guidelines in knowledge facts and processes [36]. It is important to orthopaedic practice. The presence of organisational highlight that what is considered evidence and know- processes and protocols could not ensure what guide- ledge is highly contested and influenced by the environ- lines were valued and used. What was more important ment in which it is used [10]. is what evidence and knowledge transferred from surgical In fulfilling our aim, we discovered 17 different types colleagues and professional societies. of knowledge and evidence which were used in ortho- It is likely that many of the 17 types of knowledge and paedic surgery. During our case comparison, it became evidence identified in this research would not have been clear that the dynamic selection, combination and use of discovered without the structured, comparative case each type of knowledge and evidence influenced the im- study approach used in our study. One of the aims of plementation and use of clinical guidance in practice. At our research was to go beyond reports in the previous the time of study, none of the cases could definitively guideline implementation literature [3–9, 37]. It is sig- provide evidence to demonstrate that they were acces- nificant that a large number and diverse range of know- sing, using and monitoring guideline recommendations. ledge and evidence types acting across the entire domain We examined implementation of guidelines by individ- of healthcare emerged from the three case studies of ual surgeons, groups of surgeons and the Trusts as a NHS practice. Previous research has identified a consid- whole. Interestingly, implementation could differ be- erable number of barriers and facilitators to the imple- tween these levels. mentation of clinical guidelines but often they are too In the context of orthopaedic surgery, the process of generic or limited in scope, for example, the difficultly of privileging different types of knowledge and evidence engaging individual clinicians or problems with process in the context of surgery resulted in three distinct ap- and resource issues within hospitals [7, 13, 14]. proaches to decision-making in orthopaedic surgery. We have demonstrated that guideline implementation These include ‘pragmatic EBM decision-making’ (case A), and subsequent evidence-based practice were not always where NICE guidelines failed to deliver all the knowledge possible or preferable in the three cases. Comparative and evidence needed to make clinical decisions when case study analysis revealed dynamic contextual differ- organisational context restricted surgeon choice. An ‘“it ences and variation in practices and processes between depends” approach produced by the binary characteristics’ the three hospitals. For example, some surgeons had of case B linked to the professional hierarchy of surgeons strict limits placed on the orthopaedic implants they working in separate geographical locations and ‘socialised could order within their hospital. This restricted their decision-making’ where evidence was discretionary due to implant decisions. However, this varied across the cases the strong influence of socialisation and informal know- and differences were found in implant selection practices ledge sharing between surgeons in case C. and processes. This variation had a direct impact on the The similarities and differences between the three uptake and use of evidence in practice and demonstrates approaches generate key contextual dimensions specific the problems of effectively implementing standardised Grove et al. Implementation Science (2018) 13:75 Page 11 of 14 guidance in surgery. One of the aims of clinical guidance Our study has limitations. The direct observation of is to reduce unjustified variation in practice, but we healthcare professionals in their practice may have found that this was not achievable or appropriate in all driven a change to ‘good’ or ‘better’ behaviour by partici- contexts. pants; a phenomenon known as observer bias [39]. To Our findings encourage more research into how to en- ensure the quality and rigour of our data, we extended gage individuals and groups of healthcare professionals our access and observation as much as possible, whilst to consider the content of clinical guidelines and how it also conducting crosschecks and validation during inter- might add to their decision-making processes. The over- views and between different individuals and professional arching view that guidelines are the responsibility of groups. We sampled three hospitals from the population managers and administrators demonstrates a lack of of 135 hospital Trusts in England which deliver hip re- ownership of the guideline in general in orthopaedics. placement services [40]. Although the sample was small, Also, in the context of orthopaedics, our findings pro- we aimed to achieve a broad representation of the types vide insight into approaches to knowledge mobilisation of elective orthopaedic services available in England and targeted at communities of practice as an area for inves- aimed to produce in-depth rather than a breadth in our tigation and improvement. Improvements in the uptake case study design and data. and use of NICE guidance in orthopaedic practice will require the development, presentation and dissemination Policy and practice implications of evidence-based guidelines in surgery to be better Over the last 20 years, there have been significant tailored to the orthopaedic community. changes in the way policy-making organisations such as NICE create and disseminate guidance to improve health Strengths and limitations and social care. What appears to have remained constant The key strength of our research is the use of case studies is the way in which codified knowledge in guidance is to examine the context of guideline implementation. In produced with an assumption that a linear ‘push’ fashion achieving our aim, we were able to discover and explain will ensure that it is received and acted on by clinicians the gap in implementation of clinical guidelines in ortho- working in healthcare organisations. The findings of our paedics. Uptake and use of guidelines, even when study reconfirm and extend our knowledge of the limits grounded on the findings of empirical research including of this approach for improving the use of guidance and gold standard randomised controlled trials, were not guar- for reducing variation in practice for orthopaedics. anteed in practice. This suggests that well-developed In this study, we have raised the issue of whether guidelines are necessary but not sufficient to achieve the NICE guidelines are ever likely to be appropriate for the goals of policymaking organisations such as NICE and field of orthopaedics. This is due to the wide range of Scottish Intercollegiate Guidelines Network (SIGN). Case knowledge and evidence identified as influential to study methods allowed us to describe the decision-making decision-making, coupled with the differences in guide- context in detail. This method has provided a greater line implementation across the structural levels. How- depth of description of the evidence and knowledge ever, we do not consider that the evidence contained in sources than have been outlined in previous literature, for guidelines is inappropriate. Instead, the ways in which example, expanding the tacit-explicit-group-individual knowledge and evidence were privileged differently by categorisations. The way in which evidence and know- individual practitioners, groups of surgeons and organi- ledge interacted with context produced variation in the sations meant that guidelines were rarely accessed as a extent to which guidelines are implemented and used in beneficial evidence source. Surgeons were not concerned orthopaedics. about what guidelines recommended. What was import- A second strength of our research is the use of ant was their definition of knowledge and evidence and multiple data sources (interviews, observations and how this interacted with understandings of knowledge document analysis) to study the same phenomena. The and evidence in their group and wider organisation. In combination of methods facilitated us to overcome the this study, ‘one size’ guidance could never ‘fit all’ the sur- weaknesses that emanate from selecting a single method geons’ requirements and therefore, the guidance had to study the complex process and practice of guideline limited value in their specific circumstances. implementation [38]. We triangulated data to enhance Nevertheless, evidence in guidelines represents best prac- the credibility of our analysis and findings. When data tice, and NICE and SIGN must produce recommendations from one source substantiates a pattern from another, for healthcare. We have provided evidence to suggest that the findings are stronger and better substantiated [26]. the current modes of transfer and implementation are inef- Comparison across our three comparative cases enabled fective. Changes could be made to the process of guideline us to generate a more sophisticated understanding of creation, dissemination or even regulation to move towards the data we collected [22, 23]. effective knowledge mobilisation. For example, a more Grove et al. Implementation Science (2018) 13:75 Page 12 of 14 inclusive process of involving clinicians in guideline devel- all sides of knowledge boundaries to understand and accept opment would be welcomed. The current system relies on why certain options are chosen and actions are taken, espe- clinicians being aware of guideline updates, rather than cially if they vary from guideline recommendations. being enabled to actively volunteer their contributions. Tar- geting the orthopaedic community through professional Conclusion meetings, networks and clinical leaders would communi- The research aimed to explore guideline implementation cate the need for involvement and increase discussion through the application of comparative case studies about new guidelines in contrast to the current process of which investigate the use of NICE clinical guidelines in one-way dissemination by policymakers. decisions made in elective orthopaedic surgery in the Regulation was a valuable mechanism as it achieved a NHS. The results of our study highlight the range of desired outcome for achieving targets and controlling the complex and competing sources of evidence and know- behaviour of the surgeons. However, moving towards ledge which influence the work practices of healthcare regulation as the norm did not appear to be a desirable professionals. Case study analysis revealed three distinct option for most of the professional groups in this study. styles of orthopaedic practice which represent the ways Knowledge and evidence from external regulators did not in which 17 types of knowledge and evidence were used hold the same positive status achieved by the knowledge during decision-making. The way in which evidence and and evidence emanating from colleague and professional knowledge were selected and used impacted on how networks. Restricting the discretion and authority of clin- guidelines were implemented in the orthopaedic special- ical professionals by increasing regulatory power would ity. Findings from the case comparison reflect the com- not be recommended. Instead, interventions which take plexity of evidence-based decision-making in the highly advantage of the positive knowledge mobilisation between professionalised organisationally regulated context of orthopaedic colleagues are encouraged, as they may im- surgery. Our results could be used to guide the develop- prove the sharing of evidence-based practice. ment of implementation interventions that are grounded Improvements need to be made to how healthcare in the findings of this study. New approaches to imple- professionals working in hospitals see and think about mentation need to appreciate and incorporate the di- evidence from guidelines in combination with other verse range of knowledge and evidence which influences knowledge and evidence sources. Improvement interven- clinical decisions in orthopaedics and to take account of tions are required to help users of guidance identify the changing contextual situations in which decisions ‘where they are at’ in their decision-making processes. need to be made. Clinical guidelines were often unable to provide a solu- tion to a decision problem; therefore, it is important to understand what other types of evidence and knowledge Additional file are available or used by others. Practitioners need to acknowledge the difference between certain types of Additional file 1: An example of a NICE guidance recommendation. knowledge as positive or negative to patient care. Where Technology appraisal guidance [TA304] [17]. Total hip replacement and resurfacing arthroplasty for end-stage arthritis of the hip. Interview topic possible, those working in healthcare should focus on guide. (DOCX 15 kb) reducing undesirable types of evidence and knowledge present in their organisation. This could be an area for improvement work. However, surgeons in this study Abbreviations AHPs: Allied Health Professionals (including Occupational Therapists, were often unaware of or ambivalent about the conse- Operating Department Practitioners, Physiotherapists, Nurses and Healthcare quence of their decisions because processes were not Assistants); EBM: Evidence-based medicine; NHS: National Health Service; open, transparent or subject to feedback loops. SIGN: Scottish Intercollegiate Guidelines Network Healthcare practitioners could take a more transparent approach in understanding the evidence that is driving Acknowledgements their decisions and how guidelines may fit into the pic- We thank the three hospital departments who agreed to participate in the research project, and all the NHS staff who took part in the interviews and ture. This will facilitate practitioners in deciding whether assisted with observations. guideline recommendations are appropriate in their con- text. If not, other knowledge sources such as clinical ex- Funding perience could take precedence and be shared, explained Amy Grove was supported by a National Institute for Health Research and understood, rather than frowned upon by managers Doctoral Fellowship programme project number 2013-06-064. Aileen Clarke and administrators and recorded as an organisational and Graeme Currie were partly supported by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care risk. Practice-based knowledge was rarely shared be- West Midlands at University Hospitals Birmingham NHS Foundation Trust. tween the professional groups in this study. Encouraging The views expressed are those of the authors and not necessarily those of open decision-making processes might enable those on the NHS, the NIHR or the Department of Health and Social Care. Grove et al. Implementation Science (2018) 13:75 Page 13 of 14 Availability of data and materials 9. Platt C, Larcombe J, Dudley J, McNulty C, Banerjee J, Gyoffry G, Pike K, The datasets generated and analysed during the current study are not Jadresic L. Implementation of NICE guidance on urinary tract infections in publicly available due to restrictions of ethical approvals obtained for this children in primary and secondary care. Acta Paediatr. 2015;104:630–7. study. 10. Gkeredakis E, Swan J, Powell J, Nicolini D, Scarbrough H, Roginski C, Taylor- Phillips S, Clarke A. Mind the gap: understanding utilisation of evidence and Authors’ contributions policy in health care management practice. J Health Organ Manag. 2011;25: All authors (AG, AC, GC) have made substantial contributions to the design 298–314. of the study. AG collected the data, and all authors analysed and interpreted 11. Currie G, El Enany N, Lockett A. Intra-professional dynamics in translational health the data. All authors have been involved in drafting the manuscript and research. The perspective of social scientists. Soc Sci Med. 2014;114:81–8. have given final approval of the version to be published. All authors agree to 12. Davies HTO, Powell AE, Nutley SM. Mobilising knowledge to improve UK be accountable for all aspects of the work in ensuring that questions related health care: learning from other countries and other sectors—a to the accuracy or integrity of any part of the work are appropriately multimethod mapping study. Health Serv Deliv Res. 2015;3:27. investigated and resolved. 13. Denny K. Evidence-based medicine and medical authority. J Med Humanit. 1999;20:247–63. Ethics approval and consent to participate 14. Timmermans S. From autonomy to accountability: the role of clinical practice The study was approved by the Biomedical and Scientific Research Ethics guidelines in professional power. Perspect Biol Med. 2005;48:490–501. Committee of The University of Warwick, England (approved June 2nd 2014; 15. Grove A, Johnson R, Clarke A, Currie G. Evidence and the drivers of variation reference number REGO-2014-645) and the Research and Development in orthopaedic surgical work: a mixed method systematic review. Health departments of each of the three hospital sites (case A approved on 30 June, Syst Policy Res. 2016;3:1. 2014, case B approved on 23 October, 2014, case C approved on 21 August 16. Ferlie W, Wood M, Fitzgerald L. Some limits to evidence-based medicine: a 2014). In line with the ethical approval, each participant will be asked to sign case study from elective orthopaedics. Qual Health Care. 1999;9:99–107. an informed consent form as will be provided with information outlining the 17. NICE Technology Appraisal (TA304). Total hip replacement and resurfacing purpose of the study and their rights to withdraw. arthroplasty for end-stage arthritis of the hip. NICE. 2014. https://www.nice. org.uk/guidance/ta304. Accessed 30 Nov 2017. Consent for publication 18. Stake R. The art of case study research. London: Sage; 1995. Informed consent was obtained to include anonymised data in any 19. Grove A, Clarke A, Currie G. The barriers and facilitators to the publications. Confidentiality is protected as the data is anonymised. implementation of clinical guidance in elective orthopaedic surgery: a qualitative study protocol. Implement Sci. 2015;2:81. Competing interests 20. Sandelowski M. The problem of rigor in qualitative research. Adv Nurs Sci. The authors declare that they have no competing interests. 1986;8:27–37. 21. Eisenhardt KM. Building theories from case study research. Acad Manag Rev. 1989;14:532–50. Publisher’sNote 22. Mantere S, Ketokivi M. Reasoning in organization science. Acad Manag Rev. Springer Nature remains neutral with regard to jurisdictional claims in 2013;38:70–89. published maps and institutional affiliations. 23. Parlett M, Hamilton D. Evaluation as illumination: a new approach to the study of innovatory programs. In: Glass G, editor. Evaluation studies review Author details 1 annual, vol. 1. Beverly Hills: Sage; 1976. Division of Health Sciences, Warwick Medical School, University of Warwick, 2 24. George AL, Bennett A. Case studies and theory development in the social Coventry, UK. Entrepreneurship & Innovation, Organising Healthcare sciences. Cambridge: MIT Press; 2005. Research Network, Warwick Business School, University of Warwick, Coventry, 25. Yin R. Case study research. Beverly Hills: Sage Publications; 1984. UK. 26. Creswell J, Clark V. Designing and conducting mixed methods research. London: Sage Publications; 2010. Received: 18 January 2018 Accepted: 23 May 2018 27. Kinsman L, Rotter T, James E, Snow P, Willis J. What is a clinical pathway? Development of a definition to inform the debate. BMC Med. 2010;8:131. 28. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. References 2006;3:77–101. 1. NICE. About. 2017. http://www.nice.org.uk/about. Accessed on 6 Dec 2017. 29. Johnson R, Grove A, Clarke A. Pillar integration process: a technique to 2. SIGN. Who we are. 2017. http://www.sign.ac.uk/who-we-are.html. Accessed integrate data in mixed methods research. J Mixed Methods Res. 2017:1–20. on 6 Dec 2017. https://doi.org/10.1177/1558689817743108. 3. Lowson K, Jenks M, Filby A, Carr L, Campbell B, Powell J. Examining the 30. Pettigrew AM. Longitudinal field research on change: theory and practice. implementation of NICE guidance: cross-sectional survey of the use of Organ Sci. 1990;1:267–92. NICE interventional procedures guidance by NHS Trusts. Implement Sci. 31. Pope C, Robert G, Bate P, Le May A, Gabbay J. Lost in translation: a multi- 2015;10:93. level case study of the metamorphosis of meanings and action in public 4. Sheldon T, Cullum N, Dawson D, Lankshear L, Lowson K, Watt I, West P, sector organizational innovation. Public Adm. 2006;84:59–79. Wright D, Wright J. What’s the evidence that NICE guidance has been 32. NICE. The NICE way. Lessons learned from the National Institute for Health implemented? Results from a national evaluation using time series analysis, and Care Excellence. 2014. https://www.nice.org.uk/news/article/using-nice- audit of patients’ notes, and interviews. BMJ. 2004;329:999. s-approach-to-base-policy-decisions-on-evidence-could-help-save-billions. 5. Weng Y-H, Kuo K, Yang C-Y, Lo H-L, Chen C, Chiu Y-W. Implementation of Accessed 30 Nov 2017. evidence-based practice across medical, nursing, pharmacological and allied 33. Cook S, Brown J. Bridging epistemologies: the generative dance between healthcare professionals: a questionnaire survey in nationwide hospital organizational knowledge and organizational knowing. Organ Sci. 1999;10: settings. Impact Sci Soc. 2013;8:112. 381–400. 6. Francke A, Smit M, de Veer A, Mistiaen P. Factors influencing the 34. Nonaka I. A dynamic theory of organizational knowledge creation. Organ implementation of clinical guidelines for health care professionals: a Sci. 1994;5:14–37. systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38. 7. Fitzgerald A, Lethaby A, Cikalo M, Glanville J, Wood H. Review of systematic 35. Lam A. Tacit knowledge, organizational learning and societal institutions: an reviews exploring the implementation/uptake od guidelines. York Health integrated framework. Organ Stud. 2000;21:487–513. Economics Consortium. 2014. https://www.nice.org.uk/guidance/ph56/ 36. Orlikowski W. Knowing in practice: enacting a collective capability in evidence/evidence-review-2-431762366. Accessed 8 Jan 2018. distributed organizing. Organ Sci. 2002;13:249–73. 8. Royal College of Physicians. Implementing NICE public health guidance for 37. Berry K, Haddock G. The implementation of the NICE guidelines for the workplace: a National Organisational Audit of NHS Trusts in England. schizophrenia: barriers to the implementation of psychological 2015. https://www.rcplondon.ac.uk/projects/outputs/implementing-nice- interventions and recommendations for the future. Psychol Psychother. public-health-guidance-workplace-2013-round-2. Accessed 8 Jan 2018. 2008;81:419–36. Grove et al. Implementation Science (2018) 13:75 Page 14 of 14 38. Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton S. Integrative approaches to qualitative and quantitative evidence. London: NHS Health Development Agency; 2004. 39. Snyder ML, Frankel A. Observer bias: a stringent test of behavior engulfing the field. J Pers Soc Psychol. 1976;34:857. 40. NHS Confederation. NHS statistics, facts and figures. 2017. http://www. nhsconfed.org/resources/key-statistics-on-the-nhs. Accessed on 6 Dec 2017.

Journal

Implementation ScienceSpringer Journals

Published: May 31, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off