A systematic approach to develop a core set of parameters for boards of directors to govern quality of care in the ICU

A systematic approach to develop a core set of parameters for boards of directors to govern... Abstract Objective Hospital boards are legally responsible for the quality of care delivered by healthcare professionals in their hospitals, but experience difficulties in overseeing quality and safety risks. This study aimed to select a core set of parameters for boards to govern quality of care in the intensive care unit (ICU). Design Two-round Delphi study. Setting Two university hospitals in the Netherlands. Participants An expert panel of 12 former ICU patients or their family members, 12 ICU nurses, 12 ICU physicians and 12 members of boards of directors and quality managers. Main outcome measures Participants indicated the relevance of existing parameters for assessing the quality of ICU care for governance purposes (round 1) and selected 10 quality parameters that together provide boards of directors with a good representation of quality of care in their ICU (round 2). Results We identified 122 quality parameters related to care in the ICU, which we limited to a short list to present to participants in round 1. The response rate was 94% in round 1 and 85% in round 2. The final set consisted of the 10 most frequently selected quality parameters per hospital. Five parameters were included in both sets; all related to patient safety and continuous quality improvement. Conclusions Parameters in the core set were mostly qualitative and generic, rather than quantitative and ICU-specific in nature. To engage in a true dialog about quality of care, boards are more interested in the story behind the numbers than in just the numbers themselves. consensus methods, governance, hospital care, intensive care, quality improvement, quality measurement Introduction Hospital boards have a legal and moral responsibility to ensure high quality of care delivered by healthcare professionals in their hospitals [1]. They are the ones held accountable by government and insurers but experience difficulties in overseeing quality and safety risks [2–5]. Problems such as insufficient resources, gaps in board members’ experience and skills and difficulties to oversee the quality of care in the entire hospital make it difficult for boards to govern the quality of healthcare in their hospital [3]. Currently, hospitals measure and collect hundreds of quality parameters [6]. Their large number however limits their use: it is difficult to obtain an overview and to recognize signals, meaning an important opportunity to continuously improve care is missed. For boards of directors, especially, many of these parameters are of limited use, due to their detailed and specific nature. Professionals experience a growing pressure to provide requested information, which compromises the internal use of those quality indicators [7]. To improve quality and fight the waste of energy, enthusiasm and (financial) resources, it is paramount that the generated information is actually used by healthcare professionals, managers and directors [8]. This project was aimed at providing boards of directors with insight into quality and safety of care in the intensive care unit (ICU), in order to timely recognize possible quality problems. We selected the ICU as the focus of the project because it is one of the hospital’s core departments, in which critically ill patients are cared for with high-risk interventions. Suboptimal quality and safety of care have a tremendous impact on this patient group as well as for the hospital, in the form of critical incidents and legal claims. The ICU cares for many different types of patients and is therefore an important cog in the hospital’s machine. When quality of ICU care is good (or bad), this will affect other departments in the hospital. Because of the complexity of ICU care, the importance of a core set of key quality parameters for boards of directors is clear. However, it is unknown which quality information and parameters are suitable for boards of directors to govern quality of ICU care. To assist boards, we systematically asked an expert panel of boards members, patients and their family members and healthcare professionals to determine which quality information is relevant for good governance. The aim of this study was to select a core set of parameters for boards of directors to govern quality of care in the ICU. Methods To select a core set of quality parameters, we used the modified Delphi method. This systematic, iterative methodology is used to collect and distill knowledge on a specific topic from a panel of experts. In multiple rounds, experts are confronted with each other’s ideas and viewpoints. This method has important advantages: it can be carried out via questionnaires, meaning it is both time efficient as well as anonymous, which avoids the possible negative impact of power imbalances and participant dominance [9, 10]. We performed separate Delphi studies in two Dutch academic hospitals between March and June 2016. The Delphi studies were conducted separately to allow for tailoring of the core set to the local setting and the needs and desires of each hospital’s stakeholders. Our modified Delphi study consisted of four steps, described below. Ethical approval was sought from the Research Ethics Committee of the Radboud University Nijmegen Medical Centre (registration number: 2016/2525); the committee judged that ethical approval was not required under Dutch National Law. All participants received written information about the project and its aims and were subsequently invited to participate. Step 1. Extraction of quality parameters from the available quality information The basis for the selection of the core set was all available quality information pertaining to the ICU. From January to March 2016, two researchers (AO and MZ) systematically inventoried all quality information registered in both hospitals. Quality information is generated for different information-requesting stakeholders, with different goals (quality improvement, governance and accountability), and is varied in nature (structure, outcome, or process indicators). From this inventory, two researchers (AO and MZ) extracted the quality parameters that could be used to govern quality of care in the ICU. In consultation with the head of each participating ICU, we limited the long list by removing doubles (parameters registered for separate stakeholders, with (partially) overlapping definitions) and quality parameters very likely to be considered ‘not relevant’. The resulting list of quality parameters was divided into seven domains: organization of ICU care; effectiveness of ICU treatment; incidence and prevention of complications and iatrogenic injury; learning from complications and incidents; functioning of individual healthcare professionals and teams; experiences of patients and family; patient outcomes and functional status after discharge. The domain names, quality parameter names, parameter descriptions and the domain allocation were reviewed by each ICU department head and modified if necessary. The resulting list served as the basis for the questionnaire of Delphi round 1. Step 2. Delphi round 1: relevance of quality parameters The Delphi panels consisted of 24 experts in each hospital: 6 former ICU patients and family members of former ICU patients; 6 ICU nurses; 6 intensivists; 6 managers or board members (including ICU department head, quality managers, hospital board of directors). The former patients and family members were recruited through a post-ICU care polyclinic (which sees patients 3 months (or longer) after ICU discharge for a follow-up appointment) and through the foundation for Family and patient Centered Intensive Care, meaning patients had not necessarily been admitted to the ICUs of hospital A or B. The physicians and nurses were selected on the basis of their proven interest or expertize in quality of care (for example, membership of a quality assurance committee). All experts invited agreed to participate. In the first round, the expert panel received a questionnaire with the list of quality parameters with a brief description, divided into seven domains. The participating professionals received a link to an online version of the questionnaire via e-mail. The patients and family members were sent a Word document or printed version and were guided through the process via telephone (by AO). The experts were instructed to individually rate each quality parameter on a nine-point Likert scale (ranging from ‘not at all relevant’ [1] to ‘very relevant’ [9]) by asking: ‘For each parameter, indicate how relevant you think this information would be for a board member to determine quality of care in the ICU.’ Based on the relevance scores, the parameters were divided into three categories: Non-exclusion A convincing majority of participants considered the parameter relevant: at least 70% of participants scored 7, 8 or 9 and the median was at least 8. Equivocal Extremely skewed distribution: at least 30% of participants scored 1, 2 or 3 and at least 30% of participants scored 7, 8 or 9. Or Somewhat skewed distribution: at least 70% of participants scored 7, 8 or 9 and the median was 7 or lower. Exclusion (all other cases) Parameters in the ‘non-exclusion’ or ‘equivocal’ categories were included in the questionnaire of round 2 [11, 12]. Step 3. Delphi round 2: selection of a core set of quality parameters In this second round, the experts received the questionnaire via e-mail. If so desired, patients and family members were again guided through the completion process by phone. They were presented with the remaining quality parameters divided into the same domains as in round 1, and were asked the following: ‘In the questionnaire we invite you to select 10 quality parameters from the remaining parameters. Ten quality parameters that provide a board member with sufficient means to enter into a dialog with healthcare professionals about the quality of care in the ICU.’ For each quality parameter in the questionnaire we presented the distribution of round 1 participant scores across the Likert scale, the percentage of participants that scored the parameter in the highest tertile and the median score. (For a fragment from the second round questionnaire, see the online Supplemental Material.) The 10 most frequently selected parameters in each hospital formed the two core sets. Step 4. Feasibility study To evaluate the usefulness of the identified core set of quality parameters, we performed a feasibility study in both hospitals. To this purpose, we filled each core set with data from the previous 2 years. Where relevant, we displayed trends through time or a comparison with national averages. Per parameter, we added a concise description of the most notable observations or the main problem points, with planned improvement actions and current state of affairs, where present. If possible, we displayed information in figures or tables, with the use of signal colors green, orange and red. We discussed draft versions with each ICU department head and subsequently modified the core set where needed. In both hospitals, the complete core set was used in a conversation between ICU management (department head and financial manager/nursing manager) and a member of the board of directors. A researcher (AO) observed this conversation and asked short evaluation questions afterwards. Results Step 1. Quality information inventory The inventory of quality information yielded 122 individual quality parameters in total in both hospitals. A vast majority of information (72%) was collected for quality improvement purposes. Other indicators were collected for external accountability (17%) and internal governance (11%). We can distinguish four parameter levels: structure indicators (12%, for example, the 24 h availability of a specialized ICU physician), process indicators (40%, for example, adherence to hand hygiene guidelines), outcome indicators (39%, for example, the number of complications) and patient-reported experiences (9%). For the first Delphi round, we shortened the long list of 122 parameters to a list of 54 quality parameters in hospital A and 47 quality parameters in hospital B. For the number of parameters per domain in both rounds, see Table 1. Table 1 Number of quality parameters per domain Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 aAn internal audit is a multiple-source method that evaluates whether standards and regulations are being followed. The audit involves site visits, interviews, document analysis, surveys and observations. Auditors are colleagues from a different department than the department being audited. The internal audit results in recommendations for quality improvements. Table 1 Number of quality parameters per domain Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 aAn internal audit is a multiple-source method that evaluates whether standards and regulations are being followed. The audit involves site visits, interviews, document analysis, surveys and observations. Auditors are colleagues from a different department than the department being audited. The internal audit results in recommendations for quality improvements. Step 2. Delphi round 1 The response rates for round 1 were 96% (23 experts) and 92% (22 experts) for hospital A and B, respectively. Based on the results, 20 (hospital A) and 12 (hospital B) parameters were excluded. The distribution of the parameters across the three categories is shown in Table 2. Table 2 Results round 1 Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Table 2 Results round 1 Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Step 3. Delphi round 2 In hospital A, 22 experts (92%) participated in round 2, compared with 19 experts (79%) in hospital B. The 10 quality parameters most frequently selected by the participants from hospital A and hospital B can be found in Tables 3 and 4. Table 3 Core set selected in hospital A Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Table 3 Core set selected in hospital A Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Table 4 Core set selected in hospital B Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Table 4 Core set selected in hospital B Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 The parameters that were selected for the core sets can roughly be divided into three categories: parameters as signals for possible safety problems (for example, critical incidents and discussions of complications) parameters that show whether a department is proceeding through the plan-do-check-act (PDCA) cycle (for example, follow-up after audit results, and team climate); organizational preconditions for quality and safety of care (for example, presence and availability of an intensivist and nurse–patient ratio). Additionally, the experiences of former patients or their family members were selected in both hospitals. With regard to the preference for information regarding safety and improvement culture, there were no major differences between the four participant categories (physicians, nurses, patients and managers and board members). The core sets included mostly qualitative, aggregate information (such as the results of visitations and audits) rather than quantitative indicators (such as the percentage of infections or pressure sores, and protocol adherence rates). The 2 core sets of 10 quality parameters showed overlap: 5 of the 10 parameters were selected in both hospitals. Strikingly, the SMR most often frequently selected in hospital A is not present in hospital B’s core set. Conversely, a parameter with respect to medication error policy scores highest in hospital B, while going unselected in hospital A. (For a categorization of the core set parameters, see Table 5.) Table 5 Categorization of core set parameters Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Table 5 Categorization of core set parameters Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y The raw data set is available from the authors by request. Step 4. Feasibility study Both from this observation as well as from brief evaluation with the participants afterwards, we conclude that there was a true dialog about quality of care based on the presentation of the results of the parameters included in the core set: more so than is usually the case in these meetings, as indicated by the participants. The chosen format with aggregated information about quality and safety issues, improvement actions taken, and current status regarding those actions provided an excellent basis for the conversation. After the conversation, one board member said the following: ‘What I like about this is that it leads to a true conversation and the was the intent. That you reach the essence of what it’s about. It isn’t about a checklist or a scoring list, it’s about the conversation about where are the challenges, where are the worries, where are the bottlenecks.’ Discussion This project used the Delphi method to compile two core sets of ICU quality parameters. Boards of directors are the end users of the information produced. Their role in quality improvement is different from that of healthcare professionals, and this study showed that the information boards need to govern quality of care is therefore different in nature than the information healthcare professionals need to continuously improve quality of care. In the literature on quality dashboards, however, the focus appears to be on quality information for healthcare professionals: real-time information which is continually displayed [13, 14]. The parameters selected for both core sets, however, contain few parameters that could fruitfully be used for such a dashboard. From our conversations with board members, we conclude that board members would rather receive an overview of the most important problem areas, improvement actions and current state of affairs: an overview that can serve as a basis to carry out a dialog with the head of the department about quality of care. They are interested in the story behind the numbers rather than in just the numbers themselves. This is not an isolated finding; the Netherlands has recently seen growing criticism of the dominance of numbers in different sectors [15–17]. Most parameters in the core sets are generic rather than specific to the ICU setting, such as audit results, SMR, and critical incidents. In addition to organisational preconditions and patient experiences, the core sets mostly contains information about patient safety and information that indicates to what extent departments or professionals are in improvement mode. Is a department continuously working to optimize quality of care and does it have a true improvement culture? Is a department in a PDCA cycle and if so, in which phase? This project was not aimed at producing one generic set of parameters to fit all hospitals and departments. Rather, we considered whether the Delphi method would be a useful tool in compiling a tailor-made set for a specific department or hospital. A department that uses this method would not do away with the other established quality monitoring and improvement methods, such as monitoring by the healthcare inspectorate and automatic registering of quality indicators for internal quality improvement processes. This tailor-made core set would be used as a means to instigate a dialog about quality of care between department and board of directors. When we started this project, we expected the Delphi process to yield mostly quantitative indicators that could be aggregated and visualized on a quality dashboard. One of our most striking observations, therefore, is the fact that the expert panels selected mostly qualitative parameters. Furthermore, the overlap between both core sets is noticeable: half of the parameters is the same. Future research will have to show whether this is an incidental finding or points to a trend. If this method is conducted in different departments in the same hospital, this overlap might provide the means to set several general, generic parameters and add several department-, specialty- or disease-specific parameters. The modified Delphi method to generate the core set has several important advantages. Characteristic of this method is the purposeful selection of experts, meaning a limited number of participants is sufficient to generate valuable information. The inclusion of key figures in the project group make participant recruitment relatively easy and ensured a high response rate. Representation of the most important stakeholders—both board members, healthcare professionals as well as patients and their family members—led to a broadly supported core set. Because this method makes use of quality information that is already being collected in hospitals, for internal quality improvement or external accountability, no new quality parameters have to be generated and the amount of information to be registered does not increase. In addition, it means that the core set is truly tailored to the hospital which uses it; in both generated core sets, we recognize characteristic accents of both hospitals. Many generally used quality parameters have not been scientifically evaluated: information about reliability and validity is lacking. This is also the case for the parameters in our core sets. Therefore, we cannot say with certainty whether the core sets give a reliable and valid image of the quality of care in both ICUs. As a conversation starter about quality of care, however, the core set appears to be very well-suited. Acknowledgements We would like to thank all Delphi participants: the nurses, physicians and managers, and in particular, the former patients and their family members. We would especially like to express our gratitude to the foundation of Family and patient Centered Intensive Care, in particular Ed Kuijper, for their help with participant recruitment. Funding This project took place within the program ‘Board governance’. This program is led by the Netherlands Federation of University Medical Centres consortium Quality of Care and is financed by the Citrien foundation. This foundation supports the development of sustainable and broadly applicable solutions in healthcare and is made possible by the Netherlands Organisation for Health Research and Development. References 1 Parand A , Dopson S , Renz A et al. . The role of hospital managers in quality and patient safety: a systematic review . BMJ Open 2014 ; 4 : e005055 . Google Scholar CrossRef Search ADS PubMed 2 Goeschel CA , Berenholtz SM , Culbertson RA et al. . Board quality scorecards: measuring improvement . Am J Med Qual 2011 ; 26 : 254 – 60 . Google Scholar CrossRef Search ADS PubMed 3 Bismark MM , Studdert DM . Governance of quality of care: a qualitative study of health service boards in Victoria, Australia . BMJ Qual Saf 2014 ; 23 : 474 – 82 . Google Scholar CrossRef Search ADS PubMed 4 Jha AK , Epstein AM . A survey of board chairs of English hospitals shows greater attention to quality of care than among their US counterparts . Health Aff (Millwood) 2013 ; 32 : 677 – 85 . Google Scholar CrossRef Search ADS PubMed 5 Millar R , Mannion R , Freeman T et al. . Hospital board oversight of quality and patient safety: a narrative review and synthesis of recent empirical research . Milbank Q 2013 ; 91 : 738 – 70 . Google Scholar CrossRef Search ADS PubMed 6 Blume LH , van Weert NJ , Delnoij DM . How to manage external demands in hospitals—the case of atrium MC . Healthc (Amst) 2015 ; 3 : 157 – 9 . Google Scholar CrossRef Search ADS PubMed 7 van de Bovenkamp HM , de Mul M , Quartz JGU et al. . Institutional layering in governing healthcare quality . Public Administration 2014 ; 92 : 208 – 23 . Google Scholar CrossRef Search ADS 8 Conway PH , Mostashari F , Clancy C . The future of quality measurement for improvement and accountability . JAMA 2013 ; 309 : 2215 – 6 . Google Scholar CrossRef Search ADS PubMed 9 Boulkedid R , Abdoul H , Loustau M et al. . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review . PLoS One 2011 ; 6 : e20476 . Google Scholar CrossRef Search ADS PubMed 10 Hermens RP , Ouwens MM , Vonk-Okhuijsen SY et al. . Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline . Lung Cancer 2006 ; 54 : 117 – 24 . Google Scholar CrossRef Search ADS PubMed 11 Campbell SM , Braspenning J , Hutchinson A et al. . Research methods used in developing and applying quality indicators in primary care . BMJ 2003 ; 326 : 816 – 9 . Google Scholar CrossRef Search ADS PubMed 12 Stienen JJ , Tabbers MM , Benninga MA et al. . Development of quality indicators based on a multidisciplinary, evidence-based guideline on pediatric constipation . Eur J Pediatr 2011 ; 170 : 1513 – 9 . Google Scholar CrossRef Search ADS PubMed 13 Eckerson WW . Performance Dashboards: Measuring, Monitoring, and Managing Your Business . Hoboken, New Jersey : John Wiley & Sons , 2010 . 14 Kroch E , Vaughn T , Koepke M et al. . Hospital boards and quality dashboards . J Patient Saf 2006 ; 2 : 10 – 9 . 15 Steenhuis PH Meten is weten, maar is dat wel zo? Trouw. 2017 February 13, 2017. 16 Brinkgreve C , Bloemink S , Koenen E . Weten vraagt meer dan meten . Amsterdam : Amsterdam University Press , 2017 . 17 NVTZ . Zienderogen beter II. Veiligheid in perspectief. 2016 . © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal for Quality in Health Care Oxford University Press

A systematic approach to develop a core set of parameters for boards of directors to govern quality of care in the ICU

Loading next page...
 
/lp/ou_press/a-systematic-approach-to-develop-a-core-set-of-parameters-for-boards-sQ77Asr7sw
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care.
ISSN
1353-4505
eISSN
1464-3677
D.O.I.
10.1093/intqhc/mzy048
Publisher site
See Article on Publisher Site

Abstract

Abstract Objective Hospital boards are legally responsible for the quality of care delivered by healthcare professionals in their hospitals, but experience difficulties in overseeing quality and safety risks. This study aimed to select a core set of parameters for boards to govern quality of care in the intensive care unit (ICU). Design Two-round Delphi study. Setting Two university hospitals in the Netherlands. Participants An expert panel of 12 former ICU patients or their family members, 12 ICU nurses, 12 ICU physicians and 12 members of boards of directors and quality managers. Main outcome measures Participants indicated the relevance of existing parameters for assessing the quality of ICU care for governance purposes (round 1) and selected 10 quality parameters that together provide boards of directors with a good representation of quality of care in their ICU (round 2). Results We identified 122 quality parameters related to care in the ICU, which we limited to a short list to present to participants in round 1. The response rate was 94% in round 1 and 85% in round 2. The final set consisted of the 10 most frequently selected quality parameters per hospital. Five parameters were included in both sets; all related to patient safety and continuous quality improvement. Conclusions Parameters in the core set were mostly qualitative and generic, rather than quantitative and ICU-specific in nature. To engage in a true dialog about quality of care, boards are more interested in the story behind the numbers than in just the numbers themselves. consensus methods, governance, hospital care, intensive care, quality improvement, quality measurement Introduction Hospital boards have a legal and moral responsibility to ensure high quality of care delivered by healthcare professionals in their hospitals [1]. They are the ones held accountable by government and insurers but experience difficulties in overseeing quality and safety risks [2–5]. Problems such as insufficient resources, gaps in board members’ experience and skills and difficulties to oversee the quality of care in the entire hospital make it difficult for boards to govern the quality of healthcare in their hospital [3]. Currently, hospitals measure and collect hundreds of quality parameters [6]. Their large number however limits their use: it is difficult to obtain an overview and to recognize signals, meaning an important opportunity to continuously improve care is missed. For boards of directors, especially, many of these parameters are of limited use, due to their detailed and specific nature. Professionals experience a growing pressure to provide requested information, which compromises the internal use of those quality indicators [7]. To improve quality and fight the waste of energy, enthusiasm and (financial) resources, it is paramount that the generated information is actually used by healthcare professionals, managers and directors [8]. This project was aimed at providing boards of directors with insight into quality and safety of care in the intensive care unit (ICU), in order to timely recognize possible quality problems. We selected the ICU as the focus of the project because it is one of the hospital’s core departments, in which critically ill patients are cared for with high-risk interventions. Suboptimal quality and safety of care have a tremendous impact on this patient group as well as for the hospital, in the form of critical incidents and legal claims. The ICU cares for many different types of patients and is therefore an important cog in the hospital’s machine. When quality of ICU care is good (or bad), this will affect other departments in the hospital. Because of the complexity of ICU care, the importance of a core set of key quality parameters for boards of directors is clear. However, it is unknown which quality information and parameters are suitable for boards of directors to govern quality of ICU care. To assist boards, we systematically asked an expert panel of boards members, patients and their family members and healthcare professionals to determine which quality information is relevant for good governance. The aim of this study was to select a core set of parameters for boards of directors to govern quality of care in the ICU. Methods To select a core set of quality parameters, we used the modified Delphi method. This systematic, iterative methodology is used to collect and distill knowledge on a specific topic from a panel of experts. In multiple rounds, experts are confronted with each other’s ideas and viewpoints. This method has important advantages: it can be carried out via questionnaires, meaning it is both time efficient as well as anonymous, which avoids the possible negative impact of power imbalances and participant dominance [9, 10]. We performed separate Delphi studies in two Dutch academic hospitals between March and June 2016. The Delphi studies were conducted separately to allow for tailoring of the core set to the local setting and the needs and desires of each hospital’s stakeholders. Our modified Delphi study consisted of four steps, described below. Ethical approval was sought from the Research Ethics Committee of the Radboud University Nijmegen Medical Centre (registration number: 2016/2525); the committee judged that ethical approval was not required under Dutch National Law. All participants received written information about the project and its aims and were subsequently invited to participate. Step 1. Extraction of quality parameters from the available quality information The basis for the selection of the core set was all available quality information pertaining to the ICU. From January to March 2016, two researchers (AO and MZ) systematically inventoried all quality information registered in both hospitals. Quality information is generated for different information-requesting stakeholders, with different goals (quality improvement, governance and accountability), and is varied in nature (structure, outcome, or process indicators). From this inventory, two researchers (AO and MZ) extracted the quality parameters that could be used to govern quality of care in the ICU. In consultation with the head of each participating ICU, we limited the long list by removing doubles (parameters registered for separate stakeholders, with (partially) overlapping definitions) and quality parameters very likely to be considered ‘not relevant’. The resulting list of quality parameters was divided into seven domains: organization of ICU care; effectiveness of ICU treatment; incidence and prevention of complications and iatrogenic injury; learning from complications and incidents; functioning of individual healthcare professionals and teams; experiences of patients and family; patient outcomes and functional status after discharge. The domain names, quality parameter names, parameter descriptions and the domain allocation were reviewed by each ICU department head and modified if necessary. The resulting list served as the basis for the questionnaire of Delphi round 1. Step 2. Delphi round 1: relevance of quality parameters The Delphi panels consisted of 24 experts in each hospital: 6 former ICU patients and family members of former ICU patients; 6 ICU nurses; 6 intensivists; 6 managers or board members (including ICU department head, quality managers, hospital board of directors). The former patients and family members were recruited through a post-ICU care polyclinic (which sees patients 3 months (or longer) after ICU discharge for a follow-up appointment) and through the foundation for Family and patient Centered Intensive Care, meaning patients had not necessarily been admitted to the ICUs of hospital A or B. The physicians and nurses were selected on the basis of their proven interest or expertize in quality of care (for example, membership of a quality assurance committee). All experts invited agreed to participate. In the first round, the expert panel received a questionnaire with the list of quality parameters with a brief description, divided into seven domains. The participating professionals received a link to an online version of the questionnaire via e-mail. The patients and family members were sent a Word document or printed version and were guided through the process via telephone (by AO). The experts were instructed to individually rate each quality parameter on a nine-point Likert scale (ranging from ‘not at all relevant’ [1] to ‘very relevant’ [9]) by asking: ‘For each parameter, indicate how relevant you think this information would be for a board member to determine quality of care in the ICU.’ Based on the relevance scores, the parameters were divided into three categories: Non-exclusion A convincing majority of participants considered the parameter relevant: at least 70% of participants scored 7, 8 or 9 and the median was at least 8. Equivocal Extremely skewed distribution: at least 30% of participants scored 1, 2 or 3 and at least 30% of participants scored 7, 8 or 9. Or Somewhat skewed distribution: at least 70% of participants scored 7, 8 or 9 and the median was 7 or lower. Exclusion (all other cases) Parameters in the ‘non-exclusion’ or ‘equivocal’ categories were included in the questionnaire of round 2 [11, 12]. Step 3. Delphi round 2: selection of a core set of quality parameters In this second round, the experts received the questionnaire via e-mail. If so desired, patients and family members were again guided through the completion process by phone. They were presented with the remaining quality parameters divided into the same domains as in round 1, and were asked the following: ‘In the questionnaire we invite you to select 10 quality parameters from the remaining parameters. Ten quality parameters that provide a board member with sufficient means to enter into a dialog with healthcare professionals about the quality of care in the ICU.’ For each quality parameter in the questionnaire we presented the distribution of round 1 participant scores across the Likert scale, the percentage of participants that scored the parameter in the highest tertile and the median score. (For a fragment from the second round questionnaire, see the online Supplemental Material.) The 10 most frequently selected parameters in each hospital formed the two core sets. Step 4. Feasibility study To evaluate the usefulness of the identified core set of quality parameters, we performed a feasibility study in both hospitals. To this purpose, we filled each core set with data from the previous 2 years. Where relevant, we displayed trends through time or a comparison with national averages. Per parameter, we added a concise description of the most notable observations or the main problem points, with planned improvement actions and current state of affairs, where present. If possible, we displayed information in figures or tables, with the use of signal colors green, orange and red. We discussed draft versions with each ICU department head and subsequently modified the core set where needed. In both hospitals, the complete core set was used in a conversation between ICU management (department head and financial manager/nursing manager) and a member of the board of directors. A researcher (AO) observed this conversation and asked short evaluation questions afterwards. Results Step 1. Quality information inventory The inventory of quality information yielded 122 individual quality parameters in total in both hospitals. A vast majority of information (72%) was collected for quality improvement purposes. Other indicators were collected for external accountability (17%) and internal governance (11%). We can distinguish four parameter levels: structure indicators (12%, for example, the 24 h availability of a specialized ICU physician), process indicators (40%, for example, adherence to hand hygiene guidelines), outcome indicators (39%, for example, the number of complications) and patient-reported experiences (9%). For the first Delphi round, we shortened the long list of 122 parameters to a list of 54 quality parameters in hospital A and 47 quality parameters in hospital B. For the number of parameters per domain in both rounds, see Table 1. Table 1 Number of quality parameters per domain Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 aAn internal audit is a multiple-source method that evaluates whether standards and regulations are being followed. The audit involves site visits, interviews, document analysis, surveys and observations. Auditors are colleagues from a different department than the department being audited. The internal audit results in recommendations for quality improvements. Table 1 Number of quality parameters per domain Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 Domain Examples of quality parameters Number of parameters per domain in round 1 Number of parameters per domain in round 2 Hospital A Hospital B Hospital A Hospital B Organization of ICU care Nurse–patient ratio Days with full bed occupancy Recommendations and points of improvements based on internal audit resultsa 6 5 6 5 Effectiveness of ICU treatment ICU readmissions Duration of mechanical ventilation Standardized Mortality Ratio (SMR) 6 6 1 0 Incidence and prevention of complications and iatrogenic injury Number of patients with severe sepsis Compliance with delirium diagnosis guidelines Incidence of airway-related issues 21 18 12 13 Learning from complications and incidents Preventable adverse events and deaths Complaints Critical incidents 8 8 8 7 Functioning of individual care professionals and team Team Climate Inventory Annual performance reviews with specialist physicians Compliance with Crew Resource Management guidelines 3 3 2 3 Patient and family experiences Experiences of ex-ICU patients and their recommendations Experiences of family members of patients (based on questionnaire) Experiences of family members and their recommendations (based on conversations) 3 3 3 3 Patient outcomes and functional status after discharge Quality of life Fatigue Anxiety and depression 7 4 2 4 aAn internal audit is a multiple-source method that evaluates whether standards and regulations are being followed. The audit involves site visits, interviews, document analysis, surveys and observations. Auditors are colleagues from a different department than the department being audited. The internal audit results in recommendations for quality improvements. Step 2. Delphi round 1 The response rates for round 1 were 96% (23 experts) and 92% (22 experts) for hospital A and B, respectively. Based on the results, 20 (hospital A) and 12 (hospital B) parameters were excluded. The distribution of the parameters across the three categories is shown in Table 2. Table 2 Results round 1 Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Table 2 Results round 1 Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Category Hospital A (n = 54 parameters) Hospital B (n = 47 parameters) Exclusion 20 (37%) 12 (26%) Equivocal 7 (13%) 10 (21%) Non-exclusion 27 (50%) 25 (53%) Step 3. Delphi round 2 In hospital A, 22 experts (92%) participated in round 2, compared with 19 experts (79%) in hospital B. The 10 quality parameters most frequently selected by the participants from hospital A and hospital B can be found in Tables 3 and 4. Table 3 Core set selected in hospital A Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Table 3 Core set selected in hospital A Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Rank Quality parameter Frequency (n = 22) Rank in hospital B 1–3 Standardized Mortality Ratio (SMR) 17 Recommendations and points of improvements based on internal audit results 17 7–10 Preventable adverse events and deaths 17 4 Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) 15 5 Team Climate Inventory 14 4–5 6–7 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 2–3 Incident reports 13 7–10 8–9 Critical incidents reported to the Inspectorate 12 Intensivist presence and availability 12 10 Multidisciplinary discussions of complications 11 7–10 Table 4 Core set selected in hospital B Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Table 4 Core set selected in hospital B Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 Rank Quality parameter Frequency (n = 19) Rank in hospital A 1 Compliance with policy for preventing medication errors 14 2–3 Nurse–patient ratio 13 Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists 13 6–7 4–5 Days with full bed occupancy 12 Team Climate Inventory 12 5 6 Experiences of relatives of ICU patients 11 7–10 Recommendations and points of improvement based on internal audit results 10 1–3 ICU discussions of complications 10 Multidisciplinary discussions of complications 10 10 Incident reports 10 6–7 The parameters that were selected for the core sets can roughly be divided into three categories: parameters as signals for possible safety problems (for example, critical incidents and discussions of complications) parameters that show whether a department is proceeding through the plan-do-check-act (PDCA) cycle (for example, follow-up after audit results, and team climate); organizational preconditions for quality and safety of care (for example, presence and availability of an intensivist and nurse–patient ratio). Additionally, the experiences of former patients or their family members were selected in both hospitals. With regard to the preference for information regarding safety and improvement culture, there were no major differences between the four participant categories (physicians, nurses, patients and managers and board members). The core sets included mostly qualitative, aggregate information (such as the results of visitations and audits) rather than quantitative indicators (such as the percentage of infections or pressure sores, and protocol adherence rates). The 2 core sets of 10 quality parameters showed overlap: 5 of the 10 parameters were selected in both hospitals. Strikingly, the SMR most often frequently selected in hospital A is not present in hospital B’s core set. Conversely, a parameter with respect to medication error policy scores highest in hospital B, while going unselected in hospital A. (For a categorization of the core set parameters, see Table 5.) Table 5 Categorization of core set parameters Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Table 5 Categorization of core set parameters Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y Quality parameter Donabedian categorization Domain Core set hospital 1? Core set hospital 2? Recommendations and points of improvements based on internal audit results Structure Organization of ICU care Y Y Conclusions, points of advice and recommendations from the quality visitation of the professional organization of Dutch intensivists Structure Organization of ICU care Y Y Nurse–patient ratio Structure Organization of ICU care Y Days with full bed occupancy Structure Organization of ICU care Y Intensivist presence and availability Structure Organization of ICU care Y Standardized Mortality Ratio (SMR) Outcome Effectiveness of ICU treatment Y Critical incidents reported to the Inspectorate Outcome Learning from complications and incidents Y Preventable adverse events and deaths Outcome Learning from complications and incidents Y Compliance with policy for preventing medication errors Structure Learning from complications and incidents Y Incident reports Outcome Learning from complications and incidents Y Y ICU discussions of complications Outcome Learning from complications and incidents Y Multidisciplinary discussions of complications Outcome Learning from complications and incidents Y Y Team Climate Inventory Process Functioning of individual care professionals and team Y Y Experiences of post-ICU-patients (from interviews during post-ICU polyclinic visits) Outcome/ process Patient and family experiences Y Experiences of relatives of ICU patients (survey) Outcome/ process Patient and family experiences Y The raw data set is available from the authors by request. Step 4. Feasibility study Both from this observation as well as from brief evaluation with the participants afterwards, we conclude that there was a true dialog about quality of care based on the presentation of the results of the parameters included in the core set: more so than is usually the case in these meetings, as indicated by the participants. The chosen format with aggregated information about quality and safety issues, improvement actions taken, and current status regarding those actions provided an excellent basis for the conversation. After the conversation, one board member said the following: ‘What I like about this is that it leads to a true conversation and the was the intent. That you reach the essence of what it’s about. It isn’t about a checklist or a scoring list, it’s about the conversation about where are the challenges, where are the worries, where are the bottlenecks.’ Discussion This project used the Delphi method to compile two core sets of ICU quality parameters. Boards of directors are the end users of the information produced. Their role in quality improvement is different from that of healthcare professionals, and this study showed that the information boards need to govern quality of care is therefore different in nature than the information healthcare professionals need to continuously improve quality of care. In the literature on quality dashboards, however, the focus appears to be on quality information for healthcare professionals: real-time information which is continually displayed [13, 14]. The parameters selected for both core sets, however, contain few parameters that could fruitfully be used for such a dashboard. From our conversations with board members, we conclude that board members would rather receive an overview of the most important problem areas, improvement actions and current state of affairs: an overview that can serve as a basis to carry out a dialog with the head of the department about quality of care. They are interested in the story behind the numbers rather than in just the numbers themselves. This is not an isolated finding; the Netherlands has recently seen growing criticism of the dominance of numbers in different sectors [15–17]. Most parameters in the core sets are generic rather than specific to the ICU setting, such as audit results, SMR, and critical incidents. In addition to organisational preconditions and patient experiences, the core sets mostly contains information about patient safety and information that indicates to what extent departments or professionals are in improvement mode. Is a department continuously working to optimize quality of care and does it have a true improvement culture? Is a department in a PDCA cycle and if so, in which phase? This project was not aimed at producing one generic set of parameters to fit all hospitals and departments. Rather, we considered whether the Delphi method would be a useful tool in compiling a tailor-made set for a specific department or hospital. A department that uses this method would not do away with the other established quality monitoring and improvement methods, such as monitoring by the healthcare inspectorate and automatic registering of quality indicators for internal quality improvement processes. This tailor-made core set would be used as a means to instigate a dialog about quality of care between department and board of directors. When we started this project, we expected the Delphi process to yield mostly quantitative indicators that could be aggregated and visualized on a quality dashboard. One of our most striking observations, therefore, is the fact that the expert panels selected mostly qualitative parameters. Furthermore, the overlap between both core sets is noticeable: half of the parameters is the same. Future research will have to show whether this is an incidental finding or points to a trend. If this method is conducted in different departments in the same hospital, this overlap might provide the means to set several general, generic parameters and add several department-, specialty- or disease-specific parameters. The modified Delphi method to generate the core set has several important advantages. Characteristic of this method is the purposeful selection of experts, meaning a limited number of participants is sufficient to generate valuable information. The inclusion of key figures in the project group make participant recruitment relatively easy and ensured a high response rate. Representation of the most important stakeholders—both board members, healthcare professionals as well as patients and their family members—led to a broadly supported core set. Because this method makes use of quality information that is already being collected in hospitals, for internal quality improvement or external accountability, no new quality parameters have to be generated and the amount of information to be registered does not increase. In addition, it means that the core set is truly tailored to the hospital which uses it; in both generated core sets, we recognize characteristic accents of both hospitals. Many generally used quality parameters have not been scientifically evaluated: information about reliability and validity is lacking. This is also the case for the parameters in our core sets. Therefore, we cannot say with certainty whether the core sets give a reliable and valid image of the quality of care in both ICUs. As a conversation starter about quality of care, however, the core set appears to be very well-suited. Acknowledgements We would like to thank all Delphi participants: the nurses, physicians and managers, and in particular, the former patients and their family members. We would especially like to express our gratitude to the foundation of Family and patient Centered Intensive Care, in particular Ed Kuijper, for their help with participant recruitment. Funding This project took place within the program ‘Board governance’. This program is led by the Netherlands Federation of University Medical Centres consortium Quality of Care and is financed by the Citrien foundation. This foundation supports the development of sustainable and broadly applicable solutions in healthcare and is made possible by the Netherlands Organisation for Health Research and Development. References 1 Parand A , Dopson S , Renz A et al. . The role of hospital managers in quality and patient safety: a systematic review . BMJ Open 2014 ; 4 : e005055 . Google Scholar CrossRef Search ADS PubMed 2 Goeschel CA , Berenholtz SM , Culbertson RA et al. . Board quality scorecards: measuring improvement . Am J Med Qual 2011 ; 26 : 254 – 60 . Google Scholar CrossRef Search ADS PubMed 3 Bismark MM , Studdert DM . Governance of quality of care: a qualitative study of health service boards in Victoria, Australia . BMJ Qual Saf 2014 ; 23 : 474 – 82 . Google Scholar CrossRef Search ADS PubMed 4 Jha AK , Epstein AM . A survey of board chairs of English hospitals shows greater attention to quality of care than among their US counterparts . Health Aff (Millwood) 2013 ; 32 : 677 – 85 . Google Scholar CrossRef Search ADS PubMed 5 Millar R , Mannion R , Freeman T et al. . Hospital board oversight of quality and patient safety: a narrative review and synthesis of recent empirical research . Milbank Q 2013 ; 91 : 738 – 70 . Google Scholar CrossRef Search ADS PubMed 6 Blume LH , van Weert NJ , Delnoij DM . How to manage external demands in hospitals—the case of atrium MC . Healthc (Amst) 2015 ; 3 : 157 – 9 . Google Scholar CrossRef Search ADS PubMed 7 van de Bovenkamp HM , de Mul M , Quartz JGU et al. . Institutional layering in governing healthcare quality . Public Administration 2014 ; 92 : 208 – 23 . Google Scholar CrossRef Search ADS 8 Conway PH , Mostashari F , Clancy C . The future of quality measurement for improvement and accountability . JAMA 2013 ; 309 : 2215 – 6 . Google Scholar CrossRef Search ADS PubMed 9 Boulkedid R , Abdoul H , Loustau M et al. . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review . PLoS One 2011 ; 6 : e20476 . Google Scholar CrossRef Search ADS PubMed 10 Hermens RP , Ouwens MM , Vonk-Okhuijsen SY et al. . Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline . Lung Cancer 2006 ; 54 : 117 – 24 . Google Scholar CrossRef Search ADS PubMed 11 Campbell SM , Braspenning J , Hutchinson A et al. . Research methods used in developing and applying quality indicators in primary care . BMJ 2003 ; 326 : 816 – 9 . Google Scholar CrossRef Search ADS PubMed 12 Stienen JJ , Tabbers MM , Benninga MA et al. . Development of quality indicators based on a multidisciplinary, evidence-based guideline on pediatric constipation . Eur J Pediatr 2011 ; 170 : 1513 – 9 . Google Scholar CrossRef Search ADS PubMed 13 Eckerson WW . Performance Dashboards: Measuring, Monitoring, and Managing Your Business . Hoboken, New Jersey : John Wiley & Sons , 2010 . 14 Kroch E , Vaughn T , Koepke M et al. . Hospital boards and quality dashboards . J Patient Saf 2006 ; 2 : 10 – 9 . 15 Steenhuis PH Meten is weten, maar is dat wel zo? Trouw. 2017 February 13, 2017. 16 Brinkgreve C , Bloemink S , Koenen E . Weten vraagt meer dan meten . Amsterdam : Amsterdam University Press , 2017 . 17 NVTZ . Zienderogen beter II. Veiligheid in perspectief. 2016 . © The Author(s) 2018. Published by Oxford University Press in association with the International Society for Quality in Health Care. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

Journal

International Journal for Quality in Health CareOxford University Press

Published: Apr 4, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off