A project to assess the quality of the published guidelines for managing primary spontaneous pneumothorax from the Italian Society of Thoracic Surgeons

A project to assess the quality of the published guidelines for managing primary spontaneous... Abstract OBJECTIVES A project to assess the existing literature and the quality of past guidelines on the management of primary spontaneous pneumothorax was developed by the Italian Society of Thoracic Surgeons, with particular focus on the assessment of the methods used to produce such recommendations. METHODS The different items and domains within each guideline were assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument and scored on a 7-point scale. RESULTS Five guidelines matched the inclusion criteria and were assessed. A multinational collaboration produced 2 of 5 guidelines. The observers recommended (with modifications) only 2 guidelines. Clarity of presentation, scope and purpose (objectives and health questions target the population) received the best score, whereas the applicability of the guideline received the lowest score. International development positively influenced the scope and purpose of the guidelines. Moreover, improved scores were achieved when the stakeholders were fully involved and had editorial independence. CONCLUSIONS As assessed by the AGREE II criteria, the quality of the various guidelines was extremely inconsistent. Guidelines with higher AGREE II scores were those developed with the participation of European scientific societies. Primary spontaneous pneumothorax, Benchmarking project, Quality, Appraisal of Guidelines for Research and Evaluation (AGREE) II INTRODUCTION Spontaneous pneumothorax is a clinical condition known since antiquity, but it was only identified as a proper disease in 1934 when Vrooman [1] first described a ‘spontaneous pneumothorax in the apparently healthy adult’. In this condition, intrapleural pressure is negative but, despite that, gases do not enter the pleural space under physiological conditions because a negative pressure exceeding 54 mmHg is required to move gases from the capillary vessels into the pleural space [2]. The presence of air in the pleural space, therefore, only occurs in pathological conditions, such as the presence of gas-producing bacteria, communication between the pleural space and the outside environment and communication with the bronchial tree or alveolar space. Primary spontaneous pneumothorax (PSP) is defined as a parenchymal air leak into the pleural space in the absence of any iatrogenic cause (diagnostic manoeuvres, thoracic trauma and recent thoracic surgery). Secondary spontaneous pneumothorax is defined as an air leak in the presence of contributing conditions (emphysema and cystic fibrosis). PSP classically affects young, tall, thin males who smoke, but, contrary to a common belief, PSP typically occurs at rest. Given the annual incidence of 18–28 per 100 000 in men and 1.2–6 per 100 000 in women [3], PSP is considered not only a clinical problem but also a social one. Despite its high incidence, the optimal management of this condition is still under debate. Diagnostic and therapeutic guidelines do exist, but the significant variations in clinical practice observed worldwide [4] make it difficult for clinicians to feel confident about following the recommendations in those guidelines [5]. Consequently, the Italian Society for Thoracic Surgery, with the endorsement of the Italian Ministry of Health, planned the development of better quality national guidelines for the diagnosis and treatment of spontaneous pneumothorax. The Society began by appraising the currently available guidelines before making its recommendations. The goal of this paper, therefore, was to assess the existing literature and the quality of past guidelines on the management of primary spontaneous pneumothorax, with a particular focus on the assessment of the methods used to produce these recommendations and on the definitions of variables that have influenced the quality of the studies. MATERIALS AND METHODS A narrative literature search was carried out by a qualified health professional librarian. As described by Bertolaccini et al. [6], Internet search engines and databases of guidelines were selected as appropriate. A combination of keywords and subject headings charted to the thesaurus ensured a thorough search. Other sources, such as Google search results, the National Guideline Clearinghouse and the Guidelines International Network were explored. The guidelines were first deduplicated using the Endnote Web and the Healthcare Database Search tool and then reviewed at the abstract level to ensure relevancy. Irrelevant results and residual duplicates were manually removed. Finally, the last edition of each guideline was chosen for review by the authors of this article [6]. The authors also included only those national guidelines that were available in a peer-reviewed journal; nevertheless, we did not find any other national guidelines that were not published in peer-reviewed journals. The Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument [7, 8] was used to assess each guideline. Parts of the methods regarding the use of the AGREE II instrument have been described previously [6]. The quality of guidelines was defined by the AGREE Collaboration as the adequate evaluation of the potential biases in the development of guidelines. AGREE also evaluated the validation of recommendations (internally and externally) and the feasibility of practice [9]. AGREE II provides the methodological steps for the development of guidelines and supervision on the content and strategy for the best reporting. Four observers (authors A.B., M.T.C., M.N. and P.S.) independently evaluated the guidelines. AGREE II comprises 23 critical items grouped within 6 domains and in 2 overall rating items (Table 1). The authors used a 7-point scale to score each item (1 = strongly disagree, 7 = strongly agree). The score for each scaled domain was calculated as the ratio of the obtained score minus the minimum possible score on the differential of the possible obtained scores [6, 7]. As suggested in the AGREE II manual, a score (expressed as a percentage) was calculated for each domain based on the scores for the specific items of the 4 observers [7]: For the corresponding AGREE II domain, the higher the score, the better the methodological quality of the guideline [9]. The results were summarized with values for each domain [10]. Additionally, the following data were recorded: the country, the year of publication, the language, the affiliated scientific society, publication in a peer-reviewed journal and use at the local or international level. Table 1: The domains evaluated using the AGREE II instrument Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Each item has a 7-point scale [6, 7]. AGREE II: Appraisal of Guidelines for Research and Evaluation II. Table 1: The domains evaluated using the AGREE II instrument Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Each item has a 7-point scale [6, 7]. AGREE II: Appraisal of Guidelines for Research and Evaluation II. Statistical analyses As described by Bertolaccini et al. [6], the analyses of variance for categorical variables were used to assess the effect of guideline characteristics on the AGREE II scores. Univariable analyses identified variables (P < 0.30) for the multivariable analyses. Multivariable analysis using logistic regression was performed to control the effect of confounding variables and to identify the independent predictors of the factors influencing the domain scores. As all analyses were exploratory, there was no adjustment for multiplicity. A P-value <0.05 was defined as significant. Statistical analyses were made using R (version 3.4.2, Short Summer with standard, rcmdr and irr packages) [11]. RESULTS A total of 29 guidelines were found; the 5 guidelines that most closely fit the inclusion criteria were assessed by 4 observers (Table 2, Fig. 1) [3, 12–15]. The publication year ranged from 2011 to 2015; 2 (40%) guidelines were updated between 2010 and 2015. A multinational collaboration produced 2 of the 5 (40%) guidelines. All guidelines were published in peer-reviewed journals. The 4 panellists would only recommend, with minimal modifications, 2 of the guidelines. All 4 observers rated all the AGREE II domains without data missing. Table 2: Guidelines included in the AGREE II evaluation with information on language and sources of retrieval Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 2: Guidelines included in the AGREE II evaluation with information on language and sources of retrieval Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Figure 1: View largeDownload slide The flow of information of the project to assess the quality of the published guidelines for the management of primary spontaneous pneumothorax. Figure 1: View largeDownload slide The flow of information of the project to assess the quality of the published guidelines for the management of primary spontaneous pneumothorax. Table 3 shows the scores of the 6 domains assigned to the selected guidelines. The Belgian guideline received the lowest score because of lack of applicability, whereas the British Thoracic Society guideline received the highest score because of its editorial independence. Table 3: AGREE II scores (%) by different domains of the analysed guidelines Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 3: AGREE II scores (%) by different domains of the analysed guidelines Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 4 shows the descriptive analyses of the 6 domain scores and the overall assessment. The best scores were received in 2 areas: D1 (scope and purpose) and D4 (clarity of presentation). In contrast, the D5 (applicability of the guideline) domain received the lowest score. Table 4: AGREE II scores by different domains of the analysed guidelines Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 AGREE II: Appraisal of Guidelines for Research and Evaluation II; CI: confidence interval. Table 4: AGREE II scores by different domains of the analysed guidelines Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 AGREE II: Appraisal of Guidelines for Research and Evaluation II; CI: confidence interval. Table 5 shows the results of the univariable analyses of variance for the categorical variables. The international level of the guidelines influenced the scores for 2 domains: D2 (stakeholder involvement) and D3 (rigour of development). Internationally developed guidelines also had a practical effect on the D1 (scope and purpose) domain. Lastly, guidelines were associated with a more significant and more consistent involvement of stakeholders. The multivariable analysis shows that scores that were improved (Table 6) because of stakeholders’ involvement and editorial independence were related to the internationally developed guidelines published in the last 7 years in scientific journals. Based on the results of the AGREE II appraisal, the British Thoracic Society and the European Respiratory Society guidelines could be recommended for current clinical practice. Table 5: Analysis of variance for the 6 domains in the AGREE II Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Data are presented as n (%) or mean. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 5: Analysis of variance for the 6 domains in the AGREE II Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Data are presented as n (%) or mean. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 6: Results of multivariable analysis AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 Data are presented as P-values. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 6: Results of multivariable analysis AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 Data are presented as P-values. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. DISCUSSION Patient preferences, surgeons’ perspectives and local resources determine significant variations in the clinical management of PSP. Clinical guidelines with clear structure and presentation might help to reduce the discrepancy in clinical management and potentially improve the quality of care. In this context, it would be helpful to assess the strengths and weaknesses of the current guidelines to develop further national guidelines, supported by the Italian Ministry of Health, to address existing shortcomings. This article reports a qualitative assessment of the guidelines on PSP diagnosis and treatment. Out of the more than 20 guidelines identified, 5 were evaluated using the AGREE II standardized assessment criteria by 4 thoracic surgeons. Significant limitations on the management of PSP were noted in the current literature, such as pooling of PSP and secondary spontaneous pneumothorax patients, interventions that are not standardized, lack of information on the clinical course, lack of risk stratification on the severity of underlying lung disease, absence of health-related quality-of-life outcomes and relative cost-effectiveness of approaches to care [12]. Although we recognize that there are numerous accepted methods for assessing quality improvement in health care, we chose the AGREE II instrument because it offers guideline developers a rigorous methodology [8]. Due to the full range of domain items, the AGREE II instrument offers the opportunity to systematically, accurately and objectively evaluate the quality of guidelines from all specialities. It should also be noted that these assessments are subjective, and the items or domains are not weighted but all are considered equally. The results of an AGREE II appraisal should, therefore, be viewed with caution because one may interpret the items and scoring system differently from one guideline to another. Also, guideline quality can be extremely variable; many guidelines fall short of basic guideline standards. Therefore, the higher the quality of the guideline, the more likely are the benefits of following its recommendations. This review of the principles of the management of PSP is based on currently available data and the most wide-ranging systematic analysis. The guidelines assessed differed in content and goals but had similarities in subject and structure. The methodology of guideline development was extremely variable across all AGREE II domains. In general, the quality associated with PSP guidelines was relatively low: the domains D1 (scope and purpose) and D4 (clarity of presentation) received higher scores, but D5 (applicability of the guideline) received the worst score. Multivariable analyses showed that international guidelines available in peer-reviewed journals earned higher AGREE II scores. We, therefore, advocate using the peer-review process as an additional tool to improve guideline strategy. Current guidelines are composed of moderate- or low-quality evidence recommendations [16]. Limitations We acknowledge the limitations of our article. A potential weakness is that only 1 individual (even if a qualified librarian) searched for articles. Nevertheless, the search was completed according to available standards to ensure the identification of the appropriate articles for the review. The main limitation of the narrative literature review was the exclusion of studies published outside peer-reviewed journals, with the risk of missing potential recommendations not usually indexed (e.g. documents from governmental and other authoritative bodies). Also, the exclusion of national guidelines not published in peer-reviewed journals possibly favoured the international guidelines. The main criticism of the national guidelines is that the data may not be genuinely comparable across different nations due to a diverse understanding of the clinical problems or dependence on only a few centres to submit datasets [9]. On the other hand, although we had 4 reviewers (all with content-specific knowledge) to assess the guidelines as suggested by the AGREE II procedure, the use of this tool needs careful interpretation because all the items are subjective ratings. The differences in significant aspects of clinical guidelines could be evaluated using the AGREE II instrument, and it could be used by an extensive range of researchers from diverse scientific areas. The appraisers found the AGREE II easy to use and perceived it as useful for comparing the quality of the guidelines. The methodological rigour and transparency of the guideline’s development could also be assessed using the AGREE II instrument. Although it is a subjective instrument, it is the gold standard [7]. However, there is no way to discern high- from low-quality criteria, so a domain with a low score may not consistently indicate low quality. Consequently, the overall scores should be interpreted with care and in the right context [6]. Guidelines correctly prepared and reported are vital to clinical decision making. Although the scope and purpose of current PSP guidelines were sufficiently reported, their overall quality was judged unsatisfactory. The authors are confident that, starting with a systematic review and following a strict methodology, it will be possible to develop national guidelines that will indeed guide the clinical practice of Italian thoracic surgeons. Several questions require further research, such as the exact incidence of PSP, the subgroups of patients with a substantial risk of recurrence of PSP who warrant definitive treatment at their first presentation, the role for genetic testing and thoracic high-resolution computed tomography scanning [15]. CONCLUSIONS Assessed by the AGREE II criteria, the quality of the various published guidelines on PSP varied, with moderately low average scores. The guidelines with higher AGREE II scores were developed with direct involvement of the British Thoracic Society and the European Respiratory Society. In view of the results of the AGREE II appraisal, we recommend that the British Thoracic Society and the European Respiratory Society guidelines be used in current clinical practice. Conflict of interest: none declared. REFERENCES 1 Vrooman CH. Idiopathic spontaneous pneumothorax in apparently healthy adults . Can Med Assoc J 1934 ; 30 : 265 – 8 . Google Scholar PubMed 2 Noppen M , Schramel F. Pneumothorax . Eur Respir Mon 2002 ; 22 : 279 – 96 . 3 MacDuff A , Arnold A , Harvey J. Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 . Thorax 2010 ; 65 : ii18 – 31 . Google Scholar CrossRef Search ADS PubMed 4 Grundy S , Bentley A , Tschopp JM. Primary spontaneous pneumothorax: a diffuse disease of the pleura . Respiration 2012 ; 83 : 185 – 9 . Google Scholar CrossRef Search ADS PubMed 5 Kelly AM , Clooney M. Deviation from published guidelines in the management of primary spontaneous pneumothorax in Australia . Intern Med J 2008 ; 38 : 64 – 7 . Google Scholar CrossRef Search ADS PubMed 6 Bertolaccini L , Bedetti B , Brunelli A , Marinova K , Raveglia F , Rocco G et al. Members of the ESTS Pleural Disease Working Group . A benchmarking project on the quality of previous guidelines about the management of malignant pleural effusion from the European Society of Thoracic Surgeons (ESTS) Pleural Diseases Working Group . Eur J Cardiothorac Surg 2017 ; 52 : 356 – 62 . Google Scholar CrossRef Search ADS PubMed 7 AGREE Collaboration . Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project . Qual Saf Health Care 2003 ; 12 : 18 – 23 . CrossRef Search ADS PubMed 8 AGREE Next Steps Consortium . The AGREE II Instrument [Electronic version]. 2009 . http://www.agreetrust.org (1 April 2018, date last accessed). 9 Blum TG , Rich A , Baldwin D , Beckett P , De Ruysscher D , Faivre-Finn C et al. The European initiative for quality management in lung cancer care . Eur Respir J 2014 ; 43 : 1254 – 77 . Google Scholar CrossRef Search ADS PubMed 10 Brouwers MC , Kho ME , Browman GP , Burgers J , Cluzeau F , Feder G et al. AGREE II: advancing guideline development, reporting and evaluation in healthcare . Can Med Assoc J 2010 ; 182 : 1045 – 2 . Google Scholar CrossRef Search ADS 11 R Core Team . R: A Language and Environment for Statistical Computing . Vienna, Austria : R Foundation for Statistical Computing , 2018 . www.R-project.org. 12 Baumann MH , Strange C , Heffner JE , Light R , Kirby TJ , Klein J et al. Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement . Chest 2001 ; 119 : 590 – 602 . Google Scholar CrossRef Search ADS PubMed 13 De Leyn P , Lismonde M , Ninane V , Noppen M , Slabbynck H , Van Meerhaeghe A et al. Guidelines Belgian Society of Pneumology. Guidelines on the management of spontaneous pneumothorax . Acta Chir Belg 2005 ; 105 : 265 – 7 . Google Scholar CrossRef Search ADS PubMed 14 Rivas de Andres JJ , Jimenez Lopez MF , Molins Lopez-Rodo L , Perez Trullen A , Torres Lanzas J. [ Guidelines for the diagnosis and treatment of spontaneous pneumothorax] . Arch Broncopneumol 2008 ; 44 : 437 – 48 . 15 Tschopp JM , Bintcliffe O , Astoul P , Canalis E , Driesen P , Janssen J et al. ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax . Eur Respir J 2015 ; 46 : 321 – 35 . Google Scholar CrossRef Search ADS PubMed 16 Guyatt GH , Oxman AD , Vist GE , Kunz R , Falck-Ytter Y , Alonso-Coello P et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations . BMJ 2008 ; 336 : 924 – 6 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png European Journal of Cardio-Thoracic Surgery Oxford University Press

A project to assess the quality of the published guidelines for managing primary spontaneous pneumothorax from the Italian Society of Thoracic Surgeons

Loading next page...
 
/lp/ou_press/a-project-to-assess-the-quality-of-the-published-guidelines-for-s0zUF0Pe9J
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved.
ISSN
1010-7940
eISSN
1873-734X
D.O.I.
10.1093/ejcts/ezy199
Publisher site
See Article on Publisher Site

Abstract

Abstract OBJECTIVES A project to assess the existing literature and the quality of past guidelines on the management of primary spontaneous pneumothorax was developed by the Italian Society of Thoracic Surgeons, with particular focus on the assessment of the methods used to produce such recommendations. METHODS The different items and domains within each guideline were assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument and scored on a 7-point scale. RESULTS Five guidelines matched the inclusion criteria and were assessed. A multinational collaboration produced 2 of 5 guidelines. The observers recommended (with modifications) only 2 guidelines. Clarity of presentation, scope and purpose (objectives and health questions target the population) received the best score, whereas the applicability of the guideline received the lowest score. International development positively influenced the scope and purpose of the guidelines. Moreover, improved scores were achieved when the stakeholders were fully involved and had editorial independence. CONCLUSIONS As assessed by the AGREE II criteria, the quality of the various guidelines was extremely inconsistent. Guidelines with higher AGREE II scores were those developed with the participation of European scientific societies. Primary spontaneous pneumothorax, Benchmarking project, Quality, Appraisal of Guidelines for Research and Evaluation (AGREE) II INTRODUCTION Spontaneous pneumothorax is a clinical condition known since antiquity, but it was only identified as a proper disease in 1934 when Vrooman [1] first described a ‘spontaneous pneumothorax in the apparently healthy adult’. In this condition, intrapleural pressure is negative but, despite that, gases do not enter the pleural space under physiological conditions because a negative pressure exceeding 54 mmHg is required to move gases from the capillary vessels into the pleural space [2]. The presence of air in the pleural space, therefore, only occurs in pathological conditions, such as the presence of gas-producing bacteria, communication between the pleural space and the outside environment and communication with the bronchial tree or alveolar space. Primary spontaneous pneumothorax (PSP) is defined as a parenchymal air leak into the pleural space in the absence of any iatrogenic cause (diagnostic manoeuvres, thoracic trauma and recent thoracic surgery). Secondary spontaneous pneumothorax is defined as an air leak in the presence of contributing conditions (emphysema and cystic fibrosis). PSP classically affects young, tall, thin males who smoke, but, contrary to a common belief, PSP typically occurs at rest. Given the annual incidence of 18–28 per 100 000 in men and 1.2–6 per 100 000 in women [3], PSP is considered not only a clinical problem but also a social one. Despite its high incidence, the optimal management of this condition is still under debate. Diagnostic and therapeutic guidelines do exist, but the significant variations in clinical practice observed worldwide [4] make it difficult for clinicians to feel confident about following the recommendations in those guidelines [5]. Consequently, the Italian Society for Thoracic Surgery, with the endorsement of the Italian Ministry of Health, planned the development of better quality national guidelines for the diagnosis and treatment of spontaneous pneumothorax. The Society began by appraising the currently available guidelines before making its recommendations. The goal of this paper, therefore, was to assess the existing literature and the quality of past guidelines on the management of primary spontaneous pneumothorax, with a particular focus on the assessment of the methods used to produce these recommendations and on the definitions of variables that have influenced the quality of the studies. MATERIALS AND METHODS A narrative literature search was carried out by a qualified health professional librarian. As described by Bertolaccini et al. [6], Internet search engines and databases of guidelines were selected as appropriate. A combination of keywords and subject headings charted to the thesaurus ensured a thorough search. Other sources, such as Google search results, the National Guideline Clearinghouse and the Guidelines International Network were explored. The guidelines were first deduplicated using the Endnote Web and the Healthcare Database Search tool and then reviewed at the abstract level to ensure relevancy. Irrelevant results and residual duplicates were manually removed. Finally, the last edition of each guideline was chosen for review by the authors of this article [6]. The authors also included only those national guidelines that were available in a peer-reviewed journal; nevertheless, we did not find any other national guidelines that were not published in peer-reviewed journals. The Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument [7, 8] was used to assess each guideline. Parts of the methods regarding the use of the AGREE II instrument have been described previously [6]. The quality of guidelines was defined by the AGREE Collaboration as the adequate evaluation of the potential biases in the development of guidelines. AGREE also evaluated the validation of recommendations (internally and externally) and the feasibility of practice [9]. AGREE II provides the methodological steps for the development of guidelines and supervision on the content and strategy for the best reporting. Four observers (authors A.B., M.T.C., M.N. and P.S.) independently evaluated the guidelines. AGREE II comprises 23 critical items grouped within 6 domains and in 2 overall rating items (Table 1). The authors used a 7-point scale to score each item (1 = strongly disagree, 7 = strongly agree). The score for each scaled domain was calculated as the ratio of the obtained score minus the minimum possible score on the differential of the possible obtained scores [6, 7]. As suggested in the AGREE II manual, a score (expressed as a percentage) was calculated for each domain based on the scores for the specific items of the 4 observers [7]: For the corresponding AGREE II domain, the higher the score, the better the methodological quality of the guideline [9]. The results were summarized with values for each domain [10]. Additionally, the following data were recorded: the country, the year of publication, the language, the affiliated scientific society, publication in a peer-reviewed journal and use at the local or international level. Table 1: The domains evaluated using the AGREE II instrument Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Each item has a 7-point scale [6, 7]. AGREE II: Appraisal of Guidelines for Research and Evaluation II. Table 1: The domains evaluated using the AGREE II instrument Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Domain Name Number of items Description D1 Scope and purpose 3 The overall aim of the guidelines, the specific health questions and the target population D2 Stakeholder involvement 3 The extent of development for the guideline related to the appropriate interested parties, the views of its intended users D3 Rigour of development 8 The process used to synthesize the evidence, the methods used to formulate and to update the recommendations D4 Clarity of presentation 3 Language, structure and format of the guidelines D5 Applicability 4 Possible barriers to implementation, strategies to improve uptake and resource implications of applying the guidelines D6 Editorial independence 2 Formulation of recommendations not biased with competing interests Overall Overall assessment 2 Overall quality and recommendation for the use of the guideline Each item has a 7-point scale [6, 7]. AGREE II: Appraisal of Guidelines for Research and Evaluation II. Statistical analyses As described by Bertolaccini et al. [6], the analyses of variance for categorical variables were used to assess the effect of guideline characteristics on the AGREE II scores. Univariable analyses identified variables (P < 0.30) for the multivariable analyses. Multivariable analysis using logistic regression was performed to control the effect of confounding variables and to identify the independent predictors of the factors influencing the domain scores. As all analyses were exploratory, there was no adjustment for multiplicity. A P-value <0.05 was defined as significant. Statistical analyses were made using R (version 3.4.2, Short Summer with standard, rcmdr and irr packages) [11]. RESULTS A total of 29 guidelines were found; the 5 guidelines that most closely fit the inclusion criteria were assessed by 4 observers (Table 2, Fig. 1) [3, 12–15]. The publication year ranged from 2011 to 2015; 2 (40%) guidelines were updated between 2010 and 2015. A multinational collaboration produced 2 of the 5 (40%) guidelines. All guidelines were published in peer-reviewed journals. The 4 panellists would only recommend, with minimal modifications, 2 of the guidelines. All 4 observers rated all the AGREE II domains without data missing. Table 2: Guidelines included in the AGREE II evaluation with information on language and sources of retrieval Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 2: Guidelines included in the AGREE II evaluation with information on language and sources of retrieval Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] Issuing society Title of guideline Year Country Language Reference American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 2001 USA English [12] Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 2005 Belgium English [13] Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 2008 Spain Spanish [14] British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 2010 UK English [3] European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 2015 European Union English [15] AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Figure 1: View largeDownload slide The flow of information of the project to assess the quality of the published guidelines for the management of primary spontaneous pneumothorax. Figure 1: View largeDownload slide The flow of information of the project to assess the quality of the published guidelines for the management of primary spontaneous pneumothorax. Table 3 shows the scores of the 6 domains assigned to the selected guidelines. The Belgian guideline received the lowest score because of lack of applicability, whereas the British Thoracic Society guideline received the highest score because of its editorial independence. Table 3: AGREE II scores (%) by different domains of the analysed guidelines Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 3: AGREE II scores (%) by different domains of the analysed guidelines Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 Issuing society Title of guideline D1—scope and purpose D2—stakeholder involvement D3—rigour of development D4—clarity of presentation D5—applicability D6—editorial independence Overall assessment American College of Chest Physicians Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement 68 57 69 65 47 48 38 Belgian Society of Pneumology Guidelines on the management of spontaneous pneumothorax 43 33 24 51 14 29 38 Spanish Society of Pulmonology and Thoracic Surgery Guidelines for the diagnosis and treatment of spontaneous pneumothorax 56 32 36 61 33 30 35 British Thoracic Society Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 58 63 66 58 50 83 65 European Respiratory Society ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax 60 65 67 63 48 68 58 AGREE II: Appraisal of Guidelines for Research and Evaluation II; ERS: European Respiratory Society. Table 4 shows the descriptive analyses of the 6 domain scores and the overall assessment. The best scores were received in 2 areas: D1 (scope and purpose) and D4 (clarity of presentation). In contrast, the D5 (applicability of the guideline) domain received the lowest score. Table 4: AGREE II scores by different domains of the analysed guidelines Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 AGREE II: Appraisal of Guidelines for Research and Evaluation II; CI: confidence interval. Table 4: AGREE II scores by different domains of the analysed guidelines Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 Domain Median (%) 95% CI for the mean (%) Minimum (%) Maximum (%) D1—scope and purpose 58 46–68 43 68 D2—stakeholder involvement 57 30–70 32 65 D3—rigour of development 66 26–78 24 69 D4—clarity of presentation 61 53–66 51 65 D5—applicability 47 20–57 14 50 D6—editorial independence 48 22–81 29 83 Overall assessment 38 30–64 35 65 AGREE II: Appraisal of Guidelines for Research and Evaluation II; CI: confidence interval. Table 5 shows the results of the univariable analyses of variance for the categorical variables. The international level of the guidelines influenced the scores for 2 domains: D2 (stakeholder involvement) and D3 (rigour of development). Internationally developed guidelines also had a practical effect on the D1 (scope and purpose) domain. Lastly, guidelines were associated with a more significant and more consistent involvement of stakeholders. The multivariable analysis shows that scores that were improved (Table 6) because of stakeholders’ involvement and editorial independence were related to the internationally developed guidelines published in the last 7 years in scientific journals. Based on the results of the AGREE II appraisal, the British Thoracic Society and the European Respiratory Society guidelines could be recommended for current clinical practice. Table 5: Analysis of variance for the 6 domains in the AGREE II Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Data are presented as n (%) or mean. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 5: Analysis of variance for the 6 domains in the AGREE II Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Factor Evaluated guidelines AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines  International 2 (40) 64 61 68 57 48 47  National 3 (60) 52 43 43 64 32 58  P-value 0.18 0.27 0.21 0.16 0.34 0.27 Publication year  Before 2009 3 (60) 56 41 43 59 31 36  2010–2015 2 (40) 59 64 67 61 49 76  P-value 0.75 0.12 0.27 0.81 0.25 0.026 Data are presented as n (%) or mean. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 6: Results of multivariable analysis AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 Data are presented as P-values. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. Table 6: Results of multivariable analysis AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 AGREE II domains D1 D2 D3 D4 D5 D6 Level of the guidelines 0.41 0.029 0.21 0.47 0.027 Publication year 0.019 0.24 0.50 0.011 Data are presented as P-values. AGREE II: Appraisal of Guidelines for Research and Evaluation II; D1: scope and purpose; D2: stakeholder involvement; D3: rigour of development; D4: clarity of presentation; D5: applicability; D6: editorial independence. DISCUSSION Patient preferences, surgeons’ perspectives and local resources determine significant variations in the clinical management of PSP. Clinical guidelines with clear structure and presentation might help to reduce the discrepancy in clinical management and potentially improve the quality of care. In this context, it would be helpful to assess the strengths and weaknesses of the current guidelines to develop further national guidelines, supported by the Italian Ministry of Health, to address existing shortcomings. This article reports a qualitative assessment of the guidelines on PSP diagnosis and treatment. Out of the more than 20 guidelines identified, 5 were evaluated using the AGREE II standardized assessment criteria by 4 thoracic surgeons. Significant limitations on the management of PSP were noted in the current literature, such as pooling of PSP and secondary spontaneous pneumothorax patients, interventions that are not standardized, lack of information on the clinical course, lack of risk stratification on the severity of underlying lung disease, absence of health-related quality-of-life outcomes and relative cost-effectiveness of approaches to care [12]. Although we recognize that there are numerous accepted methods for assessing quality improvement in health care, we chose the AGREE II instrument because it offers guideline developers a rigorous methodology [8]. Due to the full range of domain items, the AGREE II instrument offers the opportunity to systematically, accurately and objectively evaluate the quality of guidelines from all specialities. It should also be noted that these assessments are subjective, and the items or domains are not weighted but all are considered equally. The results of an AGREE II appraisal should, therefore, be viewed with caution because one may interpret the items and scoring system differently from one guideline to another. Also, guideline quality can be extremely variable; many guidelines fall short of basic guideline standards. Therefore, the higher the quality of the guideline, the more likely are the benefits of following its recommendations. This review of the principles of the management of PSP is based on currently available data and the most wide-ranging systematic analysis. The guidelines assessed differed in content and goals but had similarities in subject and structure. The methodology of guideline development was extremely variable across all AGREE II domains. In general, the quality associated with PSP guidelines was relatively low: the domains D1 (scope and purpose) and D4 (clarity of presentation) received higher scores, but D5 (applicability of the guideline) received the worst score. Multivariable analyses showed that international guidelines available in peer-reviewed journals earned higher AGREE II scores. We, therefore, advocate using the peer-review process as an additional tool to improve guideline strategy. Current guidelines are composed of moderate- or low-quality evidence recommendations [16]. Limitations We acknowledge the limitations of our article. A potential weakness is that only 1 individual (even if a qualified librarian) searched for articles. Nevertheless, the search was completed according to available standards to ensure the identification of the appropriate articles for the review. The main limitation of the narrative literature review was the exclusion of studies published outside peer-reviewed journals, with the risk of missing potential recommendations not usually indexed (e.g. documents from governmental and other authoritative bodies). Also, the exclusion of national guidelines not published in peer-reviewed journals possibly favoured the international guidelines. The main criticism of the national guidelines is that the data may not be genuinely comparable across different nations due to a diverse understanding of the clinical problems or dependence on only a few centres to submit datasets [9]. On the other hand, although we had 4 reviewers (all with content-specific knowledge) to assess the guidelines as suggested by the AGREE II procedure, the use of this tool needs careful interpretation because all the items are subjective ratings. The differences in significant aspects of clinical guidelines could be evaluated using the AGREE II instrument, and it could be used by an extensive range of researchers from diverse scientific areas. The appraisers found the AGREE II easy to use and perceived it as useful for comparing the quality of the guidelines. The methodological rigour and transparency of the guideline’s development could also be assessed using the AGREE II instrument. Although it is a subjective instrument, it is the gold standard [7]. However, there is no way to discern high- from low-quality criteria, so a domain with a low score may not consistently indicate low quality. Consequently, the overall scores should be interpreted with care and in the right context [6]. Guidelines correctly prepared and reported are vital to clinical decision making. Although the scope and purpose of current PSP guidelines were sufficiently reported, their overall quality was judged unsatisfactory. The authors are confident that, starting with a systematic review and following a strict methodology, it will be possible to develop national guidelines that will indeed guide the clinical practice of Italian thoracic surgeons. Several questions require further research, such as the exact incidence of PSP, the subgroups of patients with a substantial risk of recurrence of PSP who warrant definitive treatment at their first presentation, the role for genetic testing and thoracic high-resolution computed tomography scanning [15]. CONCLUSIONS Assessed by the AGREE II criteria, the quality of the various published guidelines on PSP varied, with moderately low average scores. The guidelines with higher AGREE II scores were developed with direct involvement of the British Thoracic Society and the European Respiratory Society. In view of the results of the AGREE II appraisal, we recommend that the British Thoracic Society and the European Respiratory Society guidelines be used in current clinical practice. Conflict of interest: none declared. REFERENCES 1 Vrooman CH. Idiopathic spontaneous pneumothorax in apparently healthy adults . Can Med Assoc J 1934 ; 30 : 265 – 8 . Google Scholar PubMed 2 Noppen M , Schramel F. Pneumothorax . Eur Respir Mon 2002 ; 22 : 279 – 96 . 3 MacDuff A , Arnold A , Harvey J. Management of spontaneous pneumothorax: British Thoracic Society Pleural Disease Guideline 2010 . Thorax 2010 ; 65 : ii18 – 31 . Google Scholar CrossRef Search ADS PubMed 4 Grundy S , Bentley A , Tschopp JM. Primary spontaneous pneumothorax: a diffuse disease of the pleura . Respiration 2012 ; 83 : 185 – 9 . Google Scholar CrossRef Search ADS PubMed 5 Kelly AM , Clooney M. Deviation from published guidelines in the management of primary spontaneous pneumothorax in Australia . Intern Med J 2008 ; 38 : 64 – 7 . Google Scholar CrossRef Search ADS PubMed 6 Bertolaccini L , Bedetti B , Brunelli A , Marinova K , Raveglia F , Rocco G et al. Members of the ESTS Pleural Disease Working Group . A benchmarking project on the quality of previous guidelines about the management of malignant pleural effusion from the European Society of Thoracic Surgeons (ESTS) Pleural Diseases Working Group . Eur J Cardiothorac Surg 2017 ; 52 : 356 – 62 . Google Scholar CrossRef Search ADS PubMed 7 AGREE Collaboration . Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project . Qual Saf Health Care 2003 ; 12 : 18 – 23 . CrossRef Search ADS PubMed 8 AGREE Next Steps Consortium . The AGREE II Instrument [Electronic version]. 2009 . http://www.agreetrust.org (1 April 2018, date last accessed). 9 Blum TG , Rich A , Baldwin D , Beckett P , De Ruysscher D , Faivre-Finn C et al. The European initiative for quality management in lung cancer care . Eur Respir J 2014 ; 43 : 1254 – 77 . Google Scholar CrossRef Search ADS PubMed 10 Brouwers MC , Kho ME , Browman GP , Burgers J , Cluzeau F , Feder G et al. AGREE II: advancing guideline development, reporting and evaluation in healthcare . Can Med Assoc J 2010 ; 182 : 1045 – 2 . Google Scholar CrossRef Search ADS 11 R Core Team . R: A Language and Environment for Statistical Computing . Vienna, Austria : R Foundation for Statistical Computing , 2018 . www.R-project.org. 12 Baumann MH , Strange C , Heffner JE , Light R , Kirby TJ , Klein J et al. Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement . Chest 2001 ; 119 : 590 – 602 . Google Scholar CrossRef Search ADS PubMed 13 De Leyn P , Lismonde M , Ninane V , Noppen M , Slabbynck H , Van Meerhaeghe A et al. Guidelines Belgian Society of Pneumology. Guidelines on the management of spontaneous pneumothorax . Acta Chir Belg 2005 ; 105 : 265 – 7 . Google Scholar CrossRef Search ADS PubMed 14 Rivas de Andres JJ , Jimenez Lopez MF , Molins Lopez-Rodo L , Perez Trullen A , Torres Lanzas J. [ Guidelines for the diagnosis and treatment of spontaneous pneumothorax] . Arch Broncopneumol 2008 ; 44 : 437 – 48 . 15 Tschopp JM , Bintcliffe O , Astoul P , Canalis E , Driesen P , Janssen J et al. ERS task force statement: diagnosis and treatment of primary spontaneous pneumothorax . Eur Respir J 2015 ; 46 : 321 – 35 . Google Scholar CrossRef Search ADS PubMed 16 Guyatt GH , Oxman AD , Vist GE , Kunz R , Falck-Ytter Y , Alonso-Coello P et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations . BMJ 2008 ; 336 : 924 – 6 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

European Journal of Cardio-Thoracic SurgeryOxford University Press

Published: May 16, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off