Quality indicators assessing antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure

Quality indicators assessing antibiotic use in the outpatient setting: a systematic review... Abstract Objectives Quality indicators (QIs) assessing the appropriateness of antibiotic use are essential to identify targets for improvement and guide antibiotic stewardship interventions. The aim of this study was to develop a set of QIs for the outpatient setting from a global perspective. Methods A systematic literature review was performed by searching MEDLINE and relevant web sites in order to retrieve a list of QIs. These indicators were extracted from published trials, guidelines, literature reviews or consensus procedures. This evidence-based set of QIs was evaluated by a multidisciplinary, international group of stakeholders using a RAND-modified Delphi procedure, using two online questionnaires and a face-to-face meeting between them. Stakeholders appraised the QIs’ relevance using a nine-point Likert scale. This work is part of the DRIVE-AB project. Results The systematic literature review identified 43 unique QIs, from 54 studies and seven web sites. Twenty-five stakeholders from 14 countries participated in the consensus procedure. Ultimately, 32 QIs were retained, with a high level of agreement. The set of QIs included structure, process and outcome indicators, targeting both high- and middle- to low-income settings. Most indicators focused on general practice, addressing the common indications for antibiotic use in the community (particularly urinary and respiratory tract infections), and the organization of healthcare facilities. Twelve indicators specifically addressed outpatient parenteral antimicrobial therapy (OPAT). Conclusions We identified a set of 32 outpatient QIs to measure the appropriateness of antibiotic use. These QIs can be used to identify targets for improvement and to evaluate the effects of antibiotic stewardship interventions. Introduction Antibiotics are an essential clinical and public health resource, but bacterial resistance is increasingly threatening their efficacy worldwide.1,2 Up to 80%–90% of antibiotics are prescribed in the outpatient setting,3,4 and significant inter-country variation exists regarding antibiotic consumption, with high levels of inappropriate use.5–7 Since antibiotic use is associated with the emergence of bacterial resistance at both individual and population levels,8,9 antibiotic stewardship interventions are crucial to promote responsible use of antibiotics. A reliable measurement of the quality of health care is essential to trigger improvement.10,11 A significant number of studies have assessed the quality of antibiotic prescribing in the outpatient setting, often using very different methods, underscoring the need for standardized and consensually accepted QIs for the appraisal of antibiotic use.12 Quality indicators (QIs) are ‘a measurable element of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided’.10,13,14 Ideally, they have a clear and direct association with relevant outcomes. QIs help health care providers and policy makers to set priorities for interventions to improve health care. QIs are usually divided into three categories: (i) structure indicators, reflecting the organization of the health care system (e.g. availability of copy of essential drugs list or formulary); (ii) process indicators (e.g. doing relevant diagnostic tests before prescribing antibiotics); and (iii) outcome indicators, focusing on the consequences of interventions (e.g. hospital readmission).13 The Driving Reinvestment in Research and Development and Responsible Antibiotic Use (DRIVE-AB) project is a public–private consortium, funded by the EU Innovative Medicines Initiative (IMI).15 The development of a framework to define responsible antibiotic use is one of its objectives, together with a set of current validated QIs and quantity metrics to evaluate antibiotic use. This study was part of the DRIVE-AB project and had as its primary aim the development of a set of generic QIs for the outpatient setting from a global perspective. According to the DRIVE-AB project objectives, this consensus procedure had to take into account different perspectives, including the medical community, public health and patients, antibiotic research and development (R&D), payers, policy makers, governments and regulators. The QIs should also account for diverse socioeconomic settings, thereby ensuring a global scope. The goal of DRIVE-AB was indeed to define at an overarching level (i.e. by including all relevant perspectives) what constitutes responsible antibiotic use. Materials and methods The set of QIs was developed through a systematic literature review, followed by a RAND-modified Delphi procedure.10,16,17 This review is reported following the PRISMA statement.18 This literature review was conducted alongside five other literature reviews, on responsible use of antibiotics,19 variation in antibiotic use,20 QIs for the inpatient setting,21 quantity metrics for the outpatient setting22 and quantity metrics for the inpatient setting.23 Systematic review of the literature Search strategy We searched the MEDLINE database using the PubMed interface for titles and abstracts of articles describing QIs for outpatient antibiotic use published from the inception of MEDLINE until 12 December 2014. The search strategy keywords were defined with the help of a librarian from one of the participating centres, and were organized around seven key concepts (Figure S1, available as Supplementary data at JAC Online). Inclusion and exclusion criteria We included studies written in English that focused on systemic (i.e. not topical, vaginal or inhaled) antibiotic use in humans in the outpatient setting and that described QIs. Papers on antiviral, antifungal and antiparasitic drugs were excluded, as well as articles describing antibiotic use in tuberculosis and in pathologies included in the Orphanet list of rare diseases.24 An outpatient was defined as a non-hospitalized patient who visits a physician in an ambulatory care setting. We therefore included outpatient parenteral antimicrobial therapy (OPAT). We included papers where indicators were labelled as such, based on published trials, guidelines, literature reviews or consensus procedures, and fulfilled our QI definition (see Introduction). We excluded papers whose full-text could not be retrieved from any of the libraries of the participating centres (eight different catalogues) and from Google Scholar®. Screening process, data collection and analysis All titles and abstracts were independently screened by two investigators (M. L. M. and C. P.) to search for potentially eligible papers, using the literature review management software DistillerSR® (Evidence Partners, Ottawa, Ontario, Canada). The full text of articles was systematically screened if there was no abstract available, or if the abstract was insufficiently detailed to allow a proper assessment of the eligibility criteria. All eligible full-text articles were independently screened by two reviewers (M. L. M. and C. P.), who selected relevant articles based on our predefined inclusion and exclusion criteria. The reference list of all included articles was also screened to look for potential additional papers. One investigator (M. L. M.) performed data extraction using a standardized form and another reviewer (C. P.) double-checked all extracted data from the included articles. Data on relevant QIs were collected, together with the study design and the setting of the study (high- versus middle- to low-income countries25). We categorized the study design as follows: QIs based on an expert consensus/Delphi round (consensus-based indicators); QIs based on a literature review (review-based indicators); and QIs based on guidelines (guideline-based indicators). QIs were classified as ‘Structure’, ‘Process’ and ‘Outcome’ indicators.13 We further specifically identified QIs adapted to the general practice setting and QIs specific to OPAT. During the whole screening and data extraction any disagreement between the two investigators was resolved by discussion, using advice from a third expert if needed (I. C. G. or M. E. H.). Web site search A web site search was also performed (Table S1). The selection of relevant web sites was based on discussion and consensus among the study investigators. Web sites in English from 26 national and international infectious diseases societies, quality improvement and public health organizations were included. All web sites were screened by one reviewer (M. L. M.) using ‘indicator’ and/or ‘antibiotic/antimicrobial’ as search terms. Data extraction was performed with the same standardized form as that used for published papers. RAND-modified Delphi procedure After having established a list of QIs based on the literature review and the web site search, duplicate QIs were deleted and those addressing similar topics were grouped. Each indicator was phrased as a generic statement (e.g. ‘Antibiotics should be prescribed for bacterial infections’). If applicable, a QI was further detailed in terms of one or more numerator–denominator combinations found in the literature or on web sites. The comprehensive list of QIs was presented to a multidisciplinary panel of stakeholders for a RAND-modified Delphi consensus procedure,10,16,17 consisting of two surveys (first and second round) with a face-to-face meeting between them. International stakeholders were invited by e-mail to participate. The selection process of the stakeholders is described elsewhere.21 The complete list of stakeholders is presented in the Acknowledgements section and full details are available in Table S2. First round The list of QIs was converted into an internet-based questionnaire (Figure S2) using SurveyMonkey® (Palo Alto, CA, USA). Respondents were asked to appraise the relevance of the indicators for assessing the quality of antibiotic use in the outpatient setting, using a nine-point Likert scale ranging from 1 (clearly not relevant) to 9 (clearly relevant). They also had a ‘cannot assess’ option, and a comments box for each QI. QIs were: (i) selected if the median score was 8 or 9 with agreement; (ii) held for further discussion if the median score was 8 or 9 without agreement; or (3) rejected if the median score was <8. Agreement was defined as >70% of the scores being in the upper tertile (score 7–9). Stakeholders could also suggest new QIs for further discussion. Consensus meeting Newly suggested QIs, selected QIs with many comments and those ‘held for further discussion’ (category 2) after the first round were discussed during a face-to-face meeting, to which the same stakeholders who participated in the first round were invited. The discussion was led by an expert moderator who was otherwise not part of the study team. Before the meeting, all stakeholders received a detailed summary of the survey results describing their individual replies as well as the group median scores for each suggested QI. During the meeting, stakeholders were asked to accept, reject or rephrase each QI under discussion to reach consensus. Moreover, they could rephrase the QIs that were already selected after the first round. Study investigators did not get directly involved in the discussion. Second round All previously accepted, added and rephrased QIs were presented in a second internet-based questionnaire and sent to all stakeholders who participated in the first round, together with a detailed report of the previous phases. Stakeholders were asked to definitively accept or reject each QI. A ‘cannot assess’ option was offered, as well as the possibility to add comments. Indicators were accepted if >70% of respondents agreed with their selection. Results Systematic review of the literature The literature search identified 3563 articles, of which 287 were considered eligible for full-text screening. After the application of inclusion and exclusion criteria, the screening of reference lists and the web site search, a final set of 54 articles and 7 web sites was retained (Figure 1). The majority of the papers (37/54, 69%) concerned high-income countries (Table S3).26–86 Figure 1. View largeDownload slide Flow chart of the systematic review of the literature. Figure 1. View largeDownload slide Flow chart of the systematic review of the literature. Overall, 356 QIs were extracted and 120 were retained after removing duplicates. Indicators addressing similar topics were grouped, producing a final list of 43 QIs (29 concerning general practice and 14 concerning OPAT), with 113 numerator–denominator combinations (Table S4). This list was presented to the stakeholders for the first round of the consensus procedure. RAND-modified Delphi procedure The first round was completed between August and September 2015. Forty-three stakeholders were contacted and 23 of them (from 12 countries: Belgium, Canada, Chile, Denmark, France, Nigeria, Spain, Sweden, the Netherlands, UK, USA and Vietnam) agreed to participate (Table 1 and Table S2). After the first round, 27 QIs were selected, 10 were rejected and 6 were held for further discussion. Two QIs were newly suggested (Figure 2 and Table S4). Table 1. Stakeholders participating in the RAND-modified Delphi consensus procedure Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) AMR, antimicrobial resistance; GP, general practice; ID, infectious diseases. Group A, medical community; Group B, public health and patients; Group C, antibiotic R&D; Group D, payers, policy makers, government and regulators. a These stakeholders participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study by Monnier et al.21 and Stanić Benić et al.23 They were provided with the results of the first survey on QIs for antibiotic use in the outpatient setting and took part in the consensus meeting. Table 1. Stakeholders participating in the RAND-modified Delphi consensus procedure Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) AMR, antimicrobial resistance; GP, general practice; ID, infectious diseases. Group A, medical community; Group B, public health and patients; Group C, antibiotic R&D; Group D, payers, policy makers, government and regulators. a These stakeholders participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study by Monnier et al.21 and Stanić Benić et al.23 They were provided with the results of the first survey on QIs for antibiotic use in the outpatient setting and took part in the consensus meeting. Figure 2. View largeDownload slide Flow chart of the RAND-modified Delphi consensus procedure. Figure 2. View largeDownload slide Flow chart of the RAND-modified Delphi consensus procedure. The face-to-face consensus meeting took place at Schiphol airport (the Netherlands) on 30 September 2015. Seven stakeholders were present and three participated by teleconference. Two of these stakeholders (from Greece and Slovenia) participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study in the framework of the DRIVE-AB project. They were provided with the results of the first survey and took part in the consensus discussion on outpatient QIs (OQIs) (Table 1 and Table S2). Thirty-three QIs (with 68 numerator-denominator combinations) were selected, with 6 QIs having been rephrased and 2 merged with other already existing QIs (Figure 2 and Table S4). The second round was carried out between the end of December 2015 and February 2016. Twenty stakeholders participated (Table 1). Thirty-two QIs (with 67 numerator–denominator combinations) were selected, with an average consensus score of 94%. One QI was rejected (Figure 2 and Table S4). Final set of OQIs The final set of 32 OQIs is presented in Table 2. Twenty QIs addressed general practice, 11 addressed OPAT and 1 addressed both. Full details with all numerator–denominator combinations are available in Table S5. Table 2. The final set of 32 consensually validated quality indicators assessing antibiotic use in the outpatient setting Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B a Category of the study design: A, consensus-based indicators; B, review-based indicators; C, guideline-based indicators. Table 2. The final set of 32 consensually validated quality indicators assessing antibiotic use in the outpatient setting Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B a Category of the study design: A, consensus-based indicators; B, review-based indicators; C, guideline-based indicators. Discussion Here we propose a set of 32 consensually validated QIs to assess antibiotic use in outpatients, targeting mostly general practice and OPAT, in both high-income and middle- to low-income settings. The selected QIs are purposefully generic. DRIVE-AB goals were to come up with generic metrics for responsible use to be used worldwide; stakeholders from a broad area of expertise (who probably all view ‘responsible’ differently), and not only general practitioners (GPs), were thus asked to rate the relevance of these OQIs defining responsible use, for a broad range of infections in a broad range of countries with the aim of ending up with broadly supported QIs, from a global perspective. This makes our consensus procedure very different from those performed in the original included studies, as in those studies the QIs were very specifically defined, fitting a specific setting and, most of the time, a specific patient group or disease, and involved experts representing that setting (mostly GPs). This list of QIs encompasses a wide range of clinical conditions, aiming at a comprehensive evaluation of antibiotic use. Only one outcome indicator (OQI-32, Table 2) was retained at the end of the consensus procedure. This is in line with the paucity of outcome indicators in other sets of QIs assessing antibiotic use.17,26,87 They represent the endpoint of a chain of factors and their evaluation only indirectly estimates the quality of care provided and they are thus potentially influenced by many concurrent factors. In contrast, structure and process indicators provide a direct evaluation of quality in health care and thus allow direct identification of targets for intervention. Our study is original and has several strengths. We performed a systematic review of QIs available in the literature, followed by a stepwise consensus procedure (RAND-modified Delphi procedure). This design has been previously applied to identify QIs in different health care domains, although the methodology varied significantly in published papers.88 The method we chose has been applied in several settings17,89,90 and complies with the recommendations suggested by Kötter et al.91 in their systematic review on guideline-based development of QIs, and by Boulkedid et al.88 in their systematic review on use and reporting of the Delphi method. We think it ensures a robust content (‘Does the measurement represent every element of the construct?’) and face validity (‘Does the measurement tool appear to measure what we want to measure?’) for the selected QIs. In the consensus procedure, we included stakeholders from different backgrounds, including medical professionals, patients’ representatives, R&D experts from the pharmaceutical industry, government, and relevant agencies such as the US CDC and the ECDC. Participants came from low-, middle- and high-income countries, from 12 countries and four different continents. There are, however, some limitations that need to be acknowledged. We only searched the MEDLINE database for our systematic review and we did not systematically explore the grey literature.92 However, we did screen relevant web sites and the reference lists of all included articles. Clinimetric properties of QIs were not assessed in our study, and need to be further validated in specific settings. Clinimetric properties are characteristics of indicators such as measurability (feasibility), applicability, acceptability, reliability, potential for improvement and sensitivity to change.10,89,93 Our objective was to develop a list of generic QIs that can be assessed and adapted to national specificities and guidelines. The most critical aspect for some of the selected QIs is probably measurability, since in several cases patients’ demographic data and clinical information are needed to reliably assess the numerator and the denominator defining the indicator. This information is difficult to obtain on a large scale in many countries.26,59 A few years ago, the European Surveillance of Antimicrobial Consumption Network (ESAC-Net) developed a set of so-called drug-specific quality indicators87 for antibiotic use in the outpatient setting. However, these indicators were based on a quantitative evaluation of antibiotic consumption, and were actually more quantity metrics rather than QIs, since they can be meaningfully interpreted only in terms of comparison between different settings and different periods of time. The QIs we selected reflect the quality of care, having significance per se, without the need for comparison and not requiring the setting of thresholds (such as ‘quinolones should not exceed 10% of total antibiotic prescriptions’), which can be somewhat arbitrary. Thus, we think that this set of QIs and those developed by the ESAC-Net should not be considered as mutually exclusive, since they approach the phenomenon of antibiotic prescription in different ways and with substantially different methodology, thus being able to provide complementary information. A systematic review of QIs in primary care has been recently published by Saust et al.94 However, their literature search was less extensive than our systematic review. Moreover, they did not use a consensus procedure to validate the identified indicators. Also, many of the indicators they included are based on a quantitative evaluation of antibiotic consumption, similar to those from the ESAC-Net.26,87 The WHO also provided detailed lists of QIs.47,50,81 The WHO indicators target low- to middle-income countries, but they are generic and not focused on antibiotic treatments in outpatients. We nevertheless derived a relevant number of structure and process QIs (16 combinations of numerators and denominators, grouped in 13 QIs; for details see Tables S4 and S5) from these WHO indicators. Our consensus procedure confirmed that these QIs have high face validity and that they could represent a useful tool to assess the quality of antibiotic prescriptions in low-to- middle-income countries. Regarding OPAT, we extracted QIs from two guidelines84,86 and a critical appraisal of evidence.85 To the best of our knowledge, there are no QIs for OPAT available in the literature, so we present here the first comprehensive set of validated QIs for OPAT. Outpatient parenteral antibiotic treatment is increasingly being used, mainly to optimize patient comfort and decrease risks and costs related to hospitalization,95 but current evidence to guide its implementation is limited85 and variability in practice is frequent.96 Thus, a consensually validated set of QIs could be of great value to evaluate OPAT programmes and set targets for improvement. Overall, our search identified a relatively low number of unique QIs targeting antibiotic use in the outpatient setting. Further investigation is needed to develop new QIs, including some focusing on diagnostic processes,94 using an adequate methodology.10,11,13 Assessing the relationship between process QIs and outcome indicators for appropriate antibiotic use (e.g. ecological consequences of antibiotic consumption) is also needed to further validate the QIs;97 this was not done for the QIs that we identified in our literature review. In conclusion, we listed a set of generic QIs intended to evaluate the quality of antibiotic use in the outpatient setting, in a wide range of clinical conditions, taking into account the perspectives of a wide range of stakeholders involved with antibiotics. These QIs can be used to measure the appropriateness of current antibiotic use at the individual, regional or supra-regional level. Thus they can be used to identify targets for improvement and to evaluate the effects of antibiotic stewardship interventions. Indeed, these are generic QIs that are globally applicable. These QIs need to be tested in practice, carefully evaluating their clinimetric properties in different settings and conditions. Acknowledgements Members of the DRIVE-AB WP1 group The authors are grateful to all stakeholders that participated in the consensus procedure for sharing their knowledge and insights on the subject: Bojana Beović, Luis Bavestrello, Jon Cohen, Pieter-Jan Cortoos, Harpal Dhillon, Richard Drew, Prabhavathi Fernandes, David Findlay, Ian Friedland, Ian Gould, Lauri Hicks, Christopher Longshaw, Carl Llor, Allison McGeer, Iruka N. Okeke, Diamantis Plachouras, Garyfallia Poulakou, John H. Rex, Thomas Tängdén, Laura Trolle Saust, Claudine Richon, France Roblot, Robert Vander Stichele, Marcel Verweij and Heiman Wertheim.  We thank the DRIVE-AB steering committee for critically reviewing the manuscript.  We thank also Sjaak de Gouw (Netherlands), who acted as moderator in the face-to-face meeting, and Elmie Peters and Rikie Deurenberg Vos (Netherlands) who helped with structuring the search strategy for the MEDLINE search.  The results of this study have been presented as a poster and as an oral presentation at the ‘Train the Trainer’ event at the 26th European Congress of Clinical Microbiology and Infectious Diseases (ECCMID 2016), which took take place in Amsterdam, Netherlands, from 9 to 12 April 2016. Funding This work was supported by the Innovative Medicines Initiative (IMI) Joint Undertaking (grant agreement no. 115618 - Driving re-investment in R&D and responsible antibiotic use—DRIVE-AB—www.drive-ab.eu). Resources are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007–2013) and European Federation of Pharmaceutical Industries and Associations (EFPIA) in-kind contribution. Transparency declarations I. C. G. reports an educational grant from Pfizer, outside the submitted work. The remaining authors have none to declare. This article forms part of a Supplement sponsored by DRIVE-AB. Supplementary data Figures S1 and S2 and Tables S1 to S5 are available as Supplementary data at JAC Online. References 1 Laxminarayan R , Duse A , Wattal C et al. Antibiotic resistance—the need for global solutions . Lancet Infect Dis 2013 ; 13 : 1057 – 98 . Google Scholar CrossRef Search ADS 2 Holmes AH , Moore LS , Sundsfjord A et al. Understanding the mechanisms and drivers of antimicrobial resistance . Lancet 2016 ; 387 : 176 – 87 . Google Scholar CrossRef Search ADS 3 European Centre for Disease Prevention and Control (ECDC) . Surveillance of Antimicrobial Consumption in Europe 2012. 2014 . http://ecdc.europa.eu/en/publications/Publications/antimicrobial-consumption-europe-esac-net-2012.pdf. 4 World Health Organization (WHO) . Community-Based Surveillance of Antimicrobial Use and Resistance in Resource-Constrained Settings. 2009 . http://apps.who.int/medicinedocs/documents/s16168e/s16168e.pdf? ua=1. 5 Adriaenssens N , Coenen S , Versporten A et al. European Surveillance of Antimicrobial Consumption (ESAC): quality appraisal of antibiotic use in Europe . J Antimicrob Chemother 2011 ; 66 : vi71 – 7 . Google Scholar CrossRef Search ADS 6 Fleming-Dutra KE , Hersh AL , Shapiro DJ et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011 . JAMA 2016 ; 315 : 1864 – 73 . Google Scholar CrossRef Search ADS 7 Spivak ES , Cosgrove SE , Srinivasan A. Measuring appropriate antimicrobial use: attempts at opening the black box . Clin Infect Dis 2016 ; 63 : 1639 – 44 . Google Scholar CrossRef Search ADS 8 Costelloe C , Metcalfe C , Lovering A et al. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis . BMJ 2010 ; 340 : c2096. Google Scholar CrossRef Search ADS 9 Bell BG , Schellevis F , Stobberingh E et al. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance . BMC Infect Dis 2014 ; 14 : 13. Google Scholar CrossRef Search ADS 10 Campbell S , Braspenning J , Hutchinson A et al. Research methods used in developing and applying quality indicators in primary care . Qual Saf Health Care 2002 ; 11 : 358 – 64 . Google Scholar CrossRef Search ADS 11 Pronovost PJ , Miller M , Wachter RM. The GAAP in quality measurement and reporting . JAMA 2007 ; 298 : 1800 – 2 . Google Scholar CrossRef Search ADS 12 DePestel DD , Eiland EH , Lusardi K et al. Assessing appropriateness of antimicrobial therapy: in the eye of the interpreter . Clin Infect Dis 2014 ; 59 : S154 – 61 . Google Scholar CrossRef Search ADS 13 Donabedian A. The quality of care. How can it be assessed? JAMA 1988 ; 260 : 1743 – 8 . Google Scholar CrossRef Search ADS 14 Lawrence M , Olesen F. Indicators of quality in health care . Eur J Gen Pract 1997 ; 3 : 103 – 8 . Google Scholar CrossRef Search ADS 15 Harbarth S , Theuretzbacher U , Hackett J et al. Antibiotic research and development: business as usual? J Antimicrob Chemother 2015 ; 70 : 1604 – 7 . Google Scholar CrossRef Search ADS 16 Fitch K , Bernstein SJ , Aguilar MD et al. The RAND/UCLA Appropriateness Method User’s Manual. 2001 . https://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf. 17 van den Bosch CM , Geerlings SE , Natsch S et al. Quality indicators to measure appropriate antibiotic use in hospitalized adults . Clin Infect Dis 2015 ; 60 : 281 – 91 . Google Scholar CrossRef Search ADS 18 Moher D , Liberati A , Tetzlaff J et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . BMJ 2009 ; 339 : b2535 . Google Scholar CrossRef Search ADS 19 Monnier AA , Eisenstein BI , Hulscher ME et al. Towards a global definition of responsible antibiotic use: results of an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi3 – vi16 . 20 Zanichelli V , Monnier AA , Gyssens IC et al. Variation in antibiotic use among and within different settings: a systematic review . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi17 – vi29 . 21 Monnier AA , Schouten J , Le Maréchal M et al. Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi30 – vi39 . 22 Versporten A , Gyssens IC , Pulcini C et al. Metrics to assess the quantity of antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi59 – vi66 . 23 Stanić Benić M , Milanič R , Monnier AA et al. Metrics for quantifying antibiotic use in the hospital setting: results from a systematic review and an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi50 – vi58 . 24 Orphanet . The Portal for Rare Diseases and Orphan Drugs. http://www.orpha.net/consor/cgi-bin/index.php? lng=EN. 25 The World Bank . The World Bank. Data: Countries and Economies. http://data.worldbank.org/country. 26 Adriaenssens N , Coenen S , Tonkin-Crine S et al. European Surveillance of Antimicrobial Consumption (ESAC): disease-specific quality indicators for outpatient antibiotic prescribing . BMJ Qual Saf 2011 ; 20 : 764 – 72 . Google Scholar CrossRef Search ADS 27 Juhasz Z , Benko R , Matuz M et al. Treatment of acute cystitis in Hungary: comparison with national guidelines and with disease-specific quality indicators . Scand J Infect Dis 2013 ; 45 : 612 – 5 . Google Scholar CrossRef Search ADS 28 Adriaenssens N , Bartholomeeusen S , Ryckebosch P et al. Quality of antibiotic prescription during office hours and out-of-hours in Flemish primary care, using European quality indicators . Eur J Gen Pract 2014 ; 20 : 114 – 20 . Google Scholar CrossRef Search ADS 29 Hansen MP , Bjerrum L , Gahrn-Hansen B et al. Quality indicators for diagnosis and treatment of respiratory tract infections in general practice: a modified Delphi study . Scand J Prim Health Care 2010 ; 28 : 4 – 11 . Google Scholar CrossRef Search ADS 30 Hansen MP , Bjerrum L , Gahrn-Hansen B et al. Quality indicators for treatment of respiratory tract infections? An assessment by Danish general practitioners . Eur J Gen Pract 2013 ; 19 : 85 – 91 . Google Scholar CrossRef Search ADS 31 Linder JA , Schnipper JL , Tsurikova R et al. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a cluster randomised controlled trial . Inform Prim Care 2009 ; 17 : 231 – 40 . 32 DeVries A , Li CHW , Sridhar G et al. Impact of medical homes on quality, healthcare utilization, and costs . Am J Manag Care 2012 ; 18 : 534 – 44 . 33 Gonzales R , Anderer T , McCulloch CE et al. A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis . JAMA Intern Med 2013 ; 173 : 267 – 73 . Google Scholar CrossRef Search ADS 34 Kale MS , Bishop TF , Federman AD et al. Trends in the overuse of ambulatory health care services in the U.S . JAMA Intern Med 2013 ; 173 : 142 – 8 . Google Scholar CrossRef Search ADS 35 Miller G , Britt H , Pan Y et al. Relationship between general practitioner certification and characteristics of care . Med Care 2004 ; 42 : 770. Google Scholar CrossRef Search ADS 36 Ma J , Stafford RS. Quality of US outpatient care: temporal changes and racial/ethnic disparities . Arch Intern Med 2005 ; 165 : 1354 – 61 . Google Scholar CrossRef Search ADS 37 Mainous AG , Lambourne CA , Nietert PJ. Impact of a clinical decision support system on antibiotic prescribing for acute respiratory infections in primary care: quasi-experimental trial . J Am Med Inform Assoc 2013 ; 20 : 317 – 24 . Google Scholar CrossRef Search ADS 38 Gerber JS , Prasad PA , Fiks AG et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial . JAMA 2013 ; 309 : 2345 – 52 . Google Scholar CrossRef Search ADS 39 Van Roosmalen MS , Braspenning JC , De Smet PA et al. Antibiotic prescribing in primary care: first choice and restrictive prescribing are two different traits . Qual Saf Health Care 2007 ; 16 : 105 – 9 . Google Scholar CrossRef Search ADS 40 Bisno AL , Gerber MA , Gwaltney JM Jr et al. Diagnosis and management of group A streptococcal pharyngitis: a practice guideline . Clin Infect Dis 1997 ; 25 : 574 – 83 . Google Scholar CrossRef Search ADS 41 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Acute Otitis Externa (AOE). 2011 . https://www.qualitymeasures.ahrq.gov/summaries/summary/28014. 42 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Otitis Media with Effusion (OME). 2011 . https://www.qualitymeasures.ahrq.gov/summaries/summary/28018. 43 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Emergency Medicine. 2013 . https://www.qualitymeasures.ahrq.gov/. 44 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Community-Acquired Bacterial Pneumonia. 2006 . https://www.qualitymeasures.ahrq.gov/. 45 Linder JA , Ma J , Bates DW et al. Electronic health record use and the quality of ambulatory care in the United States . Arch Intern Med 2007 ; 167 : 1400 – 5 . Google Scholar CrossRef Search ADS 46 World Health Organization (WHO) . Guideline: Updates on the Management of Severe Acute Malnutrition in Infants and Children. 2013 . http://apps.who.int/iris/bitstream/10665/95584/1/9789241506328_eng.pdf. 47 World Health Organization (WHO) . How to Investigate Drug Use in Health Facilities. 1993 . http://apps.who.int/medicinedocs/pdf/s2289e/s2289e.pdf. 48 Hazra A , Tripathi SK , Alam MS. Prescribing and dispensing activities at the health facilities of a non-governmental organization . Natl Med J India 2000 ; 13 : 177 – 82 . 49 Wensing M , Broge B , Riens B et al. Quality circles to improve prescribing of primary care physicians. Three comparative studies . Pharmacoepidemiol Drug Saf 2009 ; 18 : 763. Google Scholar CrossRef Search ADS 50 World Health Organization (WHO) . Using Indicators to Measure Country Pharmaceutical Situations. Fact Book on WHO Level I and Level II Monitoring Indicators. 2006 . http://www.who.int/medicines/publications/WHOTCM2006.2A.pdf. 51 Cheraghali AM , Idries AM. Availability, affordability, and prescribing pattern of medicines in Sudan . Pharm World Sci 2009 ; 31 : 209 – 15 . Google Scholar CrossRef Search ADS 52 Ashworth M , Golding S , Majeed A. Prescribing indicators and their use by primary care groups to influence prescribing . J Clin Pharm Ther 2002 ; 27 : 197 – 204 . Google Scholar CrossRef Search ADS 53 Williams D , Bennett K , Feely J. The application of prescribing indicators to a primary care prescription database in Ireland . Eur J Clin Pharmacol 2005 ; 61 : 127 – 33 . Google Scholar CrossRef Search ADS 54 Campbell SM , Roland MO , Quayle JA et al. Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health 1998 ; 20 : 414 – 21 . Google Scholar CrossRef Search ADS 55 Wilcock M , Mackenzie IF. Quality indicators for general practice . J Public Health Med 1999 ; 21 : 357. Google Scholar CrossRef Search ADS 56 Campbell SM , Cantrill JA , Roberts D. Prescribing indicators for UK general practice: Delphi consultation study . BMJ 2000 ; 321 : 425. Google Scholar CrossRef Search ADS 57 Fernández Urrusuno R , Montero Balosa MC , Pérez Pérez P et al. Compliance with quality prescribing indicators in terms of their relationship to financial incentives . Eur J Clin Pharmacol 2013 ; 69 : 1845 – 53 . Google Scholar CrossRef Search ADS 58 Altiner A , Wilm S , Wegscheider K et al. Fluoroquinolones to treat uncomplicated acute cough in primary care: predictors for unjustified prescribing of antibiotics . J Antimicrob Chemother 2010 ; 65 : 1521 – 5 . Google Scholar CrossRef Search ADS 59 Pulcini C , Lions C , Ventelou B et al. Approaching the quality of antibiotic prescriptions in primary care using reimbursement data . Eur J Clin Microbiol Infect Dis 2013 ; 32 : 325 – 32 . Google Scholar CrossRef Search ADS 60 National Institute for Health and Care Excellence . Respiratory Tract Infections (Self-Limiting): Prescribing Antibiotics. 2008 . https://www.nice.org.uk/guidance/cg69/chapter/1-Guidance. 61 Wessell AM , Liszka HA , Nietert PJ et al. Achievable benchmarks of care for primary care quality indicators in a practice-based research network . Am J Med Qual 2008 ; 23 : 39 – 46 . Google Scholar CrossRef Search ADS 62 Guyon AB , Barman A , Ahmed JU et al. A baseline survey on use of drugs at the primary health care level in Bangladesh . Bull World Health Organ 1994 ; 72 : 265 – 71 . 63 Otoom S , Batieha A , Hadidi H et al. Evaluation of drug use in Jordan using WHO prescribing indicators . East Mediterr Health J 2002 ; 8 : 537 – 43 . 64 Naseeb TA , Nasser MA. Drug prescribing indicators in primary health care centers in Bahrain . Saudi Med J 2005 ; 26 : 1436 – 8 . 65 Nsimba SE. Assessing prescribing and patient care indicators for children under five years old with malaria and other disease conditions in public primary health care facilities . Southeast Asian J Trop Med Public Health 2006 ; 37 : 206 – 14 . 66 Shrestha N , Samir KC , Baltussen R et al. Practical approach to lung health in Nepal: better prescribing and reduction of cost . Trop Med Int Health 2006 ; 11 : 765 – 72 . Google Scholar CrossRef Search ADS 67 De Costa A , Bhartiya S , Eltayb A et al. Patterns of drug use in the public sector primary health centers of Bhopal district . Pharm World Sci 2008 ; 30 : 584 – 9 . Google Scholar CrossRef Search ADS 68 Chen W , Tang S , Sun J et al. Availability and use of essential medicines in China: manufacturing, supply, and prescribing in Shandong and Gansu provinces . BMC Health Serv Res 2010 ; 10 : 211. Google Scholar CrossRef Search ADS 69 Dong L , Yan H , Wang D. Drug prescribing indicators in village health clinics across 10 provinces of Western China . Fam Pract 2011 ; 28 : 63 – 7 . Google Scholar CrossRef Search ADS 70 Ahmed SM , Islam QS. Availability and rational use of drugs in primary healthcare facilities following the national drug policy of 1982: is Bangladesh on right track? J Health Popul Nutr 2012 ; 30 : 99 – 108 . 71 Desalegn AA. Assessment of drug use pattern using WHO prescribing indicators at Hawassa University Teaching and Referral Hospital, south Ethiopia: a cross-sectional study . BMC Health Serv Res 2013 ; 13 : 170. Google Scholar CrossRef Search ADS 72 Banerjee I , Bhadury T. Prescribing pattern of interns in a primary health center in India . J Basic Clin Pharm 2014 ; 5 : 40 – 3 . Google Scholar CrossRef Search ADS 73 Song Y , Bian Y , Petzold M et al. The impact of China’s national essential medicine system on improving rational drug use in primary health care facilities: an empirical study in four provinces . BMC Health Serv Res 2014 ; 14 : 507. Google Scholar CrossRef Search ADS 74 Moghadamnia AA , Mirbolooki MR , Aghili MB. General practitioner prescribing patterns in Babol city, Islamic Republic of Iran . East Mediterr Health J 2002 ; 8 : 550 – 5 . 75 Esmaily HM , Savage C , Vahidi R et al. Identifying outcome-based indicators and developing a curriculum for a continuing medical education programme on rational prescribing using a modified Delphi process . BMC Med Educ 2008 ; 8 : 33. Google Scholar CrossRef Search ADS 76 Esmaily HM , Silver I , Shiva S et al. Can rational prescribing be improved by an outcome-based educational approach? A randomized trial completed in Iran . J Contin Educ Health Prof 2010 ; 30 : 11 – 8 . Google Scholar CrossRef Search ADS 77 Otoom S , Culligan K , Al-Assoomi B et al. Analysis of drug prescriptions in primary health care centres in Bahrain . East Mediterr Health J 2010 ; 16 : 511 – 5 . Google Scholar CrossRef Search ADS 78 Rasool BKA , Fahmy SA , Abu-Gharbieh EF et al. Professional practices and perception towards rational use of medicines according to WHO methodology in United Arab Emirates . Pharm Pract 2010 ; 8 : 70 – 6 . 79 Avery AJ , Dex GM , Mulvaney C et al. Development of prescribing-safety indicators for GPs using the RAND Appropriateness Method . Br J Gen Pract 2011 ; 61 : e526 – 36 . Google Scholar CrossRef Search ADS 80 Irvine L , Flynn RW , Libby G et al. Drugs dispensed in primary care during pregnancy: a record-linkage analysis in Tayside, Scotland . Drug Saf 2010 ; 33 : 593 – 604 . Google Scholar CrossRef Search ADS 81 World Health Organization (WHO) . Indicators for Monitoring National Drug Policies. A Practical Manual. 1999 . http://apps.who.int/medicinedocs/pdf/whozip14e/whozip14e.pdf. 82 Awad AI , Himad HA. Drug-use practices in teaching hospitals of Khartoum State, Sudan . Eur J Clin Pharmacol 2006 ; 62 : 1087 – 93 . Google Scholar CrossRef Search ADS 83 Steel N , Melzer D , Shekelle P et al. Developing quality indicators for older adults: transfer from the USA to the UK is feasible . Qual Saf Health Care 2004 ; 13 : 260. Google Scholar CrossRef Search ADS 84 Chapman AL , Seaton RA , Cooper MA et al. Good practice recommendations for outpatient parenteral antimicrobial therapy (OPAT) in adults in the UK: a consensus statement . J Antimicrob Chemother 2012 ; 67 : 1053 – 62 . Google Scholar CrossRef Search ADS 85 Muldoon EG , Snydman DR , Penland EC et al. Are we ready for an outpatient parenteral antimicrobial therapy bundle? A critical appraisal of the evidence . Clin Infect Dis 2013 ; 57 : 419 – 24 . Google Scholar CrossRef Search ADS 86 Tice AD , Rehm SJ , Dalovisio JR et al. Practice guidelines for outpatient parenteral antimicrobial therapy . Clin Infect Dis 2004 ; 38 : 1651 – 71 . Google Scholar CrossRef Search ADS 87 Coenen S , Ferech M , Haaijer-Ruskamp FM et al. European Surveillance of Antimicrobial Consumption (ESAC): quality indicators for outpatient antibiotic use in Europe . Qual Saf Health Care 2007 ; 16 : 440 – 5 . Google Scholar CrossRef Search ADS 88 Boulkedid R , Abdoul H , Loustau M et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review . PLoS One 2011 ; 6 : e20476. Google Scholar CrossRef Search ADS 89 Hommel I , van Gurp PJ , Tack CJ et al. Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway . BMJ Qual Saf 2015 ; 25 : 525 – 34 . Google Scholar CrossRef Search ADS 90 Hermanides HS , Hulscher ME , Schouten JA et al. Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care . Clin Infect Dis 2008 ; 46 : 703 – 11 . Google Scholar CrossRef Search ADS 91 Kötter T , Blozik E , Scherer M. Methods for the guideline-based development of quality indicators—a systematic review . Implement Sci 2012 ; 7 : 21. Google Scholar CrossRef Search ADS 92 Mahood Q , Van Eerd D , Irvin E. Searching for grey literature for systematic reviews: challenges and benefits . Res Synth Methods 2014 ; 5 : 221 – 34 . Google Scholar CrossRef Search ADS 93 van den Bosch CM , Hulscher ME , Natsch S et al. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study . Clin Microbiol Infect 2016 ; 22 : 888.e1 – 888.e9 . Google Scholar CrossRef Search ADS 94 Saust LT , Monrad RN , Hansen MP et al. Quality assessment of diagnosis and antibiotic treatment of infectious diseases in primary care: a systematic review of quality indicators . Scand J Prim Health Care 2016 ; 34 : 258 – 66 . Google Scholar CrossRef Search ADS 95 Chapman AL , Dixon S , Andrews D et al. Clinical efficacy and cost-effectiveness of outpatient parenteral antibiotic therapy (OPAT): a UK perspective . J Antimicrob Chemother 2009 ; 64 : 1316 – 24 . Google Scholar CrossRef Search ADS 96 Muldoon EG , Switkowski K , Tice A et al. A national survey of infectious disease practitioners on their use of outpatient parenteral antimicrobial therapy (OPAT) . Infect Dis (Lond) 2015 ; 47 : 39 – 45 . Google Scholar CrossRef Search ADS 97 Schuts EC , Hulscher ME , Mouton JW et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis . Lancet Infect Dis 2016 ; 16 : 847 – 56 . Google Scholar CrossRef Search ADS © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Antimicrobial Chemotherapy Oxford University Press

Quality indicators assessing antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure

Loading next page...
 
/lp/ou_press/quality-indicators-assessing-antibiotic-use-in-the-outpatient-setting-GXqlMpRbuc
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy.
ISSN
0305-7453
eISSN
1460-2091
D.O.I.
10.1093/jac/dky117
Publisher site
See Article on Publisher Site

Abstract

Abstract Objectives Quality indicators (QIs) assessing the appropriateness of antibiotic use are essential to identify targets for improvement and guide antibiotic stewardship interventions. The aim of this study was to develop a set of QIs for the outpatient setting from a global perspective. Methods A systematic literature review was performed by searching MEDLINE and relevant web sites in order to retrieve a list of QIs. These indicators were extracted from published trials, guidelines, literature reviews or consensus procedures. This evidence-based set of QIs was evaluated by a multidisciplinary, international group of stakeholders using a RAND-modified Delphi procedure, using two online questionnaires and a face-to-face meeting between them. Stakeholders appraised the QIs’ relevance using a nine-point Likert scale. This work is part of the DRIVE-AB project. Results The systematic literature review identified 43 unique QIs, from 54 studies and seven web sites. Twenty-five stakeholders from 14 countries participated in the consensus procedure. Ultimately, 32 QIs were retained, with a high level of agreement. The set of QIs included structure, process and outcome indicators, targeting both high- and middle- to low-income settings. Most indicators focused on general practice, addressing the common indications for antibiotic use in the community (particularly urinary and respiratory tract infections), and the organization of healthcare facilities. Twelve indicators specifically addressed outpatient parenteral antimicrobial therapy (OPAT). Conclusions We identified a set of 32 outpatient QIs to measure the appropriateness of antibiotic use. These QIs can be used to identify targets for improvement and to evaluate the effects of antibiotic stewardship interventions. Introduction Antibiotics are an essential clinical and public health resource, but bacterial resistance is increasingly threatening their efficacy worldwide.1,2 Up to 80%–90% of antibiotics are prescribed in the outpatient setting,3,4 and significant inter-country variation exists regarding antibiotic consumption, with high levels of inappropriate use.5–7 Since antibiotic use is associated with the emergence of bacterial resistance at both individual and population levels,8,9 antibiotic stewardship interventions are crucial to promote responsible use of antibiotics. A reliable measurement of the quality of health care is essential to trigger improvement.10,11 A significant number of studies have assessed the quality of antibiotic prescribing in the outpatient setting, often using very different methods, underscoring the need for standardized and consensually accepted QIs for the appraisal of antibiotic use.12 Quality indicators (QIs) are ‘a measurable element of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided’.10,13,14 Ideally, they have a clear and direct association with relevant outcomes. QIs help health care providers and policy makers to set priorities for interventions to improve health care. QIs are usually divided into three categories: (i) structure indicators, reflecting the organization of the health care system (e.g. availability of copy of essential drugs list or formulary); (ii) process indicators (e.g. doing relevant diagnostic tests before prescribing antibiotics); and (iii) outcome indicators, focusing on the consequences of interventions (e.g. hospital readmission).13 The Driving Reinvestment in Research and Development and Responsible Antibiotic Use (DRIVE-AB) project is a public–private consortium, funded by the EU Innovative Medicines Initiative (IMI).15 The development of a framework to define responsible antibiotic use is one of its objectives, together with a set of current validated QIs and quantity metrics to evaluate antibiotic use. This study was part of the DRIVE-AB project and had as its primary aim the development of a set of generic QIs for the outpatient setting from a global perspective. According to the DRIVE-AB project objectives, this consensus procedure had to take into account different perspectives, including the medical community, public health and patients, antibiotic research and development (R&D), payers, policy makers, governments and regulators. The QIs should also account for diverse socioeconomic settings, thereby ensuring a global scope. The goal of DRIVE-AB was indeed to define at an overarching level (i.e. by including all relevant perspectives) what constitutes responsible antibiotic use. Materials and methods The set of QIs was developed through a systematic literature review, followed by a RAND-modified Delphi procedure.10,16,17 This review is reported following the PRISMA statement.18 This literature review was conducted alongside five other literature reviews, on responsible use of antibiotics,19 variation in antibiotic use,20 QIs for the inpatient setting,21 quantity metrics for the outpatient setting22 and quantity metrics for the inpatient setting.23 Systematic review of the literature Search strategy We searched the MEDLINE database using the PubMed interface for titles and abstracts of articles describing QIs for outpatient antibiotic use published from the inception of MEDLINE until 12 December 2014. The search strategy keywords were defined with the help of a librarian from one of the participating centres, and were organized around seven key concepts (Figure S1, available as Supplementary data at JAC Online). Inclusion and exclusion criteria We included studies written in English that focused on systemic (i.e. not topical, vaginal or inhaled) antibiotic use in humans in the outpatient setting and that described QIs. Papers on antiviral, antifungal and antiparasitic drugs were excluded, as well as articles describing antibiotic use in tuberculosis and in pathologies included in the Orphanet list of rare diseases.24 An outpatient was defined as a non-hospitalized patient who visits a physician in an ambulatory care setting. We therefore included outpatient parenteral antimicrobial therapy (OPAT). We included papers where indicators were labelled as such, based on published trials, guidelines, literature reviews or consensus procedures, and fulfilled our QI definition (see Introduction). We excluded papers whose full-text could not be retrieved from any of the libraries of the participating centres (eight different catalogues) and from Google Scholar®. Screening process, data collection and analysis All titles and abstracts were independently screened by two investigators (M. L. M. and C. P.) to search for potentially eligible papers, using the literature review management software DistillerSR® (Evidence Partners, Ottawa, Ontario, Canada). The full text of articles was systematically screened if there was no abstract available, or if the abstract was insufficiently detailed to allow a proper assessment of the eligibility criteria. All eligible full-text articles were independently screened by two reviewers (M. L. M. and C. P.), who selected relevant articles based on our predefined inclusion and exclusion criteria. The reference list of all included articles was also screened to look for potential additional papers. One investigator (M. L. M.) performed data extraction using a standardized form and another reviewer (C. P.) double-checked all extracted data from the included articles. Data on relevant QIs were collected, together with the study design and the setting of the study (high- versus middle- to low-income countries25). We categorized the study design as follows: QIs based on an expert consensus/Delphi round (consensus-based indicators); QIs based on a literature review (review-based indicators); and QIs based on guidelines (guideline-based indicators). QIs were classified as ‘Structure’, ‘Process’ and ‘Outcome’ indicators.13 We further specifically identified QIs adapted to the general practice setting and QIs specific to OPAT. During the whole screening and data extraction any disagreement between the two investigators was resolved by discussion, using advice from a third expert if needed (I. C. G. or M. E. H.). Web site search A web site search was also performed (Table S1). The selection of relevant web sites was based on discussion and consensus among the study investigators. Web sites in English from 26 national and international infectious diseases societies, quality improvement and public health organizations were included. All web sites were screened by one reviewer (M. L. M.) using ‘indicator’ and/or ‘antibiotic/antimicrobial’ as search terms. Data extraction was performed with the same standardized form as that used for published papers. RAND-modified Delphi procedure After having established a list of QIs based on the literature review and the web site search, duplicate QIs were deleted and those addressing similar topics were grouped. Each indicator was phrased as a generic statement (e.g. ‘Antibiotics should be prescribed for bacterial infections’). If applicable, a QI was further detailed in terms of one or more numerator–denominator combinations found in the literature or on web sites. The comprehensive list of QIs was presented to a multidisciplinary panel of stakeholders for a RAND-modified Delphi consensus procedure,10,16,17 consisting of two surveys (first and second round) with a face-to-face meeting between them. International stakeholders were invited by e-mail to participate. The selection process of the stakeholders is described elsewhere.21 The complete list of stakeholders is presented in the Acknowledgements section and full details are available in Table S2. First round The list of QIs was converted into an internet-based questionnaire (Figure S2) using SurveyMonkey® (Palo Alto, CA, USA). Respondents were asked to appraise the relevance of the indicators for assessing the quality of antibiotic use in the outpatient setting, using a nine-point Likert scale ranging from 1 (clearly not relevant) to 9 (clearly relevant). They also had a ‘cannot assess’ option, and a comments box for each QI. QIs were: (i) selected if the median score was 8 or 9 with agreement; (ii) held for further discussion if the median score was 8 or 9 without agreement; or (3) rejected if the median score was <8. Agreement was defined as >70% of the scores being in the upper tertile (score 7–9). Stakeholders could also suggest new QIs for further discussion. Consensus meeting Newly suggested QIs, selected QIs with many comments and those ‘held for further discussion’ (category 2) after the first round were discussed during a face-to-face meeting, to which the same stakeholders who participated in the first round were invited. The discussion was led by an expert moderator who was otherwise not part of the study team. Before the meeting, all stakeholders received a detailed summary of the survey results describing their individual replies as well as the group median scores for each suggested QI. During the meeting, stakeholders were asked to accept, reject or rephrase each QI under discussion to reach consensus. Moreover, they could rephrase the QIs that were already selected after the first round. Study investigators did not get directly involved in the discussion. Second round All previously accepted, added and rephrased QIs were presented in a second internet-based questionnaire and sent to all stakeholders who participated in the first round, together with a detailed report of the previous phases. Stakeholders were asked to definitively accept or reject each QI. A ‘cannot assess’ option was offered, as well as the possibility to add comments. Indicators were accepted if >70% of respondents agreed with their selection. Results Systematic review of the literature The literature search identified 3563 articles, of which 287 were considered eligible for full-text screening. After the application of inclusion and exclusion criteria, the screening of reference lists and the web site search, a final set of 54 articles and 7 web sites was retained (Figure 1). The majority of the papers (37/54, 69%) concerned high-income countries (Table S3).26–86 Figure 1. View largeDownload slide Flow chart of the systematic review of the literature. Figure 1. View largeDownload slide Flow chart of the systematic review of the literature. Overall, 356 QIs were extracted and 120 were retained after removing duplicates. Indicators addressing similar topics were grouped, producing a final list of 43 QIs (29 concerning general practice and 14 concerning OPAT), with 113 numerator–denominator combinations (Table S4). This list was presented to the stakeholders for the first round of the consensus procedure. RAND-modified Delphi procedure The first round was completed between August and September 2015. Forty-three stakeholders were contacted and 23 of them (from 12 countries: Belgium, Canada, Chile, Denmark, France, Nigeria, Spain, Sweden, the Netherlands, UK, USA and Vietnam) agreed to participate (Table 1 and Table S2). After the first round, 27 QIs were selected, 10 were rejected and 6 were held for further discussion. Two QIs were newly suggested (Figure 2 and Table S4). Table 1. Stakeholders participating in the RAND-modified Delphi consensus procedure Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) AMR, antimicrobial resistance; GP, general practice; ID, infectious diseases. Group A, medical community; Group B, public health and patients; Group C, antibiotic R&D; Group D, payers, policy makers, government and regulators. a These stakeholders participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study by Monnier et al.21 and Stanić Benić et al.23 They were provided with the results of the first survey on QIs for antibiotic use in the outpatient setting and took part in the consensus meeting. Table 1. Stakeholders participating in the RAND-modified Delphi consensus procedure Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) Phase of the consensus procedure/ characteristics Group A Group B Group C Group D Total First survey  Number (%) 11 (48) 3 (13) 6 (26) 3 (13) 23  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (6) agencies (1 ECDC, 1 CDC) ID (2) ethics (1) Ministry of Health (1) microbiology (4) patient society (1) pharmacology (2)  Place of work (n) Africa (1) Europe (3) Europe (3) Europe (2) Asia (1) North America (3) North America (1) Europe (6) North America (2) South America (1) Face-to-face meeting  Number (%) 4 (40) 1 (10) 3 (30) 2 (20) 10  Area of expertise (n) ID (2)a anti-AMR network (1) pharmaceutical industry (3) international societies (1 ECDC, 1 CDC) microbiology (1) pharmacology (1)  Place of work (n) Europe (2) Asia (1) Europe (1) Europe (1) Europe (1) North America (1) North America (2) North America (1) Second survey  Number (%) 11 (55) 2 (10) 5 (25) 2 (10) 20  Area of expertise (n) GP (3) anti-AMR network (1) pharmaceutical industry (5) international societies (1 ECDC, 1 CDC) ID (2) Ethics (1) microbiology (4) pharmacology (2)  Place of work (n) Africa (1) Europe (2) Europe (3) Europe (1) Asia (1) North America (2) North America (1) Europe (6) North America (2) South America (1) AMR, antimicrobial resistance; GP, general practice; ID, infectious diseases. Group A, medical community; Group B, public health and patients; Group C, antibiotic R&D; Group D, payers, policy makers, government and regulators. a These stakeholders participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study by Monnier et al.21 and Stanić Benić et al.23 They were provided with the results of the first survey on QIs for antibiotic use in the outpatient setting and took part in the consensus meeting. Figure 2. View largeDownload slide Flow chart of the RAND-modified Delphi consensus procedure. Figure 2. View largeDownload slide Flow chart of the RAND-modified Delphi consensus procedure. The face-to-face consensus meeting took place at Schiphol airport (the Netherlands) on 30 September 2015. Seven stakeholders were present and three participated by teleconference. Two of these stakeholders (from Greece and Slovenia) participated in the consensus procedure on QIs and quantity metrics for antibiotic use in the inpatient setting that was performed in parallel to our study in the framework of the DRIVE-AB project. They were provided with the results of the first survey and took part in the consensus discussion on outpatient QIs (OQIs) (Table 1 and Table S2). Thirty-three QIs (with 68 numerator-denominator combinations) were selected, with 6 QIs having been rephrased and 2 merged with other already existing QIs (Figure 2 and Table S4). The second round was carried out between the end of December 2015 and February 2016. Twenty stakeholders participated (Table 1). Thirty-two QIs (with 67 numerator–denominator combinations) were selected, with an average consensus score of 94%. One QI was rejected (Figure 2 and Table S4). Final set of OQIs The final set of 32 OQIs is presented in Table 2. Twenty QIs addressed general practice, 11 addressed OPAT and 1 addressed both. Full details with all numerator–denominator combinations are available in Table S5. Table 2. The final set of 32 consensually validated quality indicators assessing antibiotic use in the outpatient setting Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B a Category of the study design: A, consensus-based indicators; B, review-based indicators; C, guideline-based indicators. Table 2. The final set of 32 consensually validated quality indicators assessing antibiotic use in the outpatient setting Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B Outpatient quality indicator (OQI) Type of indicator References Study designa OQI-1 Antibiotics should be prescribed for (most) bacterial infections (e.g. acute pneumonia, urinary tract infections) process 26–30 A OQI-2 Antibiotics should not be prescribed for viral infections or (most) self-limiting bacterial infections (e.g. acute bronchitis, influenza, acute otitis media in children >2 years old) process 26,28–42 A, B, C OQI-3 Outpatients should receive antibiotic therapy compliant with guidelines; this includes, but is not limited to, indication, choice of the antibiotic, duration, dose and timing process 26–31,34,36,39,40,43,44,46–51 A, B, C OQI-4 Some antibiotics should be rarely prescribed process 59 B OQI-5 Acute upper respiratory infections and bronchitis should not be treated with antibiotics within the first 3 days, unless there is documented indication for treatment process 29,30,60,61 A, C OQI-6 Outpatients with acute tonsillitis/pharyngitis should undergo a group A streptococcal diagnostic test to decide whether or not they should receive antibiotics process 32 C OQI-7 Outpatients with an acute tonsillitis/pharyngitis and positive group A streptococcal diagnostic test should be treated with antibiotics process 29,30 A OQI-8 Antibiotics for an acute tonsillitis/pharyngitis should be withheld, discontinued or not prescribed if an outpatient presents a diagnostic test (rapid antigen test or throat culture) negative for group A streptococci process 40 B OQI-9 Prescribed antibiotics should be chosen from an essential list/formulary process 47,48,50–52,62–73 C OQI-10 Possible contraindications should be taken into account when antibiotics are prescribed process 79,80 A, C OQI-11 Antibiotics from the list of essential antibiotics should be available in health facilities that dispense antibiotics structure 47,48,50,51,62,67,70,81 A, C OQI-12 Key antibiotics should not be out of stock in health facilities that dispense antibiotics structure 50,51 C OQI-13 Antibiotics in stock should not be beyond the expiry date structure 50,81 A, C OQI-14 Antibiotics that are dispensed to outpatients should be adequately labelled (patient name, antibiotics name, when antibiotics should be taken) structure 47,48,50,51,65,67,70,82 C OQI-15 Antibiotics should be adequately conserved and handled in health facilities structure 50,51 C OQI-16 Health facilities should keep adequate records of dispensed key antibiotics structure 50 C OQI-17 A copy of the essential antibiotics list should be available in health facilities structure 47,48,50,62,67,70 C OQI-18 Standard antibiotic treatment guidelines should be available in health facilities structure 50 C OQI-19 Health facilities should have access to the Summary of Product Characteristics of prescribed antibiotics, written in a local language structure 47 C OQI-20 Antibiotics should not be sold without prescription structure 50 C OQI-21 Outpatients and OPAT patients with an antibiotic prescription should be educated on how to take it, on the dosage, on expected side effects, and on the natural history of the disease process 47,48,50,51,60,62,65,67,70,75,76,82–86 A, B, C OQI-22 The treatment plan should be agreed between the OPAT team and the referring clinician before start of treatment process 84,85 B OQI-23 All OPAT treatment plans should include dose, frequency of administration and duration of therapy process 84 B OQI-24 OPAT antibiotics should be correctly stored, prepared, reconstituted, dispensed and administered structure 84,86 B OQI-25 Administered doses of OPAT should be documented on a medication card process 84 B OQI-26 The first dose of a new antibiotic in an OPAT should be administered in a supervised setting process 84 B OQI-27 OPAT antibiotics should be regularly reviewed to optimize speed of intravenous-to-oral switch process 84 B OQI-28 Each OPAT centre should monitor quality indicators on OPAT antibiotics structure 84,86 B OQI-29 An expert in OPAT (physician, nurse, pharmacist) should work in each OPAT centre structure 86 B OQI-30 The OPAT plan should be communicated to the general practitioner at discharge structure 85 B OQI-31 The OPAT programme should be accredited or certified structure 86 B OQI-32 In an OPAT programme, clinical and/or microbiological outcomes, including treatment failure and adverse events (including Clostridium difficile infections), should be recorded outcome 84,86 B a Category of the study design: A, consensus-based indicators; B, review-based indicators; C, guideline-based indicators. Discussion Here we propose a set of 32 consensually validated QIs to assess antibiotic use in outpatients, targeting mostly general practice and OPAT, in both high-income and middle- to low-income settings. The selected QIs are purposefully generic. DRIVE-AB goals were to come up with generic metrics for responsible use to be used worldwide; stakeholders from a broad area of expertise (who probably all view ‘responsible’ differently), and not only general practitioners (GPs), were thus asked to rate the relevance of these OQIs defining responsible use, for a broad range of infections in a broad range of countries with the aim of ending up with broadly supported QIs, from a global perspective. This makes our consensus procedure very different from those performed in the original included studies, as in those studies the QIs were very specifically defined, fitting a specific setting and, most of the time, a specific patient group or disease, and involved experts representing that setting (mostly GPs). This list of QIs encompasses a wide range of clinical conditions, aiming at a comprehensive evaluation of antibiotic use. Only one outcome indicator (OQI-32, Table 2) was retained at the end of the consensus procedure. This is in line with the paucity of outcome indicators in other sets of QIs assessing antibiotic use.17,26,87 They represent the endpoint of a chain of factors and their evaluation only indirectly estimates the quality of care provided and they are thus potentially influenced by many concurrent factors. In contrast, structure and process indicators provide a direct evaluation of quality in health care and thus allow direct identification of targets for intervention. Our study is original and has several strengths. We performed a systematic review of QIs available in the literature, followed by a stepwise consensus procedure (RAND-modified Delphi procedure). This design has been previously applied to identify QIs in different health care domains, although the methodology varied significantly in published papers.88 The method we chose has been applied in several settings17,89,90 and complies with the recommendations suggested by Kötter et al.91 in their systematic review on guideline-based development of QIs, and by Boulkedid et al.88 in their systematic review on use and reporting of the Delphi method. We think it ensures a robust content (‘Does the measurement represent every element of the construct?’) and face validity (‘Does the measurement tool appear to measure what we want to measure?’) for the selected QIs. In the consensus procedure, we included stakeholders from different backgrounds, including medical professionals, patients’ representatives, R&D experts from the pharmaceutical industry, government, and relevant agencies such as the US CDC and the ECDC. Participants came from low-, middle- and high-income countries, from 12 countries and four different continents. There are, however, some limitations that need to be acknowledged. We only searched the MEDLINE database for our systematic review and we did not systematically explore the grey literature.92 However, we did screen relevant web sites and the reference lists of all included articles. Clinimetric properties of QIs were not assessed in our study, and need to be further validated in specific settings. Clinimetric properties are characteristics of indicators such as measurability (feasibility), applicability, acceptability, reliability, potential for improvement and sensitivity to change.10,89,93 Our objective was to develop a list of generic QIs that can be assessed and adapted to national specificities and guidelines. The most critical aspect for some of the selected QIs is probably measurability, since in several cases patients’ demographic data and clinical information are needed to reliably assess the numerator and the denominator defining the indicator. This information is difficult to obtain on a large scale in many countries.26,59 A few years ago, the European Surveillance of Antimicrobial Consumption Network (ESAC-Net) developed a set of so-called drug-specific quality indicators87 for antibiotic use in the outpatient setting. However, these indicators were based on a quantitative evaluation of antibiotic consumption, and were actually more quantity metrics rather than QIs, since they can be meaningfully interpreted only in terms of comparison between different settings and different periods of time. The QIs we selected reflect the quality of care, having significance per se, without the need for comparison and not requiring the setting of thresholds (such as ‘quinolones should not exceed 10% of total antibiotic prescriptions’), which can be somewhat arbitrary. Thus, we think that this set of QIs and those developed by the ESAC-Net should not be considered as mutually exclusive, since they approach the phenomenon of antibiotic prescription in different ways and with substantially different methodology, thus being able to provide complementary information. A systematic review of QIs in primary care has been recently published by Saust et al.94 However, their literature search was less extensive than our systematic review. Moreover, they did not use a consensus procedure to validate the identified indicators. Also, many of the indicators they included are based on a quantitative evaluation of antibiotic consumption, similar to those from the ESAC-Net.26,87 The WHO also provided detailed lists of QIs.47,50,81 The WHO indicators target low- to middle-income countries, but they are generic and not focused on antibiotic treatments in outpatients. We nevertheless derived a relevant number of structure and process QIs (16 combinations of numerators and denominators, grouped in 13 QIs; for details see Tables S4 and S5) from these WHO indicators. Our consensus procedure confirmed that these QIs have high face validity and that they could represent a useful tool to assess the quality of antibiotic prescriptions in low-to- middle-income countries. Regarding OPAT, we extracted QIs from two guidelines84,86 and a critical appraisal of evidence.85 To the best of our knowledge, there are no QIs for OPAT available in the literature, so we present here the first comprehensive set of validated QIs for OPAT. Outpatient parenteral antibiotic treatment is increasingly being used, mainly to optimize patient comfort and decrease risks and costs related to hospitalization,95 but current evidence to guide its implementation is limited85 and variability in practice is frequent.96 Thus, a consensually validated set of QIs could be of great value to evaluate OPAT programmes and set targets for improvement. Overall, our search identified a relatively low number of unique QIs targeting antibiotic use in the outpatient setting. Further investigation is needed to develop new QIs, including some focusing on diagnostic processes,94 using an adequate methodology.10,11,13 Assessing the relationship between process QIs and outcome indicators for appropriate antibiotic use (e.g. ecological consequences of antibiotic consumption) is also needed to further validate the QIs;97 this was not done for the QIs that we identified in our literature review. In conclusion, we listed a set of generic QIs intended to evaluate the quality of antibiotic use in the outpatient setting, in a wide range of clinical conditions, taking into account the perspectives of a wide range of stakeholders involved with antibiotics. These QIs can be used to measure the appropriateness of current antibiotic use at the individual, regional or supra-regional level. Thus they can be used to identify targets for improvement and to evaluate the effects of antibiotic stewardship interventions. Indeed, these are generic QIs that are globally applicable. These QIs need to be tested in practice, carefully evaluating their clinimetric properties in different settings and conditions. Acknowledgements Members of the DRIVE-AB WP1 group The authors are grateful to all stakeholders that participated in the consensus procedure for sharing their knowledge and insights on the subject: Bojana Beović, Luis Bavestrello, Jon Cohen, Pieter-Jan Cortoos, Harpal Dhillon, Richard Drew, Prabhavathi Fernandes, David Findlay, Ian Friedland, Ian Gould, Lauri Hicks, Christopher Longshaw, Carl Llor, Allison McGeer, Iruka N. Okeke, Diamantis Plachouras, Garyfallia Poulakou, John H. Rex, Thomas Tängdén, Laura Trolle Saust, Claudine Richon, France Roblot, Robert Vander Stichele, Marcel Verweij and Heiman Wertheim.  We thank the DRIVE-AB steering committee for critically reviewing the manuscript.  We thank also Sjaak de Gouw (Netherlands), who acted as moderator in the face-to-face meeting, and Elmie Peters and Rikie Deurenberg Vos (Netherlands) who helped with structuring the search strategy for the MEDLINE search.  The results of this study have been presented as a poster and as an oral presentation at the ‘Train the Trainer’ event at the 26th European Congress of Clinical Microbiology and Infectious Diseases (ECCMID 2016), which took take place in Amsterdam, Netherlands, from 9 to 12 April 2016. Funding This work was supported by the Innovative Medicines Initiative (IMI) Joint Undertaking (grant agreement no. 115618 - Driving re-investment in R&D and responsible antibiotic use—DRIVE-AB—www.drive-ab.eu). Resources are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007–2013) and European Federation of Pharmaceutical Industries and Associations (EFPIA) in-kind contribution. Transparency declarations I. C. G. reports an educational grant from Pfizer, outside the submitted work. The remaining authors have none to declare. This article forms part of a Supplement sponsored by DRIVE-AB. Supplementary data Figures S1 and S2 and Tables S1 to S5 are available as Supplementary data at JAC Online. References 1 Laxminarayan R , Duse A , Wattal C et al. Antibiotic resistance—the need for global solutions . Lancet Infect Dis 2013 ; 13 : 1057 – 98 . Google Scholar CrossRef Search ADS 2 Holmes AH , Moore LS , Sundsfjord A et al. Understanding the mechanisms and drivers of antimicrobial resistance . Lancet 2016 ; 387 : 176 – 87 . Google Scholar CrossRef Search ADS 3 European Centre for Disease Prevention and Control (ECDC) . Surveillance of Antimicrobial Consumption in Europe 2012. 2014 . http://ecdc.europa.eu/en/publications/Publications/antimicrobial-consumption-europe-esac-net-2012.pdf. 4 World Health Organization (WHO) . Community-Based Surveillance of Antimicrobial Use and Resistance in Resource-Constrained Settings. 2009 . http://apps.who.int/medicinedocs/documents/s16168e/s16168e.pdf? ua=1. 5 Adriaenssens N , Coenen S , Versporten A et al. European Surveillance of Antimicrobial Consumption (ESAC): quality appraisal of antibiotic use in Europe . J Antimicrob Chemother 2011 ; 66 : vi71 – 7 . Google Scholar CrossRef Search ADS 6 Fleming-Dutra KE , Hersh AL , Shapiro DJ et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011 . JAMA 2016 ; 315 : 1864 – 73 . Google Scholar CrossRef Search ADS 7 Spivak ES , Cosgrove SE , Srinivasan A. Measuring appropriate antimicrobial use: attempts at opening the black box . Clin Infect Dis 2016 ; 63 : 1639 – 44 . Google Scholar CrossRef Search ADS 8 Costelloe C , Metcalfe C , Lovering A et al. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis . BMJ 2010 ; 340 : c2096. Google Scholar CrossRef Search ADS 9 Bell BG , Schellevis F , Stobberingh E et al. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance . BMC Infect Dis 2014 ; 14 : 13. Google Scholar CrossRef Search ADS 10 Campbell S , Braspenning J , Hutchinson A et al. Research methods used in developing and applying quality indicators in primary care . Qual Saf Health Care 2002 ; 11 : 358 – 64 . Google Scholar CrossRef Search ADS 11 Pronovost PJ , Miller M , Wachter RM. The GAAP in quality measurement and reporting . JAMA 2007 ; 298 : 1800 – 2 . Google Scholar CrossRef Search ADS 12 DePestel DD , Eiland EH , Lusardi K et al. Assessing appropriateness of antimicrobial therapy: in the eye of the interpreter . Clin Infect Dis 2014 ; 59 : S154 – 61 . Google Scholar CrossRef Search ADS 13 Donabedian A. The quality of care. How can it be assessed? JAMA 1988 ; 260 : 1743 – 8 . Google Scholar CrossRef Search ADS 14 Lawrence M , Olesen F. Indicators of quality in health care . Eur J Gen Pract 1997 ; 3 : 103 – 8 . Google Scholar CrossRef Search ADS 15 Harbarth S , Theuretzbacher U , Hackett J et al. Antibiotic research and development: business as usual? J Antimicrob Chemother 2015 ; 70 : 1604 – 7 . Google Scholar CrossRef Search ADS 16 Fitch K , Bernstein SJ , Aguilar MD et al. The RAND/UCLA Appropriateness Method User’s Manual. 2001 . https://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf. 17 van den Bosch CM , Geerlings SE , Natsch S et al. Quality indicators to measure appropriate antibiotic use in hospitalized adults . Clin Infect Dis 2015 ; 60 : 281 – 91 . Google Scholar CrossRef Search ADS 18 Moher D , Liberati A , Tetzlaff J et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . BMJ 2009 ; 339 : b2535 . Google Scholar CrossRef Search ADS 19 Monnier AA , Eisenstein BI , Hulscher ME et al. Towards a global definition of responsible antibiotic use: results of an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi3 – vi16 . 20 Zanichelli V , Monnier AA , Gyssens IC et al. Variation in antibiotic use among and within different settings: a systematic review . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi17 – vi29 . 21 Monnier AA , Schouten J , Le Maréchal M et al. Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi30 – vi39 . 22 Versporten A , Gyssens IC , Pulcini C et al. Metrics to assess the quantity of antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi59 – vi66 . 23 Stanić Benić M , Milanič R , Monnier AA et al. Metrics for quantifying antibiotic use in the hospital setting: results from a systematic review and an international multidisciplinary consensus procedure . J Antimicrob Chemother 2018 ; 73 Suppl 6 : vi50 – vi58 . 24 Orphanet . The Portal for Rare Diseases and Orphan Drugs. http://www.orpha.net/consor/cgi-bin/index.php? lng=EN. 25 The World Bank . The World Bank. Data: Countries and Economies. http://data.worldbank.org/country. 26 Adriaenssens N , Coenen S , Tonkin-Crine S et al. European Surveillance of Antimicrobial Consumption (ESAC): disease-specific quality indicators for outpatient antibiotic prescribing . BMJ Qual Saf 2011 ; 20 : 764 – 72 . Google Scholar CrossRef Search ADS 27 Juhasz Z , Benko R , Matuz M et al. Treatment of acute cystitis in Hungary: comparison with national guidelines and with disease-specific quality indicators . Scand J Infect Dis 2013 ; 45 : 612 – 5 . Google Scholar CrossRef Search ADS 28 Adriaenssens N , Bartholomeeusen S , Ryckebosch P et al. Quality of antibiotic prescription during office hours and out-of-hours in Flemish primary care, using European quality indicators . Eur J Gen Pract 2014 ; 20 : 114 – 20 . Google Scholar CrossRef Search ADS 29 Hansen MP , Bjerrum L , Gahrn-Hansen B et al. Quality indicators for diagnosis and treatment of respiratory tract infections in general practice: a modified Delphi study . Scand J Prim Health Care 2010 ; 28 : 4 – 11 . Google Scholar CrossRef Search ADS 30 Hansen MP , Bjerrum L , Gahrn-Hansen B et al. Quality indicators for treatment of respiratory tract infections? An assessment by Danish general practitioners . Eur J Gen Pract 2013 ; 19 : 85 – 91 . Google Scholar CrossRef Search ADS 31 Linder JA , Schnipper JL , Tsurikova R et al. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a cluster randomised controlled trial . Inform Prim Care 2009 ; 17 : 231 – 40 . 32 DeVries A , Li CHW , Sridhar G et al. Impact of medical homes on quality, healthcare utilization, and costs . Am J Manag Care 2012 ; 18 : 534 – 44 . 33 Gonzales R , Anderer T , McCulloch CE et al. A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis . JAMA Intern Med 2013 ; 173 : 267 – 73 . Google Scholar CrossRef Search ADS 34 Kale MS , Bishop TF , Federman AD et al. Trends in the overuse of ambulatory health care services in the U.S . JAMA Intern Med 2013 ; 173 : 142 – 8 . Google Scholar CrossRef Search ADS 35 Miller G , Britt H , Pan Y et al. Relationship between general practitioner certification and characteristics of care . Med Care 2004 ; 42 : 770. Google Scholar CrossRef Search ADS 36 Ma J , Stafford RS. Quality of US outpatient care: temporal changes and racial/ethnic disparities . Arch Intern Med 2005 ; 165 : 1354 – 61 . Google Scholar CrossRef Search ADS 37 Mainous AG , Lambourne CA , Nietert PJ. Impact of a clinical decision support system on antibiotic prescribing for acute respiratory infections in primary care: quasi-experimental trial . J Am Med Inform Assoc 2013 ; 20 : 317 – 24 . Google Scholar CrossRef Search ADS 38 Gerber JS , Prasad PA , Fiks AG et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial . JAMA 2013 ; 309 : 2345 – 52 . Google Scholar CrossRef Search ADS 39 Van Roosmalen MS , Braspenning JC , De Smet PA et al. Antibiotic prescribing in primary care: first choice and restrictive prescribing are two different traits . Qual Saf Health Care 2007 ; 16 : 105 – 9 . Google Scholar CrossRef Search ADS 40 Bisno AL , Gerber MA , Gwaltney JM Jr et al. Diagnosis and management of group A streptococcal pharyngitis: a practice guideline . Clin Infect Dis 1997 ; 25 : 574 – 83 . Google Scholar CrossRef Search ADS 41 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Acute Otitis Externa (AOE). 2011 . https://www.qualitymeasures.ahrq.gov/summaries/summary/28014. 42 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Otitis Media with Effusion (OME). 2011 . https://www.qualitymeasures.ahrq.gov/summaries/summary/28018. 43 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Emergency Medicine. 2013 . https://www.qualitymeasures.ahrq.gov/. 44 National Quality Measures Clearinghouse . Agency for Healthcare Research and Quality. Community-Acquired Bacterial Pneumonia. 2006 . https://www.qualitymeasures.ahrq.gov/. 45 Linder JA , Ma J , Bates DW et al. Electronic health record use and the quality of ambulatory care in the United States . Arch Intern Med 2007 ; 167 : 1400 – 5 . Google Scholar CrossRef Search ADS 46 World Health Organization (WHO) . Guideline: Updates on the Management of Severe Acute Malnutrition in Infants and Children. 2013 . http://apps.who.int/iris/bitstream/10665/95584/1/9789241506328_eng.pdf. 47 World Health Organization (WHO) . How to Investigate Drug Use in Health Facilities. 1993 . http://apps.who.int/medicinedocs/pdf/s2289e/s2289e.pdf. 48 Hazra A , Tripathi SK , Alam MS. Prescribing and dispensing activities at the health facilities of a non-governmental organization . Natl Med J India 2000 ; 13 : 177 – 82 . 49 Wensing M , Broge B , Riens B et al. Quality circles to improve prescribing of primary care physicians. Three comparative studies . Pharmacoepidemiol Drug Saf 2009 ; 18 : 763. Google Scholar CrossRef Search ADS 50 World Health Organization (WHO) . Using Indicators to Measure Country Pharmaceutical Situations. Fact Book on WHO Level I and Level II Monitoring Indicators. 2006 . http://www.who.int/medicines/publications/WHOTCM2006.2A.pdf. 51 Cheraghali AM , Idries AM. Availability, affordability, and prescribing pattern of medicines in Sudan . Pharm World Sci 2009 ; 31 : 209 – 15 . Google Scholar CrossRef Search ADS 52 Ashworth M , Golding S , Majeed A. Prescribing indicators and their use by primary care groups to influence prescribing . J Clin Pharm Ther 2002 ; 27 : 197 – 204 . Google Scholar CrossRef Search ADS 53 Williams D , Bennett K , Feely J. The application of prescribing indicators to a primary care prescription database in Ireland . Eur J Clin Pharmacol 2005 ; 61 : 127 – 33 . Google Scholar CrossRef Search ADS 54 Campbell SM , Roland MO , Quayle JA et al. Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health 1998 ; 20 : 414 – 21 . Google Scholar CrossRef Search ADS 55 Wilcock M , Mackenzie IF. Quality indicators for general practice . J Public Health Med 1999 ; 21 : 357. Google Scholar CrossRef Search ADS 56 Campbell SM , Cantrill JA , Roberts D. Prescribing indicators for UK general practice: Delphi consultation study . BMJ 2000 ; 321 : 425. Google Scholar CrossRef Search ADS 57 Fernández Urrusuno R , Montero Balosa MC , Pérez Pérez P et al. Compliance with quality prescribing indicators in terms of their relationship to financial incentives . Eur J Clin Pharmacol 2013 ; 69 : 1845 – 53 . Google Scholar CrossRef Search ADS 58 Altiner A , Wilm S , Wegscheider K et al. Fluoroquinolones to treat uncomplicated acute cough in primary care: predictors for unjustified prescribing of antibiotics . J Antimicrob Chemother 2010 ; 65 : 1521 – 5 . Google Scholar CrossRef Search ADS 59 Pulcini C , Lions C , Ventelou B et al. Approaching the quality of antibiotic prescriptions in primary care using reimbursement data . Eur J Clin Microbiol Infect Dis 2013 ; 32 : 325 – 32 . Google Scholar CrossRef Search ADS 60 National Institute for Health and Care Excellence . Respiratory Tract Infections (Self-Limiting): Prescribing Antibiotics. 2008 . https://www.nice.org.uk/guidance/cg69/chapter/1-Guidance. 61 Wessell AM , Liszka HA , Nietert PJ et al. Achievable benchmarks of care for primary care quality indicators in a practice-based research network . Am J Med Qual 2008 ; 23 : 39 – 46 . Google Scholar CrossRef Search ADS 62 Guyon AB , Barman A , Ahmed JU et al. A baseline survey on use of drugs at the primary health care level in Bangladesh . Bull World Health Organ 1994 ; 72 : 265 – 71 . 63 Otoom S , Batieha A , Hadidi H et al. Evaluation of drug use in Jordan using WHO prescribing indicators . East Mediterr Health J 2002 ; 8 : 537 – 43 . 64 Naseeb TA , Nasser MA. Drug prescribing indicators in primary health care centers in Bahrain . Saudi Med J 2005 ; 26 : 1436 – 8 . 65 Nsimba SE. Assessing prescribing and patient care indicators for children under five years old with malaria and other disease conditions in public primary health care facilities . Southeast Asian J Trop Med Public Health 2006 ; 37 : 206 – 14 . 66 Shrestha N , Samir KC , Baltussen R et al. Practical approach to lung health in Nepal: better prescribing and reduction of cost . Trop Med Int Health 2006 ; 11 : 765 – 72 . Google Scholar CrossRef Search ADS 67 De Costa A , Bhartiya S , Eltayb A et al. Patterns of drug use in the public sector primary health centers of Bhopal district . Pharm World Sci 2008 ; 30 : 584 – 9 . Google Scholar CrossRef Search ADS 68 Chen W , Tang S , Sun J et al. Availability and use of essential medicines in China: manufacturing, supply, and prescribing in Shandong and Gansu provinces . BMC Health Serv Res 2010 ; 10 : 211. Google Scholar CrossRef Search ADS 69 Dong L , Yan H , Wang D. Drug prescribing indicators in village health clinics across 10 provinces of Western China . Fam Pract 2011 ; 28 : 63 – 7 . Google Scholar CrossRef Search ADS 70 Ahmed SM , Islam QS. Availability and rational use of drugs in primary healthcare facilities following the national drug policy of 1982: is Bangladesh on right track? J Health Popul Nutr 2012 ; 30 : 99 – 108 . 71 Desalegn AA. Assessment of drug use pattern using WHO prescribing indicators at Hawassa University Teaching and Referral Hospital, south Ethiopia: a cross-sectional study . BMC Health Serv Res 2013 ; 13 : 170. Google Scholar CrossRef Search ADS 72 Banerjee I , Bhadury T. Prescribing pattern of interns in a primary health center in India . J Basic Clin Pharm 2014 ; 5 : 40 – 3 . Google Scholar CrossRef Search ADS 73 Song Y , Bian Y , Petzold M et al. The impact of China’s national essential medicine system on improving rational drug use in primary health care facilities: an empirical study in four provinces . BMC Health Serv Res 2014 ; 14 : 507. Google Scholar CrossRef Search ADS 74 Moghadamnia AA , Mirbolooki MR , Aghili MB. General practitioner prescribing patterns in Babol city, Islamic Republic of Iran . East Mediterr Health J 2002 ; 8 : 550 – 5 . 75 Esmaily HM , Savage C , Vahidi R et al. Identifying outcome-based indicators and developing a curriculum for a continuing medical education programme on rational prescribing using a modified Delphi process . BMC Med Educ 2008 ; 8 : 33. Google Scholar CrossRef Search ADS 76 Esmaily HM , Silver I , Shiva S et al. Can rational prescribing be improved by an outcome-based educational approach? A randomized trial completed in Iran . J Contin Educ Health Prof 2010 ; 30 : 11 – 8 . Google Scholar CrossRef Search ADS 77 Otoom S , Culligan K , Al-Assoomi B et al. Analysis of drug prescriptions in primary health care centres in Bahrain . East Mediterr Health J 2010 ; 16 : 511 – 5 . Google Scholar CrossRef Search ADS 78 Rasool BKA , Fahmy SA , Abu-Gharbieh EF et al. Professional practices and perception towards rational use of medicines according to WHO methodology in United Arab Emirates . Pharm Pract 2010 ; 8 : 70 – 6 . 79 Avery AJ , Dex GM , Mulvaney C et al. Development of prescribing-safety indicators for GPs using the RAND Appropriateness Method . Br J Gen Pract 2011 ; 61 : e526 – 36 . Google Scholar CrossRef Search ADS 80 Irvine L , Flynn RW , Libby G et al. Drugs dispensed in primary care during pregnancy: a record-linkage analysis in Tayside, Scotland . Drug Saf 2010 ; 33 : 593 – 604 . Google Scholar CrossRef Search ADS 81 World Health Organization (WHO) . Indicators for Monitoring National Drug Policies. A Practical Manual. 1999 . http://apps.who.int/medicinedocs/pdf/whozip14e/whozip14e.pdf. 82 Awad AI , Himad HA. Drug-use practices in teaching hospitals of Khartoum State, Sudan . Eur J Clin Pharmacol 2006 ; 62 : 1087 – 93 . Google Scholar CrossRef Search ADS 83 Steel N , Melzer D , Shekelle P et al. Developing quality indicators for older adults: transfer from the USA to the UK is feasible . Qual Saf Health Care 2004 ; 13 : 260. Google Scholar CrossRef Search ADS 84 Chapman AL , Seaton RA , Cooper MA et al. Good practice recommendations for outpatient parenteral antimicrobial therapy (OPAT) in adults in the UK: a consensus statement . J Antimicrob Chemother 2012 ; 67 : 1053 – 62 . Google Scholar CrossRef Search ADS 85 Muldoon EG , Snydman DR , Penland EC et al. Are we ready for an outpatient parenteral antimicrobial therapy bundle? A critical appraisal of the evidence . Clin Infect Dis 2013 ; 57 : 419 – 24 . Google Scholar CrossRef Search ADS 86 Tice AD , Rehm SJ , Dalovisio JR et al. Practice guidelines for outpatient parenteral antimicrobial therapy . Clin Infect Dis 2004 ; 38 : 1651 – 71 . Google Scholar CrossRef Search ADS 87 Coenen S , Ferech M , Haaijer-Ruskamp FM et al. European Surveillance of Antimicrobial Consumption (ESAC): quality indicators for outpatient antibiotic use in Europe . Qual Saf Health Care 2007 ; 16 : 440 – 5 . Google Scholar CrossRef Search ADS 88 Boulkedid R , Abdoul H , Loustau M et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review . PLoS One 2011 ; 6 : e20476. Google Scholar CrossRef Search ADS 89 Hommel I , van Gurp PJ , Tack CJ et al. Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway . BMJ Qual Saf 2015 ; 25 : 525 – 34 . Google Scholar CrossRef Search ADS 90 Hermanides HS , Hulscher ME , Schouten JA et al. Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care . Clin Infect Dis 2008 ; 46 : 703 – 11 . Google Scholar CrossRef Search ADS 91 Kötter T , Blozik E , Scherer M. Methods for the guideline-based development of quality indicators—a systematic review . Implement Sci 2012 ; 7 : 21. Google Scholar CrossRef Search ADS 92 Mahood Q , Van Eerd D , Irvin E. Searching for grey literature for systematic reviews: challenges and benefits . Res Synth Methods 2014 ; 5 : 221 – 34 . Google Scholar CrossRef Search ADS 93 van den Bosch CM , Hulscher ME , Natsch S et al. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study . Clin Microbiol Infect 2016 ; 22 : 888.e1 – 888.e9 . Google Scholar CrossRef Search ADS 94 Saust LT , Monrad RN , Hansen MP et al. Quality assessment of diagnosis and antibiotic treatment of infectious diseases in primary care: a systematic review of quality indicators . Scand J Prim Health Care 2016 ; 34 : 258 – 66 . Google Scholar CrossRef Search ADS 95 Chapman AL , Dixon S , Andrews D et al. Clinical efficacy and cost-effectiveness of outpatient parenteral antibiotic therapy (OPAT): a UK perspective . J Antimicrob Chemother 2009 ; 64 : 1316 – 24 . Google Scholar CrossRef Search ADS 96 Muldoon EG , Switkowski K , Tice A et al. A national survey of infectious disease practitioners on their use of outpatient parenteral antimicrobial therapy (OPAT) . Infect Dis (Lond) 2015 ; 47 : 39 – 45 . Google Scholar CrossRef Search ADS 97 Schuts EC , Hulscher ME , Mouton JW et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis . Lancet Infect Dis 2016 ; 16 : 847 – 56 . Google Scholar CrossRef Search ADS © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

Journal

Journal of Antimicrobial ChemotherapyOxford University Press

Published: Jun 6, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off