Influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis—authors’ response

Influence of empirical double-active combination antimicrobial therapy compared with active... Sir, We thank Aho Glélé et al.1 for their interest in our paper2 and for their valuable comments. As the authors correctly pointed out in their letter, in order to avoid the limitations of using the propensity score as a covariate in the multivariate analysis, we also decided to perform a propensity score-matched analysis. Indeed, even after using both approaches we did not find a significant association between empirical double-active antimicrobial chemotherapy and survival. We therefore believe that the possible bias related to the use of the propensity score as a covariate was addressed by performing a propensity score-matched analysis. A true consensus regarding which variables should be included in a propensity score model (i.e. all variables, only the variables affecting treatment assignment, all variables affecting outcomes or all variables affecting both treatment assignment and outcome) is lacking. According to Monte Carlo simulation studies,3 reduction in bias when estimating a null treatment effect is similar for all four of the models mentioned above when propensity score matching is used. Omitting a confounder will increase bias, whereas including only covariates related to outcome or that are true confounders will improve precision. However, we are still confident in our propensity score model because, in spite of being constructed as a true treatment allocation model, it contained all the variables apparently associated with outcome but none of the variables that were strongly related to treatment and not to outcome. We are aware of the possible influence of the bacterial aetiology on the outcome of septic shock, and we therefore included the causative pathogens both in the logistic regression analysis using the propensity score as a covariate and in the propensity score-matched conditional logistic regression analysis. However, the bacterial aetiology was not apparently related to outcome, hence there was not a particular reason for including this variable in the propensity score. Concerning the use of multivariate models to estimate mortality, we purposely decided to assess the influence of baseline characteristics on very early (7 days) or early (30 days) mortality, as we expected the effect of antimicrobial chemotherapy on mortality to be more evident in the early stages of infection.4 Indeed, in a preliminary analysis of our unmatched cohort, we also performed a Cox proportional regression analysis with censoring at the 30 day follow-up. No major differences were evident when comparing these results with the 30 day logistic regression analysis, and double-active combination therapy still failed to be a significant predictor of mortality [adjusted HR 0.880 (95% CI 0.633–1.222)]. We are aware of the extensive criticism regarding automated stepwise methods in the statistical literature. However, we think they are still acceptable procedures provided that the set of original variables to start with suffices for unconfoundedness of treatment effect estimates.5 We mentioned the use of a forward stepwise algorithm as a variable selection procedure in the logistic regression analysis of the entire series and we acknowledge the reasons in favour of backward elimination over forward selection, especially when collinearity between predictors is present.6 However, when collinearity is low, agreement between these two procedures is frequent, and that was the case in our study. With regard to alternative procedures, we hope that further studies and expert discussions will open a clear and definitive path to deciding the best variable selection procedure for a given set of data. This will be greatly appreciated by the community of clinical investigators. Finally, the question of whether to perform adjustment for multiple testing is a long-lasting controversial issue frequently confronting statisticians and epidemiologists. In our work, we followed the line of thought expressed by several authors7,8 that such correction is unnecessary and even potentially deleterious by inflating type II error. In summary, we agree that propensity score methods are far from being a panacea for assessing treatment effects in observational studies and that prudence in reporting and interpreting the results of such studies is advisable. Transparency declarations None to declare. References 1 Aho Glélé LS , Guilloteau A , Blot M et al. Comment on: influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis . J Antimicrob Chemother 2018 ; 73 : 1731 – 2 . 2 Ripa M , Rodríguez-Núñez O , Cardozo C et al. Influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis . J Antimicrob Chemother 2017 ; 72 : 3443 – 52 . Google Scholar CrossRef Search ADS PubMed 3 Austin PC , Grootendorst P , Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study . Stat Med 2007 ; 26 : 734 – 53 . Google Scholar CrossRef Search ADS PubMed 4 Delano MJ , Ward PA. The immune system's role in sepsis progression, resolution, and long-term outcome . Immunol Rev 2016 ; 274 : 330 – 3 . Google Scholar CrossRef Search ADS PubMed 5 Sauer BC , Brookhart MA , Roy J et al. A review of covariate selection for non-experimental comparative effectiveness research . Pharmacoepidemiol Drug Saf 2013 ; 22 : 1139 – 45 . Google Scholar CrossRef Search ADS PubMed 6 Sauerbrei W , Royston P , Schumacher M. Comments on ‘Performance of using multiple stepwise algorithms for variable selection’ by Ryan E. Wiegand, Statistics in Medicine 2010; 29: 1647-1659 . Stat Med 2011 ; 30 : 892 – 4 . Google Scholar CrossRef Search ADS PubMed 7 Savitz DA , Olshan AF. Describing data requires no adjustment for multiple comparisons: a reply from Savitz and Olshan . Am J Epidemiol 1998 ; 147 : 813 – 4 . Google Scholar CrossRef Search ADS PubMed 8 Perneger TV. What's wrong with Bonferroni adjustments . BMJ 1998 ; 316 : 1236 – 8 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For Permissions, please email: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Antimicrobial Chemotherapy Oxford University Press

Influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis—authors’ response

Loading next page...
 
/lp/ou_press/influence-of-empirical-double-active-combination-antimicrobial-therapy-xp6JxbVWUb
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For Permissions, please email: journals.permissions@oup.com.
ISSN
0305-7453
eISSN
1460-2091
D.O.I.
10.1093/jac/dky063
Publisher site
See Article on Publisher Site

Abstract

Sir, We thank Aho Glélé et al.1 for their interest in our paper2 and for their valuable comments. As the authors correctly pointed out in their letter, in order to avoid the limitations of using the propensity score as a covariate in the multivariate analysis, we also decided to perform a propensity score-matched analysis. Indeed, even after using both approaches we did not find a significant association between empirical double-active antimicrobial chemotherapy and survival. We therefore believe that the possible bias related to the use of the propensity score as a covariate was addressed by performing a propensity score-matched analysis. A true consensus regarding which variables should be included in a propensity score model (i.e. all variables, only the variables affecting treatment assignment, all variables affecting outcomes or all variables affecting both treatment assignment and outcome) is lacking. According to Monte Carlo simulation studies,3 reduction in bias when estimating a null treatment effect is similar for all four of the models mentioned above when propensity score matching is used. Omitting a confounder will increase bias, whereas including only covariates related to outcome or that are true confounders will improve precision. However, we are still confident in our propensity score model because, in spite of being constructed as a true treatment allocation model, it contained all the variables apparently associated with outcome but none of the variables that were strongly related to treatment and not to outcome. We are aware of the possible influence of the bacterial aetiology on the outcome of septic shock, and we therefore included the causative pathogens both in the logistic regression analysis using the propensity score as a covariate and in the propensity score-matched conditional logistic regression analysis. However, the bacterial aetiology was not apparently related to outcome, hence there was not a particular reason for including this variable in the propensity score. Concerning the use of multivariate models to estimate mortality, we purposely decided to assess the influence of baseline characteristics on very early (7 days) or early (30 days) mortality, as we expected the effect of antimicrobial chemotherapy on mortality to be more evident in the early stages of infection.4 Indeed, in a preliminary analysis of our unmatched cohort, we also performed a Cox proportional regression analysis with censoring at the 30 day follow-up. No major differences were evident when comparing these results with the 30 day logistic regression analysis, and double-active combination therapy still failed to be a significant predictor of mortality [adjusted HR 0.880 (95% CI 0.633–1.222)]. We are aware of the extensive criticism regarding automated stepwise methods in the statistical literature. However, we think they are still acceptable procedures provided that the set of original variables to start with suffices for unconfoundedness of treatment effect estimates.5 We mentioned the use of a forward stepwise algorithm as a variable selection procedure in the logistic regression analysis of the entire series and we acknowledge the reasons in favour of backward elimination over forward selection, especially when collinearity between predictors is present.6 However, when collinearity is low, agreement between these two procedures is frequent, and that was the case in our study. With regard to alternative procedures, we hope that further studies and expert discussions will open a clear and definitive path to deciding the best variable selection procedure for a given set of data. This will be greatly appreciated by the community of clinical investigators. Finally, the question of whether to perform adjustment for multiple testing is a long-lasting controversial issue frequently confronting statisticians and epidemiologists. In our work, we followed the line of thought expressed by several authors7,8 that such correction is unnecessary and even potentially deleterious by inflating type II error. In summary, we agree that propensity score methods are far from being a panacea for assessing treatment effects in observational studies and that prudence in reporting and interpreting the results of such studies is advisable. Transparency declarations None to declare. References 1 Aho Glélé LS , Guilloteau A , Blot M et al. Comment on: influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis . J Antimicrob Chemother 2018 ; 73 : 1731 – 2 . 2 Ripa M , Rodríguez-Núñez O , Cardozo C et al. Influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis . J Antimicrob Chemother 2017 ; 72 : 3443 – 52 . Google Scholar CrossRef Search ADS PubMed 3 Austin PC , Grootendorst P , Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study . Stat Med 2007 ; 26 : 734 – 53 . Google Scholar CrossRef Search ADS PubMed 4 Delano MJ , Ward PA. The immune system's role in sepsis progression, resolution, and long-term outcome . Immunol Rev 2016 ; 274 : 330 – 3 . Google Scholar CrossRef Search ADS PubMed 5 Sauer BC , Brookhart MA , Roy J et al. A review of covariate selection for non-experimental comparative effectiveness research . Pharmacoepidemiol Drug Saf 2013 ; 22 : 1139 – 45 . Google Scholar CrossRef Search ADS PubMed 6 Sauerbrei W , Royston P , Schumacher M. Comments on ‘Performance of using multiple stepwise algorithms for variable selection’ by Ryan E. Wiegand, Statistics in Medicine 2010; 29: 1647-1659 . Stat Med 2011 ; 30 : 892 – 4 . Google Scholar CrossRef Search ADS PubMed 7 Savitz DA , Olshan AF. Describing data requires no adjustment for multiple comparisons: a reply from Savitz and Olshan . Am J Epidemiol 1998 ; 147 : 813 – 4 . Google Scholar CrossRef Search ADS PubMed 8 Perneger TV. What's wrong with Bonferroni adjustments . BMJ 1998 ; 316 : 1236 – 8 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For Permissions, please email: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Journal of Antimicrobial ChemotherapyOxford University Press

Published: Feb 28, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off