Sir, We read with interest the work of Ripa et al.,1 which evaluated the influence on mortality of empirical double-active combination antimicrobial therapy, compared with active monotherapy in septic shock patients. The authors performed a retrospective cohort analysis of 576 patients in a single centre. They found a significant protective effect on mortality at some timepoints within the 30 day period after bacteraemia onset in patients with neutropenia or those with infection due to Pseudomonas aeruginosa. We commend the authors for performing this interesting study as these results would be useful for balanced treatment decision planning in clinical practice. However, we have several statistical suggestions and queries that we would like to communicate with the authors. First, the authors use propensity score as a covariate. Although this approach is widely mentioned, it is not considered a best practice in propensity score methods. By including propensity score as a covariate in a model, one cannot take full advantage of the propensity score’s features. Covariate adjustment does not allow for balancing of covariates across treated and control groups as well as could be achieved with matching (also performed by the authors) or weighting. Selection bias is therefore much observed in comparison with the other methods using propensity scores.2 Covariate adjustment is also sensitive to distributional assumptions2 and this may lead to inefficient estimates. Moreover, this method assumes that the nature of the relationship between the propensity score and the outcome has been correctly modelled.3 A non-parsimonious logistic regression model was used to estimate the propensity score. A large number of factors that could potentially affect the decision to use combination therapy were included, but not the genus of bacteria. Covariates associated with the outcome should be included in the propensity score regardless of their association with exposure, and covariates strongly associated with exposure and unassociated or only weakly associated with the outcome should be avoided, as these covariates can increase the variance and bias of effect estimates.4 Second, the authors performed a multivariate model and mortality was estimated using a logistic model. This approach may be used for short follow-up times, which is the case here. A Cox’s survival model or another survival model such as a flexible parametric survival model5 may be useful as it has the advantage, in our case, of directly estimating mortality at 7, 15 and 30 days. A survival model may be more appropriate to obtain more precise estimates.6 Third, automatic variable selection was used with this model. This approach is controversial7 and, when used, backward is preferable to forward. Alternative variable selection methods are available, e.g. penalized regression methods, including the lasso,8 the elastic net9 and their extensions. Finally, the authors conducted subgroup analyses, on subgroups defined a priori. There are, however, many subgroups that result in multiple comparisons. When multiple subgroup analyses are performed, the probability of a false positive finding can be substantial (effect on type I errors). To limit bias, formal adjustments for multiplicity should have been performed.10 In conclusion, the results of this study are interesting, but readers should interpret them with caution. Propensity score methods are not necessarily superior to conventional covariate adjustment.11 Transparency declarations None to declare. References 1 Ripa M , Rodríguez-Núñez O , Cardozo C et al. Influence of empirical double-active combination antimicrobial therapy compared with active monotherapy on mortality in patients with septic shock: a propensity score-adjusted and matched analysis . J Antimicrob Chemother 2017 ; 72 : 3443 – 52 . Google Scholar CrossRef Search ADS PubMed 2 Austin PC. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies . Med Decis Making 2009 ; 29 : 661 – 77 . Google Scholar CrossRef Search ADS PubMed 3 Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies . Multivariate Behav Res 2011 ; 46 : 399 – 424 . Google Scholar CrossRef Search ADS PubMed 4 Brookhart MA , Schneeweiss S , Rothman KJ et al. Variable selection for propensity score models . Am J Epidemiol 2006 ; 163 : 1149 – 56 . Google Scholar CrossRef Search ADS PubMed 5 Royston P. Flexible Parametric Survival Analysis Using Stata: Beyond the Cox Model . College Station, TX, USA : Stata Press , 2011 . 6 Callas PW , Pastides H , Hosmer DW. Empirical comparisons of proportional hazards, Poisson, and logistic regression modeling of occupational cohort data . Am J Ind Med 1998 ; 33 : 33 – 47 . Google Scholar CrossRef Search ADS PubMed 7 Wiegand RE. Performance of using multiple stepwise algorithms for variable selection . Stat Med 2010 ; 29 : 1647 – 59 . Google Scholar PubMed 8 Tibshirani R. Regression shrinkage and selection via the lasso . J R Stat Soc Ser B Methodol 1996 ; 58 : 267 – 88 . 9 Zou H , Hastie T. Regularization and variable selection via the elastic net . J R Stat Soc Ser B Methodol 2005 ; 67 : 301 – 20 . Google Scholar CrossRef Search ADS 10 Wang R , Lagakos SW , Ware JH et al. Statistics in medicine—reporting of subgroup analyses in clinical trials . N Engl J Med 2007 ; 357 : 2189 – 94 . Google Scholar CrossRef Search ADS PubMed 11 Elze MC , Gregson J , Baber U et al. Comparison of propensity score methods and covariate adjustment: evaluation in 4 cardiovascular studies . J Am Coll Cardiol 2017 ; 69 : 345 – 57 . Google Scholar CrossRef Search ADS PubMed © The Author(s) 2018. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For Permissions, please email: firstname.lastname@example.org. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
Journal of Antimicrobial Chemotherapy – Oxford University Press
Published: Feb 16, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera