Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs

The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial... Invited Commentary | Infectious Diseases The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs Michael Rosenblum, PhD; Bingkai Wang, BS The proposed randomized clinical trial design by Stankiewicz Karita et al addresses the important Related article question of whether the human papillomavirus (HPV) vaccine Gardasil-9 reduces recurrence of anal Author affiliations and article information are and vulvar high-grade squamous intraepithelial lesions (HSIL). The target population is unvaccinated listed at the end of this article. adults with a history of anal or vulvar HSIL. The trial is generally well designed to achieve the study objective. We first describe a potential improvement to the trial design, which involves the primary statistical analysis. The protocol’s primary analysis involves using a Cox proportional hazards model with time to recurrence as the outcome. No baseline variables are adjusted for in the primary analysis. We recommend that the primary analysis be changed to adjust for the 3 baseline variables (called covariates) used in the covariate-adaptive randomization procedure. Such adjustment is generally recommended in randomized trials that use stratified randomization or covariate-adaptive 2-4 randomization. One of the main reasons for such adjustment is to improve power. The authors are correct that using covariate-adaptive randomization can lead to greater power. However, the power gains are only realized if the reduced estimator variance due to covariate-adaptive randomization is accounted for in the analysis. Covariate adjustment can be used to obtain an accurate estimate of this variance, which could be overestimated (leading to power loss) if the covariates were ignored in the analysis. In brief, not adjusting for the covariates would effectively throw away any power gain from covariate-adaptive randomization; it could also result in confidence intervals that are wider than necessary. There are multiple statistical methods available to adjust for baseline covariates in 5-7 5 randomized trials with time-to-event outcomes. The method of Díaz et al avoids making the proportional hazards assumption by focusing on estimation of the restricted mean survival time, ie, the average survival time truncated at a final point such as 36 months. All of the aforementioned methods also adjust for missing data due to loss to follow-up. It is possible to include other baseline variables, in addition to the ones used in the covariate- adaptive randomization, in the primary analysis. This may lead to improved precision and power, as long as the new variables explain some of the residual variation that is left after adjustment for the variables in the covariate-adaptive randomization. Previous data sets and clinical knowledge could be used to identify which (if any) variables are likely to add substantial value. An important consideration is to avoid using too many baseline variables because this may lead to poor estimator performance; this is of special concern at smaller sample sizes such as when subpopulations are analyzed. All aspects of the primary analysis need to be prespecified in the study protocol, including the set of variables to be adjusted for and the statistical method to be used. The trial goals include estimation of treatment effects not only in the combined population but also in the HIV-positive and HIV-negative subpopulations. We recommend that analyses of these subpopulations be conducted using the same approach we describe except with analyses done separately for each subpopulation. The variables adjusted for could be the same as for the combined population (as long as there are not too many variables compared with the sample sizes of the subpopulations), except excluding the indicator of baseline HIV status (as the analyses would be done separately using data from HIV-positive and HIV-negative participants, respectively). Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2019;2(4):e190789. doi:10.1001/jamanetworkopen.2019.0789 (Reprinted) April 12, 2019 1/2 JAMA Network Open | Infectious Diseases Using Statistical Analyses to Maximize Power Gains From Adaptive Trial Designs It may be advantageous to have the trial be event driven, ie, to let follow-up continue until the total number of events (pooled across study groups) exceeds a predefined threshold (chosen to ensure the desired power at the minimum, clinically meaningful treatment effect size) or a maximum time is reached. This can help to ensure a correctly powered trial, as long as the required number of events occurs before the maximum time is reached. A final recommendation is that the intention-to-treat analysis set be defined as all randomized participants (or all randomized participants who receive1 dose of treatment or placebo). Currently, the intention-to-treat analysis set is defined as all randomized participants for whom events accrue starting at day 30 after the first vaccine dose (section 16.2 of the protocol, version 7.0.) This could lead to bias if participants drop out during the first 30 days after the first treatment or placebo dose owing to, eg, adverse effects caused by treatment. ARTICLE INFORMATION Published: April 12, 2019. doi:10.1001/jamanetworkopen.2019.0789 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Rosenblum Metal. JAMA Network Open. Corresponding Author: Michael Rosenblum, PhD, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Baltimore, MD 21205 (mrosen@jhu.edu). Author Affiliations: Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland. Conflict of Interest Disclosures: None reported. REFERENCES 1. Stankiewicz Karita HC, Hauge K, Magaret A, et al. Effect of human papillomavirus vaccine to interrupt recurrence of vulvar and anal neoplasia (VIVA): a trial protocol. JAMA Netw Open. 2019;2(4):e190819. doi:10.1001/ jamanetworkopen.2019.0819 2. European Medicines Agency. Guideline on adjustment for baseline covariates in clinical trials. https://www.ema. europa.eu/en/documents/scientific-guideline/guideline-adjustment-baseline-covariates-clinical-trials_en.pdf. Published February 26, 2015. Accessed February 25, 2019. 3. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. E9: Statistical principles for clinical trials. https://www.ich.org/fileadmin/Public_Web_Site/ICH_ Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf. Published February 5, 1998. Accessed February 25, 2019. 4. Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2012;31(4):328-340. doi:10.1002/sim.4431 5. Díaz I, Colantuoni E, Hanley DF, Rosenblum M. Improved precision in the analysis of randomized trials with survival outcomes, without assuming proportional hazards [published online February 28, 2018]. Lifetime Data Anal. doi:10.1007/s10985-018-9428-5 6. Lu X, Tsiatis AA. Semiparametric estimation of treatment effect with time-lagged response in the presence of informative censoring. Lifetime Data Anal. 2011;17(4):566-593. doi:10.1007/s10985-011-9199-8 7. Lu X, Tsiatis AA. Improving the efficiency of the log-rank test using auxiliary covariates. Biometrika. 2008;95 (3):679-694. doi:10.1093/biomet/asn003 JAMA Network Open. 2019;2(4):e190789. doi:10.1001/jamanetworkopen.2019.0789 (Reprinted) April 12, 2019 2/2 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JAMA Network Open American Medical Association

The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs

Rosenblum, Michael; Wang, Bingkai
JAMA Network Open , Volume 2 (4) – Apr 12, 2019
Download PDF
2 pages

Loading next page...
 
/lp/american-medical-association/the-critical-role-of-statistical-analyses-in-maximizing-power-gains-h1yPwr4wCQ

References (4)

Publisher
American Medical Association
Copyright
Copyright 2019 Rosenblum M et al. JAMA Network Open.
eISSN
2574-3805
DOI
10.1001/jamanetworkopen.2019.0789
Publisher site
See Article on Publisher Site

Abstract

Invited Commentary | Infectious Diseases The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs Michael Rosenblum, PhD; Bingkai Wang, BS The proposed randomized clinical trial design by Stankiewicz Karita et al addresses the important Related article question of whether the human papillomavirus (HPV) vaccine Gardasil-9 reduces recurrence of anal Author affiliations and article information are and vulvar high-grade squamous intraepithelial lesions (HSIL). The target population is unvaccinated listed at the end of this article. adults with a history of anal or vulvar HSIL. The trial is generally well designed to achieve the study objective. We first describe a potential improvement to the trial design, which involves the primary statistical analysis. The protocol’s primary analysis involves using a Cox proportional hazards model with time to recurrence as the outcome. No baseline variables are adjusted for in the primary analysis. We recommend that the primary analysis be changed to adjust for the 3 baseline variables (called covariates) used in the covariate-adaptive randomization procedure. Such adjustment is generally recommended in randomized trials that use stratified randomization or covariate-adaptive 2-4 randomization. One of the main reasons for such adjustment is to improve power. The authors are correct that using covariate-adaptive randomization can lead to greater power. However, the power gains are only realized if the reduced estimator variance due to covariate-adaptive randomization is accounted for in the analysis. Covariate adjustment can be used to obtain an accurate estimate of this variance, which could be overestimated (leading to power loss) if the covariates were ignored in the analysis. In brief, not adjusting for the covariates would effectively throw away any power gain from covariate-adaptive randomization; it could also result in confidence intervals that are wider than necessary. There are multiple statistical methods available to adjust for baseline covariates in 5-7 5 randomized trials with time-to-event outcomes. The method of Díaz et al avoids making the proportional hazards assumption by focusing on estimation of the restricted mean survival time, ie, the average survival time truncated at a final point such as 36 months. All of the aforementioned methods also adjust for missing data due to loss to follow-up. It is possible to include other baseline variables, in addition to the ones used in the covariate- adaptive randomization, in the primary analysis. This may lead to improved precision and power, as long as the new variables explain some of the residual variation that is left after adjustment for the variables in the covariate-adaptive randomization. Previous data sets and clinical knowledge could be used to identify which (if any) variables are likely to add substantial value. An important consideration is to avoid using too many baseline variables because this may lead to poor estimator performance; this is of special concern at smaller sample sizes such as when subpopulations are analyzed. All aspects of the primary analysis need to be prespecified in the study protocol, including the set of variables to be adjusted for and the statistical method to be used. The trial goals include estimation of treatment effects not only in the combined population but also in the HIV-positive and HIV-negative subpopulations. We recommend that analyses of these subpopulations be conducted using the same approach we describe except with analyses done separately for each subpopulation. The variables adjusted for could be the same as for the combined population (as long as there are not too many variables compared with the sample sizes of the subpopulations), except excluding the indicator of baseline HIV status (as the analyses would be done separately using data from HIV-positive and HIV-negative participants, respectively). Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2019;2(4):e190789. doi:10.1001/jamanetworkopen.2019.0789 (Reprinted) April 12, 2019 1/2 JAMA Network Open | Infectious Diseases Using Statistical Analyses to Maximize Power Gains From Adaptive Trial Designs It may be advantageous to have the trial be event driven, ie, to let follow-up continue until the total number of events (pooled across study groups) exceeds a predefined threshold (chosen to ensure the desired power at the minimum, clinically meaningful treatment effect size) or a maximum time is reached. This can help to ensure a correctly powered trial, as long as the required number of events occurs before the maximum time is reached. A final recommendation is that the intention-to-treat analysis set be defined as all randomized participants (or all randomized participants who receive1 dose of treatment or placebo). Currently, the intention-to-treat analysis set is defined as all randomized participants for whom events accrue starting at day 30 after the first vaccine dose (section 16.2 of the protocol, version 7.0.) This could lead to bias if participants drop out during the first 30 days after the first treatment or placebo dose owing to, eg, adverse effects caused by treatment. ARTICLE INFORMATION Published: April 12, 2019. doi:10.1001/jamanetworkopen.2019.0789 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Rosenblum Metal. JAMA Network Open. Corresponding Author: Michael Rosenblum, PhD, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Baltimore, MD 21205 (mrosen@jhu.edu). Author Affiliations: Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland. Conflict of Interest Disclosures: None reported. REFERENCES 1. Stankiewicz Karita HC, Hauge K, Magaret A, et al. Effect of human papillomavirus vaccine to interrupt recurrence of vulvar and anal neoplasia (VIVA): a trial protocol. JAMA Netw Open. 2019;2(4):e190819. doi:10.1001/ jamanetworkopen.2019.0819 2. European Medicines Agency. Guideline on adjustment for baseline covariates in clinical trials. https://www.ema. europa.eu/en/documents/scientific-guideline/guideline-adjustment-baseline-covariates-clinical-trials_en.pdf. Published February 26, 2015. Accessed February 25, 2019. 3. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. E9: Statistical principles for clinical trials. https://www.ich.org/fileadmin/Public_Web_Site/ICH_ Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf. Published February 5, 1998. Accessed February 25, 2019. 4. Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2012;31(4):328-340. doi:10.1002/sim.4431 5. Díaz I, Colantuoni E, Hanley DF, Rosenblum M. Improved precision in the analysis of randomized trials with survival outcomes, without assuming proportional hazards [published online February 28, 2018]. Lifetime Data Anal. doi:10.1007/s10985-018-9428-5 6. Lu X, Tsiatis AA. Semiparametric estimation of treatment effect with time-lagged response in the presence of informative censoring. Lifetime Data Anal. 2011;17(4):566-593. doi:10.1007/s10985-011-9199-8 7. Lu X, Tsiatis AA. Improving the efficiency of the log-rank test using auxiliary covariates. Biometrika. 2008;95 (3):679-694. doi:10.1093/biomet/asn003 JAMA Network Open. 2019;2(4):e190789. doi:10.1001/jamanetworkopen.2019.0789 (Reprinted) April 12, 2019 2/2

Journal

JAMA Network Open – American Medical Association

Published: Apr 12, 2019

There are no references for this article.