Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You and Your Team.

Learn More →

Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies

Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Key Points Question Can the duration of phase 1 IMPORTANCE Phase 1 cancer studies, which guide dose selection for subsequent studies, are studies using either the 3 + 3 or rolling 6 almost 3 times more prevalent than phase 3 studies and have a median study duration considerably designs be substantially reduced longer than 2 years, which constitutes a major component of drug development time. without exceeding the patient risk limits or changing the operating OBJECTIVE To discern a method to reduce the duration of phase 1 studies in adult and pediatric characteristics of the parent design? cancer studies without violating risk limits by better accommodating the accrual and evaluation Findings This decision analytical model process (or queue). found that the modified study designs were associated with reduced expected DESIGN The process modeled, the phase 1 queue (IQ), includes patient interarrival time, screening, study durations. The modified designs and dose-limiting toxicity evaluation. For this proof of principle, the rules of the 3 + 3 and rolling 6 were associated with minimal changes phase 1 designs were modified to improve patient flow through the queue without exceeding the in the number of patients treated and maximum risk permitted in the parent designs. The resulting designs, the IQ 3 + 3 and the IQ rolling the determination of the maximum 6, were each compared with their parent design by simulations in 12 different scenarios. tolerated dose, without changing the operating characteristics or exceeding MAIN OUTCOMES AND MEASURES (1) The time from study opening to determination of the the risk limits of the parent design. maximum tolerated dose (MTD), (2) the number of patients treated to determine the MTD, and (3) the association of the design with the dose selected as the MTD. Meaning Per this analysis, a substantial reduction in the time required to bring RESULTS Based on 800 simulations, for all 12 scenarios considered, the IQ 3 + 3 and the IQ rolling 6 new advances to the clinic can be designs were associated with reduced expected study durations compared with the parent design. accomplished by simple modifications The expected IQ 3 + 3 reduction ranged from 1.6 to 10.4 months (with 3.7 months for the standard of 2 commonly used phase 1 scenario), and the expected reduction associated with IQ rolling 6 ranged from 0.4 to 10.5 months trial designs. (with 3.4 months for the standard scenario). The increase in the mean number of patients treated in the IQ 3 + 3 compared with the 3 + 3 ranged from 0.6 to 3.2 patients. No increase in the number of Supplemental content patients was associated with the IQ rolling 6 compared with the rolling 6 design. The probability of selecting a dose level as the MTD changed by less than 3% for all dose levels and scenarios in both Author affiliations and article information are listed at the end of this article. parent designs. CONCLUSIONS AND RELEVANCE This study found that IQ designs were associated with reduced mean duration of phase 1 studies compared with their parent designs without changing the risk limits or MTD selection operating characteristics. These approaches have been successfully implemented in both hematology and solid tumor phase 1 studies. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 Introduction 1,2 While clinical trial timeliness factors into cost and relevance, phase 1 oncology trials are often underappreciated for their association with the time to translate a therapeutic advance into practice. Unlike most phase 1 studies, which take less than 1 year, phase 1 oncology studies take a median of approximately 32 months after study activation, and the number of phase 1 cancer clinical trials Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 1/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies is consistently almost 3 times the number of phase 3 studies (eTable 1 in the Supplement). In addition, while multiple phase 2 and 3 studies can run in parallel, they must wait for the relevant phase 1 study to complete. Because phase 1 study duration is often slot limited and minimally affected by adding sites, we must look elsewhere for opportunities to alter this early bottleneck that delays therapeutic advances in oncology. Here, we focus on the patient queue and evaluation process to reduce phase 1 study duration. As a proof of principle, we focus on the traditional 3 + 3 phase 1 design and the rolling 6 design often used in pediatric studies. To our knowledge, this represents the most beneficial modification of 2 of the most commonly used designs (1 for adults and 1 for pediatric populations) to reduce expected study duration without affected the operating characteristics. These new designs have been successfully implemented in several completed and ongoing clinical trials. Methods Phase 1 oncology designs typically consider only the first cycle of experimental treatment in both the determination of dose-limiting toxicities (DLTs) and the formal guidelines for dose-escalation decisions. To limit the number of patients at risk for a DLT, most phase 1 designs restrict the number of patients enrolled on the current dose level during the first cycle of experimental therapy. This is our starting point for the study of the phase 1 queue. The activities performed for the purposes of this simulation study are not considered to be human subjects research (per US Department of Health and Human Services regulation under the 45 CFR 46 Common Rule). Figure 1 illustrates the major steps involved in a phase 1 trial that form the basis for the patient queue that are reflected in the simulation tool. After each patient provides consent or any change occurs in any patient’s evaluation status, the protocol team decides the dose level and availability of slots for the accrual of additional patients based on the accumulated data and the protocol- specified phase 1 design. The decision can be to escalate (accrue at the next higher dose level), accrue at the same dose level, deescalate (accrue at the next lower dose level), hold accrual, or end accrual and either declare a maximum tolerated dose (MTD) or declare that the lowest dose level tested is too toxic. The phase 1 design guides these decisions. We modified the 3 + 3 design and rolling 6 design decisions to better accommodate the phase 1 queue. The modifications were evaluated on the time and number of patients required to determine the MTD while constrained to (1) inherit the maximum level of patient risk associated with the parent design, (2) provide at least an equally rigorous assessment of toxicity, (3) maintain the parent design’s operating characteristics with respect to the MTD determination, and (4) revert to the parent design if accrual is slow or the principal investigator chooses to delay securing consent from patients when the queue-based designs permit accrual but the parent design does not. 7,8 The respective queue-based modifications (Table 1 and Table 2) are denoted the IQ 3 + 3 and the IQ rolling 6 designs. At any time, on the current dose level, there is the total number of patients enrolled (eg, promised or taking a slot, excluding those deemed inevaluable for DLT determination), the number of DLTs, and the number of patients evaluable. The number evaluable represents patients who were fully assessed and either had a DLT or did not (termed a pass), whereas the difference between total and evaluable numbers represents patients whose evaluations are pending. The column for the 3 + 3 design can been seen as representing the traditional rules: accrue 3 patients, escalate with 0 DLTs, deescalate with 2 DLTs, and expand to 6 patients with 1 DLT (in which 2 or more DLTs results in dose deescalation and 1 DLT in 6 patients results in dose escalation if the next higher dose is open, and the MTD requires 6 patients treated with 1 DLT at most). Both IQ-based modifications can have up to 8 patients per dose level during the escalation phase. In addition, when deescalating, applying the rule that all patients (up to as many as 10 patients, which occurs in <0.1% of simulations) should not be denied treatment once given a consent form are possible in both IQ designs. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 2/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Figure 1. Representations of Patient Flow Through a Typical Phase 1 Trial A Phase 1 clinical trials simulation: process flow diagram Patient arrives (patients arrive as Screening Screen failure, Consent possible test Yes Yes Is a slot Screen patient exits (screening (patient is accepted candidates per available? failure? duration sampled (patient to leave as a candidate) a stochastic from distribution) model) interarrival time distribution) No No Patient waits Start test Determine (patient waits for (patient to disposition a maximum available slot for Yes tolerable wait current dose level) time sampled from a waiting distribution) Inevaluable DLT Pass Willing to Delay until Delay until Delay until wait further inevaluable DLT test complete No Inevaluable event, DLT event Test complete patient exits (patient stays in (patient stays in Patient exits slot) slot) (patient to leave (patient removed slot and model) from wait list and leaves model) B Screenshot of simulation software reflecting patient flow A, The process accounts for 3 outcomes. Patients who receive protocol-specified deescalation of the rolling 6 design and queue-based modified designs, so all consenting minimum therapy without a dose-limiting toxicity (DLT) pass. Patients with a DLT are patients are treated without delay. Patients go to the waitlist or screening (yellow). If designated DLT. Inevaluable denotes ones who do not meet criteria for pass or DLT, often they exceed maximum wait time or do not meet criteria, they exit. Patients meeting because of inadequate treatment or insufficient follow-up. Disposition is found by criteria take a bed, are treated, and are pending (blue). Patients who pass are green; with randomly sampling times of occurrence of inevaluable and DLT distributions. If both DLTs, red. If inevaluable, they exit. A summary data dashboard for many of the same occur, the disposition is the first event. B, Simulation software showing patient flow. scenario and various design parameters is at right (https://www.flexsim.com/clinical-trials). Each dose level has 2 cohorts of 3 beds, plus 2 extra beds for use in queue-based MAD indicates maximum administered dose; MTD, maximum tolerated dose; TDPS, total, modifications during escalation. There are 2 more beds (gray) for rare instances during DLTs, pending, status (dose above closed). JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 3/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Table 1. Comparison of the 3 + 3 and the Phase 1 Queue 3 + 3 Design Decisions No. on current level Dose level for next patient Dose-limiting b c d e e Row Total Evaluable Toxicity IQ 3+3 3+3 1 0-2 0 0 Same dose level Same dose level 2 3 0 0 Hold accrual Hold accrual 3 1-2 1 0 Same dose level Same dose level 4 3 1 0 Same dose level Hold accrual 5 4 1 0 Hold accrual Not allowed 6 2 2 0 Same dose level Same dose level 7 3 2 0 Same dose level Hold accrual 8 4-5 2 0 Same dose level Not allowed 9 6 2 0 Hold accrual Not allowed f f 10 3 3 0 Escalate Escalate 11 4-6 3-5 0 Escalate Same (up to 6; implies that higher dose cannot be tested further) 12 6 6 0 Escalate (or MTD) MTD 13 1-2 1 1 Same dose level Same dose level 14 3 1 1 Hold accrual Hold accrual 15 2 2 1 Same dose level Same dose level 16 3 2 1 Same dose level Hold accrual 17 4 2 1 Hold accrual Not allowed 18 3-5 3-5 1 Same dose level Same dose level 19 6 3 1 Hold accrual Hold accrual 20 6 4 1 Same dose level Hold accrual 21 6 5 1 Same dose level Hold accrual 22 7 4 1 Hold accrual Not allowed 23 7 5 1 Same dose level Not allowed 24 6-8 6-8 1 Escalate Escalate (patients 7 and 8 not allowed) 25 2-7 2-6 2 Deescalate Deescalate (patient 7 not allowed) 26 7 7 2 MTD Not allowed 27 8 7 2 Hold accrual Not allowed 28 8 8 2 MTD Not allowed g g 29 Any Any 3 Deescalate Deescalate Abbreviations: IQ, phase 1 queue; MTD, maximum tolerated dose. The IQ design and the corresponding parent design are side by side. Rare scenarios are not listed, but the full decision grid 7,8 is available in the software input module available. The number of patients who consented to that dose level, excluding individuals who did not meet screening criteria or patients considered inevaluable with respect to dose-limiting toxicity. The number of patients who provided an answer to the dose-limiting question (yes or no). The number of patients who experienced a dose-limiting toxicity. The number pending is the difference between the total number and the number evaluable. The action to be taken for the next patient for the IQ design and the parent design, respectively. If a patient pending evaluation on a lower dose experiences a dose-limiting toxicity, the principal investigator in consultation with the sponsor may choose to reduce the dose level of any patients currently on a higher dose level, pending review of the adverse event data. If the next higher dose level is not available (there is no higher dose level or the higher dose level was already tested and found to be too toxic), a maximum of 8 patients can be treated at the current dose level and the principal investigator should declare the MTD with 0 or 1 dose-limiting toxicity of 6 (or 0 of 5) patients. For IQ 3 + 3, no more than 4 patients at risk are allowed, with no more than 6 patients at risk for the IQ rolling 6 design. Two dose-limiting toxicities in 7 or 8 patients means that the principal investigator can also declare the MTD (and it is suggested to continued using monitoring rules for the expanded cohort). Current level exceeds the MTD. The MTD is the highest level at which less than 33% of patients had dose-limiting toxicities, with at least 6 patients evaluable. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 4/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies As an example modification, for the IQ 3 + 3 design (Table 1; row 4), enrolling a fourth patient at the same dose level, knowing that 1 of 3 patients has a pass with 2 patients pending, is considered less risky than putting 3 patients at risk without any patient data, as is permitted by the 3 + 3 design, so this modification does not exceed the risk allowed by the 3 + 3 but rather starts a patient through the process in case a patient who started earlier is considered not to meet screening criteria or becomes inevaluable. This is the key principle behind the IQ 3 + 3 design. Additional details and rationale for special IQ 3 + 3 decisions can be found in the eAppendix in the Supplement, including why and when up to 8 patients can be accrued during dose-level escalation. For the IQ rolling 6 design (Table 2), the parent design allows 6 patients to be put at risk but does not allow escalation of the first 3 patients pass when there are patients pending. As a result, the rolling 6 does not achieve its anticipated speed advantage. The IQ rolling 6 allows escalation with 3 or more patients who are treated and fully evaluated with no DLTs, with the additional knowledge that none of the patients whose data are pending or have begun treatment have reported a DLT, which is consistent with the rules for the IQ 3 + 3 design with patients pending. The decision rules have also been modified so the IQ rolling 6 design improves patient flow but does not exceed the maximum risk permitted in the rolling 6 design (eAppendix in the Supplement). Table 2. Comparison of Rolling 6 and Phase 1 Queue Rolling 6 Design Decisions Abbreviations: IQ, phase 1 queue; MTD, maximum tolerated dose. No. on current level Dose level for next patient Dose-limiting The IQ design and the corresponding parent design b c d e e Row Total Evaluaable toxicities IQ rolling 6 Rolling 6 are side by side. Rare scenarios are not listed, but the 1 0-5 0 0 Same dose level Same dose level full decision grid is available in the software input 7,8 2 6 0 0 Hold accrual Hold accrual module available. 3 1-5 1 0 Same dose level Same dose level The number of patients who consented to that dose level, excluding individuals who did not meet 4 6 1 0 Same dose level Hold accrual screening criteria or patients considered inevaluable 5 7 1 0 Hold accrual Not allowed with respect to dose-limiting toxicity. 6 2-5 2 0 Same dose level Same dose level The number of patients who provided an answer to 7 6-7 2 0 Same dose level Hold accrual (patient 7 not the dose-limiting question (yes or no). allowed) 8 8 2 0 Hold accrual Not allowed The number of patients who experienced a dose- limiting toxicity. The number pending is the 9 3 3 0 Escalate Escalate difference between the total number and the 10 4-5 3 0 Escalate Same dose level number evaluable. 11 6 3 0 Escalate Hold accrual The action to be taken for the next patient for the IQ 12 7-8 3 0 Escalate Not allowed design and the parent design, respectively. If a 13 4 4 0 Escalate Escalate patient pending evaluation on a lower dose 14 5 4 0 Escalate Same dose level experiences a dose-limiting toxicity, the principal investigator in consultation with the sponsor may 15 6-8 4 0 Escalate Hold accrual (patients 7 and 8 not allowed) choose to reduce the dose level of any patients f f 16 5 5 0 Escalate (or MTD) Escalate (or MTD) currently on a higher dose level, pending review of f f the adverse event data. 17 6 5 0 Escalate (or MTD) Escalate (or MTD) If the next higher dose level is not available (there is 18 7-8 5 0 Escalate (or MTD) Not allowed no higher dose level or the higher dose level was 19 1-5 1-5 1 Same dose level Same dose level already tested and found to be too toxic), a 20 6 1-3 1 Hold accrual Hold accrual maximum of 8 patients can be treated at the current 21 6 4 1 Same dose level Hold accrual dose level and the principal investigator should 22 6 5 1 Same dose level Hold accrual declare the MTD with 0 or 1 dose-limiting toxicity of 6 (or 0 of 5) patients. For IQ 3 + 3, no more than 4 23 7 4 1 Hold accrual Not allowed patients at risk are allowed, with no more than 6 24 7 5 1 Same dose level Not allowed patients at risk for the IQ rolling 6 design. Two dose- 25 6-8 6-8 1 Escalate Escalate (patients 7 and 8 not f limiting toxicities in 7 or 8 patients means that the allowed) g principal investigator can also declare the MTD (and 26 2-8 2-6 2 Deescalate Deescalate (patients 7 and 8 not it is suggested to continued using monitoring rules allowed) for the expanded cohort). 27 7 7 2 MTD Not allowed Current level exceeds the MTD. The MTD is the 28 8 7 2 Hold accrual Not allowed highest level at which less than 33% of patients had 29 8 8 2 MTD Not allowed dose-limiting toxicities, with at least 6 patients g g 30 Any any 3 Deescalate Deescalate evaluable. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 5/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Using decision tables 1 and 2, the operating characteristics of the 3 + 3 and IQ 3 + 3 designs and the rolling 6 and IQ rolling 6 designs were evaluated for 12 scenarios detailed in Table 3, each motivated by our clinical trial experience. In each scenario, we specify a starting dose level, the lowest and highest dose levels, and other parameters (Table 3). The maximum waiting time was assumed to be 0 in the simulations, representing patients who will not wait if slots are unavailable and instead will be accrued to a different phase 1 study (or treated outside of a study). Scenarios A1 through A3 (Table 3) are based on our experience with phase 1 studies in the California Cancer Consortium, and scenarios A4 through A7 explore the results of changes in the screen failure rate, the interarrival time distribution, the course length, and the dose-toxicity association. For scenarios A1 through A3, we collected data on 14 phase 1 (or dose-finding portions of phase 2) cancer studies that had completed their phase 1 portions between October 2004 and November 2014 in the California Cancer Consortium. Along with the more traditional 3 + 3 designs, these studies include our published phase 1 study that first used a queue-based method and a study using isotonic regression-guided decisions in patients with organ dysfunction. One key parameter, largely independent of the design, that affected the duration of trials is the rate of inevaluability (the frequency with which patients who are treated cannot be assessed for DLTs and subsequent dose- level escalation decisions). In Skolnik et al, the median inevaluability rate for patients in pediatric phase 1 studies in the Children’s Oncology Group was 13.6% (range, 3.4%-22.7%). Our consortium experience with adult solid tumors suggests a higher rate of patients who are inevaluable; in the 14 completed studies reviewed, there were 375 patients with 75 patients inevaluable, representing a 20% overall rate (scenario A1), with considerable variability between studies (3.6% to 44.0% in scenarios A2 and A3, respectively). Scenario A1 (Table 3), our standard example, has 5 dose levels, has a 28-day DLT period (cycle 1), starts at level 2, identifies candidates for the study at a mean of every 10 days, has a 1-month maximum screening period (with a screen failure of 30%), and a 20% inevaluable rate. Reasons for inevaluability in our experience included patient choice; non-dose-associated complications, including rapid disease progression; noncompliance with the protocol; changes in insurance status; drug supply issues; reaction to a poisonous bug bite; and late discovery of disease that rendered the patient ineligible. For another important parameter, the screen failure rate, which is not automatically tracked in the consortium, we reviewed single-institution phase 1 clinical trials at the center City of Hope. The screen failure rate was fairly consistent across the solid tumor studies at approximately 30%, which represented all causes that would result in a patient not receiving treatment after providing consent. This screen failure rate was used for all scenarios other than A4, in which we looked to demonstrate the result of an increase in screen failures, and scenario D, in which we modeled a specific clinical trial with a known screen failure rate. Scenario B was provided to compare the simulation of the IQ designs with its respective parent design for a safety lead-in study. Scenario C1 is based on a study of blinatumomab and lenalidomide (NCT02568553). This study’s experimental treatment was introduced on cycle 2, increasing the number of inevaluable patients (whose data could not be considered for a dose-escalation decision), while simultaneously allowing patients to hold a slot for an extended period. Early in the study, two-thirds of the patients were inevaluable, and combined with the cycle 2 evaluation, this provides the opportunity for a very substantial improvement while using queue-based methods. We amended the study to change from a 3 + 3 design to an IQ 3 + 3 design, which was subsequently approved by the Cancer Therapy Evaluation Program and the central institutional review board based on the simulation work. Scenarios C2 and C3 are variations that show the result of changes in accrual rate or inevaluability rate. Scenario D is based on a study of intraperitoneal chemotherapy (NCT00825201), in which surgery preceding chemotherapy increased the time a patient can occupy a slot; the cycle length was 28 days, but surgery and recovery could delay chemotherapy up to 90 days (screening time). Forty percent of the patients were ineligible per screening criteria, but only 7.5% were inevaluable for DLTs. We estimated the screening distribution and the arrival distribution from the study data. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 6/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 7/13 Table 3. Scenarios Simulated for the 3 + 3, Phase 1 Queue 3 + 3, Rolling 6, and Phase 1 Queue Rolling 6 Scenario A2: Low A3: High A4: Standard B: 21-d C2: Second C3: Second A1: Standard inevaluability, inevaluability, Δscreen A5: Standard A6: Standard A7: Standard safety C1: Second cycle DLT cycle DLT Δin Characteristic phase 1 phase 1 phase 1 failure Δarrival ΔCL Δtoxicity lead-in cycle DLT Δarrival evaluability D: IP phase 1 Starting dose level 2 2 2 2 22223 3 3 1 Lowest possible dose level 1 1 1 1 11111 1 1 1 Highest possible dose level 5 5 5 5 55524 4 4 6 Course length, d 28 28 28 28 28 21 28 21 56 56 56 28 Maximum waiting list time, d 0 0 0 0 00000 0 0 0 Screening duration, d β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,90,1,1.97) Screen failure probability, % 30 30 30 60 30 30 30 30 30 30 30 40 Probability of patient being 20 3.6 44 20 20 20 20 20 66 66 33 7.5 inevaluable , % Time until patient becomes β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) inevaluable, d Dose-limiting toxicity F(0.2, 8.5) F(0.2, 8.5) F(0.2, 8.5) F(0.2, 8.5) F(0.2,8.5) F(0.2, 8.5) F(0.2, 5.5) F(0.2, 5.5) F(0.2, 9.5) F(0.2, 9.5) F(0.2, 9.5) F(0.2, 10.5) probability function Time until dose-limiting β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(29,CL,1.5,1) β(29,CL,1.5,1) β(29,CL,1.5,1) β(0,CL,1.5,1) toxicity event, d Patient interarrival time, d exp(0,10,1) exp(0,10,1) exp(0,10,1) exp(0,10,1) exp(0,15,1) exp (0,10,1) exp (0,10,1) exp(0,10,1) exp(0,10,1) exp (0,15,1) exp (0,10,1) exp(0,15,1) Abbreviations: β, beta distribution; CL, course length; DLTs, dose-limiting toxicities; exp, exponential distribution F(x,y) = 100 × (0.5 + atan(x×π× [current dose level − y])/π). with the second parameter as the mean; IP, intraperitoneal. JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies For each design (3 + 3, IQ 3 + 3, rolling 6, and IQ rolling 6) scenarios were simulated 800 times, representing different random samplings of patient arrivals, DLT events (yes or no) based on the dose-toxicity DLT probability curve, DLT time (if a DLT occurred), evaluable results and time, and screening results and time. The inclusion of 800 samplings provides high confidence in the results (eg, a standard error <1.8%) while providing adequate numbers to observe unusual events (an occurrence rate of 0.3% will be observed with a probability >90%). Each sampling used the same patients arriving at the same time across the designs, and if the nth patient had a DLT while being treated at level 1 in the rolling 6 design, this same patient treated at level 2 in the IQ rolling 6 design would have a DLT because it is a higher dose level. Calculations were carried out in FlexSim HC version 5.3 (FlexSim Healthcare). The module is freely available online. Results are presented in graphical and tabular forms using means, medians, and ranges to be consistent with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist for transparency on issues associated with analytic methods (eg, skewness), study parameters (eg, values, ranges) and outcomes (eg, mean differences). Results For all scenarios, the IQ 3 + 3 design had shorter expected study durations than the 3 + 3 design, ranging from a reduction of 1.6 to 10.4 months (Figure 2A), and likewise, the IQ rolling 6 design has lower expected study durations than the rolling 6 design, ranging from a reduction of 0.4 months to 10.5 months (Figure 2B). There was a small increase in the mean number of patients treated on the IQ 3 + 3 design (difference in mean number of patients in all scenarios, <3.2 patients; Figure 2C), while the IQ rolling 6 had a smaller mean number of patients than the rolling 6 in 9 of the 12 scenarios, with a difference that did not exceed 3.3 patients (Figure 2D). Detailed results can be found in eTables 3 and 4 in the Supplement. For scenario A1, the typical phase 1 study in the consortium, the expected (mean) duration of the phase 1 study using the traditional 3 + 3 design was 19.5 months (range, 7.1-41.3 months), whereas the expected duration was 15.8 months (range, 5.3-27.0 months) for the IQ 3 + 3 design. This represents an expected reduction of 3.7 months (difference in medians, 3.6 months; a 19% reduction). There is a mean increase of 2.8 patients. In eTable 5 in the Supplement, we extended this mean scenario to 9 dose levels, so the MTD would almost assuredly be reached before the highest allowable dose level. In that setting, the expected duration was reduced by 5.4 months (26.0 vs 20.6 months; a 21% reduction), and we also present additional metrics (expected percentage of patients on each dose level) that confirm that the operating characteristics of the 3 + 3 design are maintained. By reducing inevaluability to 3.6% (scenario A2), the expected duration of both the 3 + 3 and the IQ 3 + 3 designs were shorter compared with scenario A1. The expected times were 16.5 vs 13.9 months, respectively (a 16% difference). When increasing the rate of patients who are inevaluable to 44% (scenario A3), the expected duration of the 3 + 3 and the IQ 3 + 3 designs increased. The reduction because of the IQ-based design became 6.4 months (24%); compared with a reduction of 3.7 months (19%) associated with scenario A1. With all designs, a higher number of patients who are inevaluable tended to reduce the dose level selected as the MTD (eTable 3 and 4 in the Supplement). This is because of the competing events of a patient experiencing a DLT and a patient being deemed inevaluable. As an extreme example, if the DLT evaluation period was very long, all patients would eventually be inevaluable, so the only patients who are evaluable would be those with a DLT before being deemed inevaluable, making all doses appear toxic. In scenarios A4 through A7, we explored the results of (1) increasing screen failures, (2) increasing the availability of patients (reducing the interarrival time), (3) reducing the course length, and (4) increasing toxicity. The IQ designs had uniformly shorter expected durations (reduction of 6.7 months, 4.2 months, 3.7 months, and 3.1 months, respectively). The frequency of the selection of JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 8/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Figure 2. Box-Whisker Plots of Study Duration for the 3 + 3 and Phase 1 Queue (IQ) 3 + 3 and Rolling 6 and IQ Rolling 6 Designs A Study duration for the 3+3 and IQ 3+3 designs Study design 80 3+3 IQ 3+3 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario B Study duration for the Rolling 6 and IQ Rolling 6 designs Study design Rolling 6 IQ Rolling 6 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario No. of patients who started the 3+3 and IQ 3+3 study designs Study design 3+3 IQ 3+3 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario D No. of patients who started the Rolling 6 and IQ Rolling 6 study designs Study design Rolling 6 IQ Rolling 6 A, Study duration for the 3 + 3 and IQ 3 + 3 studies. B, Study duration for the rolling 6 and IQ rolling 6 studies. C, Number of patients who started treatment for the 3 + 3 and IQ 3 + 3 designs. D, Number of patients who started treatment for the rolling 6 and IQ rolling 6 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D designs. Graphs are based on 800 simulations per Scenario scenario (Table 3). JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 9/13 Patients starting treatment, No. Patients starting treatment, No. Study duration, mo Study duration, mo JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies the MTD between the IQ design and the parent design differed by a range of less than 0.1% to 3%, and the mean number of DLTs at a dose level greater than the MTD differed by 0.01 to 0.10 patients. For scenario B, which was considered the safety lead-in, even though 77.8% and 77.1% of the simulations selected the starting dose as the MTD (for the 3 + 3 and the IQ 3 + 3, respectively), the expected study duration was reduced from 7.6 months to 6.0 months (a 21% reduction). The expected number of patients differed by 0.6 patients. For scenario C1, the expected study duration was reduced from 34.2 months to 24.5 months, a 9.7-month (28%) reduction. This was expected given the inevaluable rate of 66% (with a 30% screen failure rate) and the amount of time a patient will be pending (up to 84 days). The expected number of patients required increased by 3.2, and the frequency of the MTD selected differed by less than 1%. The results of changing the arrival patterns or inevaluability rate are reflected in C2 and C3. The last scenario, D, demonstrated a lengthier expected study duration of 42.1 months using the 3 + 3 design. This was 31.9 months with the IQ 3 + 3 design. The selection of the MTD differed by 0.5% to 1.5%. For comparisons of the rolling 6 and IQ rolling 6 designs on the same 12 scenarios, a similar pattern emerged. For scenario A1, for example, the reduction in expected study duration for the IQ rolling 6 design is 3.4 months (16.4 vs 13.0 months), with a difference in the proportion of simulations choosing any particular dose as the MTD ranging from 0.1% to 1.1%; the difference in median study duration was 3.9 months. There were no extra patients treated, and in fact, the expected number of patients was reduced by 0.6 patients, reflecting that the rolling 6 design generally requires all patients treated to complete their assessment before escalating (even if 0 DLTs are noted in the first 3 patients), which adds time at that dose level and potentially results in even more patients at that dose level. For extended scenario A1 (eTable 5 in the Supplement), the expected reduction was 4.7 months. Discussion The phase 1 queue depends on the treatment, patient population, and specific language in the protocol. For example, the screen failure rate is dependent on specific eligibility criteria, insurance issues, and the frailty of the patient population. Inevaluability for the purpose of DLT determination depends in part on the length of the evaluation period, the frailty of the patient population, the frequency of rapid disease progression, and other causes of drug discontinuation or unplanned dose reductions that are not associated with adverse events attributed to the dose of the investigational agent (or agents). Patients may also be considered inevaluable for DLT determination if critical tests are missed. We observed in our phase 1 studies that these details can dramatically affect phase 1 study duration and such imperfections are present in every study. We can adapt designs to these imperfections to reduce the study duration while not exceeding the risk permitted in the parent design or affecting the operating characteristics. Because the rolling 6 design was originally developed for special settings, such as pediatrics, after the adult study, we compared the 3 + 3 with the IQ 3 + 3 design and the rolling 6 with the IQ rolling 6 design. In our typical phase 1 study, the IQ 3 + 3 design is expected to save 3.7 months in study duration compared with the 3 + 3 design, and the IQ rolling 6 design 3.4 months compared with the rolling 6 design, allowing phase 2 studies to start months earlier. Two scenarios (C1 or D) demonstrate approximate 10-month reductions in study duration when using the IQ 3 + 3 design. The IQ modifications allow accrual of patients in certain situations when the parent design would not permit this, and in the case of the rolling 6 design, it permits escalation when the parent design would not because of patients who are pending, a key limitation of the original rolling 6 design rules. Several time-saving advantages include (1) the ability to more readily replace patients who do not meet screening criteria or become inevaluable for dose-escalation decisions; (2) if dose JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 10/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies deescalation is required, the possibility that additional patients may have already been treated on the lower dose levels, accelerating the deescalation portion of the study; (3) the IQ rolling 6 design, which permits escalation with 0 of 3 or 0 of 4 patient with a DLT in the presence of additional patients who have started treatment without any DLT to date. Other advantages include situations in which physicians can treat a patient on the highest priority trial rather than hold accrual; especially for the IQ 3 + 3, a further advantage is to provide more information per dose level (if additional patients are treated prior to escalating), which can help select the recommended phase 2 dose. There is minimal difference between the parent and IQ designs in the selection of the MTD. Also, the principal investigator and/or treating physicians are not required to offer a trial to a patient if they wish to wait until the current patients at the current dose level complete cycle 1 evaluation. In that case, the IQ design reverts to the parent design. eTable 2 in the Supplement lists several studies either with completed phase 1 portions (6 studies) or ongoing phase 1 portions (3 studies) using the IQ designs. This includes 2 lymphoma phase 1 trials (including the trial that motivated scenario C1); 2 hematology phase 1 studies, including a phase 1 stem cell transplant study ; 4 breast cancer studies; and an IQ rolling 6 study of escalating doses of radiation therapy to the prostate fossa, in which prior data were judged sufficient to allow 6 patients to accrue, analogous to the pediatric setting. After the first queue-based variation of the 3 + 3 design started accrual in May 2012, the experience of investigators, coordinating centers, and statistical teams with these designs have increased. Their use has increased, and additional refinements have been implemented that are reflected in the rules and simulations presented here. The use of queuing methods to evaluate and optimize the operating characteristics of phase 1 designs or other aspects of clinical research is a nascent field. As with the parent designs, the determination of the MTD with the IQ variations is not usually the end of the dose-finding study. An expansion cohort at the recommended phase 2 dose or MTD is recommended with additional monitoring rules (as noted in the eAppendix in the Supplement for 1 study [NCT02568553]). When appropriate, a randomized dose-ranging phase 2 study evaluating candidate doses may also be suggested. Expanding the queue-based methods beyond the MTD determination is one possible area of future work. Queue-based modifications of DLT-rate targeting phase 1 designs with an implicit or 14,15 explicit queue can also be explored. The direct cost of the IQ 3 + 3 design when compared with the traditional 3 + 3 design comes in the form of a few extra patients. There is no similar cost when converting the rolling 6 to the IQ rolling 6 design. However, the IQ rolling 6 design does allows escalation with patients pending, which is less conservative than the rolling 6 designs. Indirect costs for the IQ designs include more frequent review of the data and more clinical judgment as to whether to add new patients or escalate with patients pending (when permitted) or to hold accrual and revert to the parent design. Limitations We did not model time to acquire data or decision time, which is in control of the data coordinating center and is usually short relative to the other intervals. We also did not implement separate screening-time distributions for patients considered screen failures vs successes or model delays in treatment. Conclusions The IQ 3 + 3 and the IQ rolling 6 designs should be considered as alternatives to the 3 + 3 and rolling 6 designs, respectively. The IQ designs better adapt to the patient queue to reduce study duration without exceeding the risk limits of the parent design or affecting operating characteristics. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 11/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies ARTICLE INFORMATION Accepted for Publication: February 25, 2020. Published: May 13, 2020. doi:10.1001/jamanetworkopen.2020.4787 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Frankel PH et al. JAMA Network Open. Corresponding Author: Paul H. Frankel, PhD, Division of Biostatistics, Department of Research Information Sciences, City of Hope, Duarte, CA 91010 (pfrankel@coh.org). Author Affiliations: Division of Biostatistics, Department of Research Information Sciences, City of Hope, Duarte, California (Frankel, Longmate); Department of Medical Oncology, City of Hope, Duarte, California (Chung, Newman); Hematology/Oncology, University of California, Davis, Davis (Tuscano); Hematology and Hematopoietic Cell Transplantation, City of Hope, Duarte, California (Siddiqi); Radiation Oncology, City of Hope, Duarte, California (Sampath); Biostatistics Core, Norris Cancer Center, University of Southern California, Los Angeles (Groshen); Developmental Cancer Therapeutics Program, Division of Molecular Pharmacology, City of Hope, Duarte, California (Newman). Author Contributions: Dr Frankel had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Newman is the senior author. Concept and design: Frankel, Longmate, Groshen. Acquisition, analysis, or interpretation of data: Frankel, Chung, Tuscano, Siddiqi, Sampath, Newman. Drafting of the manuscript: Frankel, Groshen, Newman. Critical revision of the manuscript for important intellectual content: Frankel, Chung, Tuscano, Siddiqi, Sampath, Longmate, Groshen. Statistical analysis: Frankel, Siddiqi, Groshen. Obtained funding: Frankel, Newman. Administrative, technical, or material support: Frankel, Chung, Tuscano. Supervision: Frankel, Chung. Conflict of Interest Disclosures: Dr Chung reported personal fees from Celgene, Ipsen, Gritstone, Westwood Bioscience, and Perthera outside the submitted work. Dr Tuscano reported grants from the National Cancer Institute during the conduct of the study. Dr Siddiqi reported personal fees from Pharmacyclics, Janssen, AstraZeneca, Seattle Genetics, Juno Therapeutics, Kite Pharma, and AstraZeneca outside the submitted work. Dr Newman reported grants from the National Institutes of Health during the conduct of the study. No other disclosures were reported. Funding/Support: This work is supported in part by the National Cancer Institute (grants UM1CA186717, U01CA62505, and N01CM-62209) and included work performed in the Biostatistics Core (grant P30CA033572). Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Additional Contributions: The authors thank Stella Khoo, MBA, BS, City of Hope, for collecting the information on the National Cancer Institute/Cancer Therapy Evaluation Program studies; Richard Sposto, PhD, University of Southern California, for his input and simulation code; Cliff King, BS, FlexSim Software Products, for his assistance with the FlexSim simulation tool; and Sandra Thomas, PhD, City of Hope, for manuscript editing. They were not compensated for their contributions. Additional Information: City of Hope, the University of Southern California, and University of California at Davis are National Cancer Institute–Designated Comprehensive Cancer Centers. REFERENCES 1. Abrams JS, Mooney MM, Zwiebel JA, et al. Implementation of timeline reforms speeds initiation of National Cancer Institute-sponsored trials. J Natl Cancer Inst. 2013;105(13):954-959. doi:10.1093/jnci/djt137 2. Doroshow JH. Timely completion of scientifically rigorous cancer clinical trials: an unfulfilled priority. J Clin Oncol. 2013;31(27):3312-3314. doi:10.1200/JCO.2013.51.3192 3. US Food and Drug Administration. Step 3: clinical research. Published January 4, 2008. Accessed April 7, 2020. https://www.fda.gov/patients/drug-development-process/step-3-clinical-research JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 12/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies 4. Alsumidaie M, Schiemann P. Why are cancer clinical trials increasing in duration? Published 2015. Accessed April 24, 2020. http://www.appliedclinicaltrialsonline.com/why-are-cancer-clinical-trials-increasing-duration 5. Storer BE. Design and analysis of phase I clinical trials. Biometrics. 1989;45(3):925-937. doi:10.2307/2531693 6. Skolnik JM, Barrett JS, Jayaraman B, Patel D, Adamson PC. Shortening the timeline of pediatric phase I trials: the rolling six design. J Clin Oncol. 2008;26(2):190-195. doi:10.1200/JCO.2007.12.7712 7. FlexSim Healthcare. Register to create your free clinical trials FlexSim account. Published 2019. Accessed April 20, 2020. https://healthcare.flexsim.com/clinical-trials 8. Queue. A queueing approach to reduce phase I study duration in oncology. Published 2015. Accessed April 20, 2020. https://oneq.netlify.app 9. Sposto R, Groshen S. A wide-spectrum paired comparison of the properties of the Rolling 6 and 3+3 Phase I study designs. Contemp Clin Trials. 2011;32(5):694-703. doi:10.1016/j.cct.2011.04.009 10. Siddiqi T, Frankel P, Beumer JH, et al. Phase 1 study of the Aurora kinase A inhibitor alisertib (MLN8237) combined with the histone deacetylase (HDAC) inhibitor vorinostat in lymphoid malignancies. Leuk Lymphoma. 2020;61(2):309-317. doi:10.1080/10428194.2019.1672052 11. Stein A, Palmer J, Tsai NC, et al. Phase I trial of total marrow and lymphoid irradiation transplant conditioning in patients with relapsed/refractory acute leukemia. Biol Blood Marrow Transplant. 2017;23(4):618-624. doi:10.1016/ j.bbmt.2017.01.067 12. Sampath S, Frankel P, Vecchio BD, et al. Stereotactic body radiation therapy to the prostate bed: results of a Phase 1 dose-escalation trial. Int J Radiat Oncol Biol Phys. 2020;106(3):537-545. doi:10.1016/j.ijrobp.2019.11.005 13. Ratain MJ. Targeted therapies: redefining the primary objective of phase I oncology trials. Nat Rev Clin Oncol. 2014;11(9):503-504. doi:10.1038/nrclinonc.2014.135 14. Iasonos A, O’Quigley J. Design considerations for dose-expansion cohorts in phase I trials. J Clin Oncol. 2013;31 (31):4014-4021. doi:10.1200/JCO.2012.47.9949 15. Zhou T, Guo W, Ji Y. PoD-TPI: Probability-of-decision toxicity probability interval design to accelerate phase I trials. Published 2019. Accessed April 8, 2020. https://arxiv.org/abs/1904.12981 SUPPLEMENT. eTable 1. Phase I, II and III studies by year. eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods. eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). eAppendix. Design Rationale. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 13/13 Supplementary Online Content Frankel PH, Chung V, Tuscano J, et al. Model of a queuing approach for patient accrual in phase I oncology studies. JAMA Netw Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 eTable 1. Phase I, II and III studies by year. eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods. eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). eAppendix. Design Rationale. This supplementary material has been provided by the authors to give readers additional information about their work. © 2020 Frankel PH et al. JAMA Network Open. eTable 1. Phase I, II and III studies by year: Study Start Phase I Phase II* Phase III 2019 (1/1/2019-10/11/2019) 1143 1552 395 2018 1278 1778 465 2017 1239 1751 429 2016 1208 1573 426 2015 1047 1499 482 *Phase II studies in clinicaltrials.gov can have a Phase I portion (e.g. safety lead-in), and can be Phase II/III studies, so we have focused on the number of pure Phase I studies (Early Phase 1 or Phase I) vs pure Phase III studies in clinicaltrials.gov (cancer, interventional, study start date). eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods NCT Title NCT02568553 Le na lid om id e a nd Blin a t um om a b in Tre a t ing Pa t ie nt s Wit h Re la p se d Non - Hod gkin Lym p hom a NCT00576979 Intensity-Modula t e d Ra dia t ion The ra py, Et op osid e , a nd Cyclop hosp ha m id e Followe d By Donor St e m Ce ll Tra nsp la nt in Tre a t ing Pa t ient s Wit h Re la p se d or Re fra ct ory Acut e Lym p hob la st ic Le uke m ia or Acut e Mye loid Leukemia NCT01567709 Alise rt ib in Com b ina t ion Wit h V orin ost a t in Tre a t ing Pa t ie nt s Wit h Re la p se d or Re curre nt Hod gkin Lym p hom a , B-Ce ll Non- Hod gkin Lym p hom a , or Pe rip he ra l T-Cell Lymphoma NCT01041443 5-Fluoro-2' -De oxycyt id ine a nd Te t ra hyd rourid ine in Tre a t ing Pa t ie nt s Wit h Acut e Mye loid Le uke m ia or Mye lod ysp la st ic Synd romes NCT02778685 Pe m brolizu m a b, Le t rozole , a nd Pa lbociclib in Tre a t ing Post m e nop a usa l Pa t ient s Wit h Ne wly Dia gnose d Me t a st a t ic St a ge IV Est roge n Re ce p t or Posit ive Bre a st Ca nce r NCT02648477 Pe m brolizu m a b a nd Doxorubicin Hyd rochlorid e or Ant i-Est roge n The ra p y in Tre a t ing Pa t ie nt s Wit h Trip le-Ne ga t ive or Horm one Re ce p t or-Posit ive Me t a st a t ic Bre a st Ca nce r NCT02971761 Pe m b rolizu m a b a nd Enob osa rm in Tre a t ing Pa t ie nt s Wit h And roge n Re ce p t or Posit ive Me t a st a t ic Trip le Ne ga t ive Bre a st Ca nce r © 2020 Frankel PH et al. JAMA Network Open. NCT01923506 St e re ot a ct ic Bod y Ra d ia t ion The ra p y in Tre a t ing Pa t ie nt s Wit h Prost a t e Ca nce r Aft e r Und e rgoing Surge ry NCT03853707 Ipatasertib a nd Ca rb op la t in Wit h or Wit hout Pa clit a xe l in Tre a t ing Pa t ie nt s Wit h Me t a st a t ic Trip le Ne ga t ive Bre a st Ca nce r eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). Variation #Treated #Treated #Months #Months Mean 3+3 IQ 3+3 DLT rate 3+3 IQ3+3 3+3 IQ3+3 #DLTs % MTD % MTD (based Mean ,Med Mean,Med Mean,Med Mean, Med above At level At level on model) (range) (range) (range) (range) MTD Scenario A1 19.1, 19.0 21.9, 22.0 19.5, 19.4 15.8, 15.8 3+3: 0.87 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (8-37) (7.1-41.3) (5.3-27.0) IQ3: 0.95 1 7.4% 1 7.3% 1 6.7% Phase I 2 8.4% 2 9.3% 2 7.6% 20% Ineval 3 9.9% 3 12.5% 3 9.0% 4 16.3% 4 15.6% 4 10.8% 5 58.0% 5 55.0% 5 13.6% Scenario A2 16.1,16.0 18.6,19.0 16.5,16.5 13.9,14.0 3+3: 0.81 NA <1.0% NA <1.0% NA NA Low Ineval (8-26) (8-29) (6.3-32.7) (5.1-25.3) IQ3: 0.87 1 5.6% 1 6.1% 1 6.7% Phase I 2 7.8% 2 8.9% 2 7.6% 3.6% Ineval 3 9.5% 3 10.0% 3 9.0% 4 15.4% 4 15.9% 4 10.8% 5 61.3% 5 58.5% 5 13.6% Scenario A3 26.0, 26.0 29.0, 29.0 26.6, 26.3 20.2, 20.3 3+3: 1.08 NA 1.0% NA 1.0% NA NA High Ineval (9-54) (9-54) (7.3-59.6) (6.9-40.8) IQ3: 1.13 1 7.6% 1 8.4% 1 6.7% Phase I 2 11.8% 2 11.9% 2 7.6% 44% Ineval 3 13.5% 3 15.0% 3 9.0% 4 17.6% 4 17.4% 4 10.8% 5 48.5% 5 46.4% 5 13.6% Scenario A4 18.8, 19.0 21.1,21.0 30.0,29.4 23.3,23.1 3+3: 0.90 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (7-37) (7.7-63.8) (6.9-45.9) IQ3: 0.89 1 6.3% 1 6.6% 1 6.7% Phase I 2 8.3% 2 8.1% 2 7.6% 3 11.1% 3 12.1% 3 9.0% 60% 4 17.1% 4 15.8% 4 10.8% 5 56.6% 5 56.9% 5 13.6% Scenario A5 19.0, 19.0 21.3, 22.0 23.5, 23.4 19.3, 19.1 3+3: 0.87 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (8-37) (8.6-48.5) (5.4-34.6) IQ3: 0.92 1 7.1% 1 6.8% 1 6.7% Phase I 2 8.5% 2 9.4% 2 7.6% 3 10.0% 3 12.6% 3 9.0% 15 day mean) 4 16.3% 4 14.6% 4 10.8% 5 57.9% 5 56.1% 5 13.6% Scenario A6 19.0,19.0 21.4,22.0 18.1, 17.9 14.4, 14.4 3+3: 0.88 NA <1.0% NA <1.0% NA NA Ave Ineval (9-34) (7-36) (6.4-35.1) (4.7-26.4) IQ3: 0.93 1 7.3% 1 7.1% 1 6.7% Phase I 2 8.4% 2 8.6% 2 7.6% 3 9.8% 3 12.3% 3 9.0% days 4 16.5% 4 15.6% 4 10.8% 5 57.8% 5 55.9% 5 13.6% Scenario A7 16.6, 16.0 18.1, 17.0 16.5, 15.9 13.4, 12.9 3+3: 2.1 NA 3.8% NA 4.4% NA NA Ave Ineval (4-31) (4-38) (3.8-36.5) (3.5-28.3) IQ3: 2.2 1 15.4% 1 16.0% 1 10.8% Phase I 2 23.5% 2 24.3% 2 13.6% 3 27.8% 3 29.8% 3 18.0% 4 22.5% 4 20.6% 4 25.9% 5 7.1% 5 5.0% 5 40.3% Scenario B 8.2, 7.0 8.8, 8.0 7.6, 6.9 6.0, 5.5 3+3: 0.53 NA 4.3% NA 4.6% NA NA Safety Lead- (4-21) (4-21) (2.5-20.4) (2.4-17.0) IQ3: 0.56 1 18.0% 1 18.3% 1 10.8% in Phase I 2 77.8% 2 77.1% 2 13.6% Scenario C1 26.7, 26.0 29.9, 29.0 34.2, 33.3 24.5, 23.6 3+3: 0.48 NA <1% NA <1% NA NA nd 2 Cycle with (11-62) (11-72) (11.2-90.2) (9.9-65.3) IQ3: 0.51 1 <1% 1 1.3% 1 5.9% experimental 2 7.8% 2 8.1% 2 6.7% combination 3 13% 3 13% 3 7.6% 4 78.1% 4 77.4% 4 9.0% Scenario C2 26.7, 26.0 29.6, 29.0 39.5, 38.4 29.1, 28.2 3+3: 0.47 NA <1% NA <1% NA NA nd 2 Cycle (11-71) (13-72) (11.4- (12.0-75.0) IQ3: 0.48 1 1% 1 <1% 1 5.9% 100.4) 2 6.5% 2 8.1% 2 6.7% 3 12.3% 3 13.3% 3 7.6% © 2020 Frankel PH et al. JAMA Network Open. 4 79.0% 4 77.8% 4 9.0% Scenario C3 13.9, 13.0 16.5, 16.0 18.2, 17.6 14.6, 14.3 3+3: 0.32 NA <1.0% NA <1.0% NA NA nd 2 Cycle (8-25) (8-32) (9.0-40.0) (8.2-29.1) IQ3: 0.36 1 <1.0% 1 <1.0% 1 5.9% 2 6.5% 2 6.4% 2 6.7% 3 9.0% 3 10.6% 3 7.6% 33% 4 84.4% 4 82.6% 4 9.0% Scenario D 22.5, 23.0 25.3, 26.5 42.1, 42.9 31.9, 32.9 3+3:.71 NA 3.6% NA 3.8% NA NA IP Phase I (2-36) (2-40) (3.5-81.5) (3.5-57.8) IQ3:.75 1 3.5% 1 3.6% 1 5.3% 2 4.6% 2 4.1% 2 5.9% 3 5.9% 3 7.4% 3 6.7% 4 6.5% 4 6.6% 4 7.6% 5 10.9% 5 10.9% 5 9.0% 6 65.0% 6 63.6% 6 10.8% eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). Variation #Treated #Treated #Months #Months Mean Rolling 6 IQ R6 DLT rate Rolling 6 IQ R6 Rolling 6 IQ R6 #DLTs % MTD % MTD (based Mean,Med Mean,Med Mean,Med Mean, Med above At level At level on model) (range) (range) (range) (range) MTD Scenario A1 23.9, 26.0 23.3, 24.0 16.4,16.9 13.0,13.0 R6 : 0.89 NA <1.0% NA <1.0% NA NA Ave Ineval (8-39) (7-38) (4.9-38.0) (3.1-23.2) IQR: 0.94 1 7.6% 1 7.8% 1 6.7% Phase I 2 9.0% 2 8.9% 2 7.6% 20% Ineval 3 12% 3 13% 3 9.0% 4 14.9% 4 15.4% 4 10.8% 5 56.0% 5 54.9% 5 13.6% Scenario A2 20.5,23.0 20.1,21.0 14.0,14.4 11.4,11.6 R6 : 0.90 NA <1.0% NA <1.0% NA NA Low Ineval (8-28) (7-33) (4.1-26.2) (3.2-19.8) IQR: 0.88 1 6.5% 1 6.5% 1 6.7% Phase I 2 9.3% 2 9.3% 2 7.6% 3.6% Ineval 3 13.1% 3 11.9% 3 9.0% 4 14.9% 4 14.0% 4 10.8% 5 55.8% 5 57.8% 5 13.6% Scenario A3 31.5,33.0 30.0,31.0 21.4,21.7 16.4,16.3 R6 : 1.06 NA 1.0% NA 1.1% NA NA High Ineval (8-59) (9-51) (5.6-50.6) (5.3-31.6) IQR:1.13 1 8.1% 1 7.9% 1 6.7% Phase I 2 14.3% 2 12.3% 2 7.6% 44% Ineval 3 14.0% 3 15.5% 3 9.0% 4 14.3% 4 17.0% 4 10.8% 5 48.4% 5 46.3% 5 13.6% Scenario A4 23.1,24.0 21.8,22.0 25.2,25.3 19.8,19.5 R6 : 0.93 NA <1.0% NA < 1.0% NA NA Ave Ineval (8-39) (7-37) (5.7-56.1) (4.3-42.7) IQR:0.94 1 7.5% 1 7.1% 1 6.7% Phase I 2 8.1% 2 8.6% 2 7.6% 3 13.8% 3 12.9% 3 9.0% 60% 4 15.4% 4 15.8% 4 10.8% 5 54.5% 5 55.3% 5 13.6% Scenario A5 23.2,25.0 22.1,23.0 20.7,21.0 17.1,17.1 R6 : 0.91 NA < 1.0% NA < 1.0% NA NA Ave Ineval (8-37) (9-38) (5.2-38.8) (4.9-34.9) IQR:0.94 1 7.3% 1 6.9% 1 6.7% Phase I 2 8.6% 2 9.4% 2 7.6% 3 12.8% 3 12.6% 3 9.0% 15 day mean) 4 15.4% 4 15.1% 4 10.8% 5 55.5% 5 55.4% 5 13.6% Scenario A6 23.5,25.0 22.5,23.0 15.4,15.7 12.1,12.1 R6 : 0.91 NA < 1.0% NA < 1.0% NA NA Ave Ineval (8-36) (9-38) (4.2-35.0) (3.7-23.2) IQR: 0.91 1 7.8% 1 7.5% 1 6.7% Phase I 2 8.4% 2 8.5% 2 7.6% 3 13.1% 3 12.5% 3 9.0% days 4 14.6% 4 14.5% 4 10.8% 5 55.6% 5 56.8% 5 13.6% Scenario A7 18.0,17.0 18.7,18.0 12.3,11.6 10.7,10.1 R6 : 2.01 NA 3.5% NA 3.9% NA NA Ave Ineval (4-34) (4-37) (3.4-28.3) (2.7-22.9) IQR:2.16 1 18.8% 1 17.4% 1 10.8% Phase I 2 26.1% 2 24.8% 2 13.6% 3 26.8% 3 28.3% 3 18.0% 4 19.5% 4 21.1% 4 25.9% 5 5.4% 5 4.6% 5 40.3% Scenario B 8.3, 7.0 9.0, 8.0 5.5, 4.9 5.1, 4.7 R6 : 0.53 NA 3.3% NA 4.3% NA NA Safety Lead- (4-21) (4-22) (1.6-15.8) (1.6-16.4) IQR: 0.57 1 18.9% 1 18.9% 1 10.8% in Phase I 2 77.9% 2 76.9% 2 13.6% Scenario C1 31.1, 31.0 31.1, 30.0 24.9, 23.9 18.7, 18.0 R6 : 0.48 NA <1.0% NA <1.0% NA NA nd 2 Cycle with (11-71) (13-72) (7.3-57.4) (7.3-46.2) IQR: 0.47 1 <1.0% 1 <1.0% 1 5.9% experimental 2 9.6% 2 7.5% 2 6.7% combination 3 12.0% 3 12.8% 3 7.6% 4 77.5% 4 78.8% 4 9.0% © 2020 Frankel PH et al. JAMA Network Open. Scenario C2 30.7, 30.0 30.0, 29.0 30.4, 29.1 23.9, 23.0 R6 : 0.49 NA <1.0% NA <1.0% NA NA nd 2 Cycle (11-63) (13-73) (8.6-77.3) (8.5-55.8) IQR: 0.47 1 1.0% 1 <1.0% 1 5.9% 2 8.8% 2 7.9% 2 6.7% 3 12.6% 3 12.6% 3 7.6% 4 77.4% 4 78.5% 4 9.0% Scenario C3 16.7, 16.5 17.7, 17.0 13.8, 13.5 11.2, 10.9 R6 : 0.36 NA <1.0% NA <1.0% NA NA nd 2 Cycle (9-27) (9-32) (5.8-29.6) (5.2-21.1) IQR: 0.39 1 <1.0% 1 <1.0% 1 5.9% 2 7.3% 2 7.0% 2 6.7% 3 9.8% 3 11.0% 3 7.6% 33% 4 82.8% 4 81.6% 4 9.0% Scenario D 30.0, 34.0 26.7, 28.0 36.1, 38.6 25.6, 26.6 R6 : 0.78 NA 4.4% NA 4.0% NA NA IP Phase I (2-41) (2-40) (3.1-76.9) (3.1-45.7) IQR: 0.74 1 4.6% 1 4.5% 1 5.3% 2 6.4% 2 4.4% 2 5.9% 3 6.0% 3 7.3% 3 6.7% 4 7.1% 4 6.3% 4 7.6% 5 10.5% 5 10.0% 5 9.0% 6 61.0% 6 63.6% 6 10.8% eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). Based on Model #Treated #Treated #Treated #Treated 3+3 IQ 3+3 Rolling 6 IQ R6 Scenario A1 with 3+3 IQ3+3 Rolling 6 IQ R6 % MTD % MTD % MTD % MTD an additional 4 Mean=25.6 Mean=28.4 Mean=31.6 Mean=30.3 At level At level At level At level higher dose levels Median=26.0 Median=28.5 Median=33.0 Median=31.0 Range 8-53 Range 8-58 Range 8-62 Range 7-66 Dose DLT level Rate Average Average Average Average fraction at Fraction at Fraction at Fraction at dose level: dose level: dose level: dose level: *NA NA -- -- -- -- <1.0% <1.0% <1.0% <1.0% 1 6.7% 0.04 0.04 0.04 0.04 7.4% 7.3% 7.6% 7.8% 2 7.6% 0.22 0.24 0.26 0.24 8.4% 9.3% 9.0% 8.9% 3 9.0% 0.19 0.21 0.22 0.22 9.9% 12.4% 12.1% 13.0% 4 10.8% 0.17 0.17 0.17 0.17 14.3% 14.1% 14.8% 13.8% 5 13.6% 0.14 0.14 0.13 0.13 17.3% 18.6% 14.8% 17.8% 6 18.0% 0.11 0.11 0.09 0.10 21.1% 20.9% 22.0% 20.8% 7 25.9% 0.08 0.07 0.06 0.06 16.6% 14.6% 15.6% 14.0% 8 40.3% 0.04 0.02 0.02 0.02 4.7% 2.1% 3.4% 3.8% 9 59.7% 0.01 <0.01 <0.01 <0.01 <1% <1% <1% <1% *NA represents when dose level 1 was above the MTD Study Duration (months) 3+3 IQ3+3 Rolling 6 IQ R6 Study Duration (Mean or Expected) 26.0 20.6 21.5 16.8 Study Duration Median (Range) 25.7 20.7 22.2 16.7 (7.1-63.9) (5.3-42.6) (4.9-45.9) (3.1-36.4) #Started Treatment 25.6 28.4 31.6 30.4 (Mean or Expected) #Started Treatment 26.0 28.5 33 .0 31.0 Median (Range) (8-53) (8-58) (8-62) (7-66) #DLTs above MTD 2.2 2.2 2.0 2.2 (Mean or Expected) #DLTs above MTD 2.0 2.0 2.0 2.0 Median (range) (0-7) (0-6) (0-5) (0-7) eAppendix. Design Rationale: IQ 3+3 Rationale: Row 4 differences were discussed in the manuscript. Additionally, When 2 patients have a “Pass” (fully assessed and evaluable with no DLT) a fifth patient can be accrued without © 2020 Frankel PH et al. JAMA Network Open. exceeding the 3+3 risks. In that situation the worst case scenario in the 3+3 is that third patient has a DLT, in which case, 3 additional patients can be accrued. As a result, with 2 patients without a DLT, we are within the risk limits to accrue up to 4 additional patients, allowing us to get to 6 enrolled, with 2 evaluable each with a “Pass” (rows 7-8). In row 11, with 4-6 treated and 3-5 evaluable with no DLT, the 3+3 would have previously escalated if 0/3, so the 3+3 could only be in that situation if expanding to 6 patients with the dose above closed. That is not automatically the case with the IQ 3+3, however, we would know that there were no DLTs in at least 3 patients, and no DLTs in the pending patients reported to date, so if the above level is open, we can escalate, with a stronger safety signal than with only information on 3 patients, hence reduced risk to future patients. The next major difference (row 16) occurs with three patients accrued (consented and promised a slot), 1 DLT and 1 patient with a “Pass” (or 1 of the 2 evaluable patients with a DLT). The 3+3 holds accrual in that setting, whereas the IQ 3+3 can th accrue a 4 patient. In the standard 3+3, if 1 of 1 patients have a DLT, 2 additional patients can be accrued, so we are within that risk to have 2 patients accrued with 1 of 2 patients with a DLT, again, not exceeding the risks of the 3+3. Risk-based logic also allows the IQ 3+3 to accrue additional patients with 1 DLT, and 4 or 5 patients evaluable and up to 2 patients pending (rows 20, 21). The IQ 3+3 also allows escalation with 1 out of 6 evaluable patients with DLTs and no DLTs reported in pending patients. To consider why eight patients are permitted on the IQ 3+3 consider: If we have 3 patients enrolled, 2 with a “PASS” (meaning fully evaluable with no DLT), and 1 DLT, we can enroll th another 3 patients per the standard 3+3. If the 4 patient is a “PASS”, we are safer with 1 of 4 DLT than 1 of 3, so we are not exceeding the risks to still enroll 3 patients. That means we could th get to 7 patients. If the 5 patient is “PASS”, the same argument can be made, and we get to 8 th patients. However, if a 6 patient is a “PASS” we escalate, so we don’t need more than 8 slots © 2020 Frankel PH et al. JAMA Network Open. during escalation. Also note that if the patient consented and was promised a slot based on a dose level but has not started treatment when an escalation has been declared, that patient is put on the currently accruing (higher) dose level, and this is modelled in the simulations. To address DLTs occurring in pending patients at a lower dose level after escalation we note that information would not have been observed in the 3+3, so any conservative action reduces the risk to future patients when compared to the original 3+3. IQ Rolling 6 Where the original Rolling 6 allows 6 patients to be put at risk at the same time, the IQ Rolling 6 maintains that risk limit or lower, and when 1 patient has a “Pass” or a second patient has a “Pass”, an additional 1 or 2 patients, respectively, can be accrued to maintain 6 patients at risk when there are no DLTs. If 1 patient has a DLT, the Rolling 6 permits accrual of up to 5 additional patients, as does the IQ Rolling 6. If 1 patient has a DLT, the IQ Rolling 6 as presented is not maximally aggressive. When there is a DLT, we revert to the risk rules of the 3+3 for accruing beyond 6 patients due to the conditions for using the Rolling 6 being in question in the presence of a DLT . Additional Details: Note 1: Some decisions in Table 1-2 required special notes. For example, the traditional 3+3 design and the Rolling 6 design do not specify what to do with 2 DLTs in 7 patients or 2 DLTs in 8 patients (row 26 and 28 in Table 1 and rows 27 and 29 in Table 2). We first addressed this situation in the context of the NCI-CTEP sponsored study (NCT02568553, Table S1). Considerations © 2020 Frankel PH et al. JAMA Network Open. included (1) the principal investigator did not want to be obligated to lower the dose if 2 of 7 patients had DLTs and wanted the option to continue to treat at that dose (but not escalate), suggesting that dose as the MTD; (2) additional principal investigators agreed; (3) as recommended , the study had additional monitoring rules for the expansion cohort so that if 3 or more DLTs occurred in the first 12 patients treated at that dose, the recommended Phase 2 dose and continued accrual would be re-evaluated; (4) Children’s Oncology Group has set a precedent allowing the possibility of cohort expansion when 2 DLTs occur in 6 evaluable patients if the two DLTs are of different classes (e.g. hepatotoxicity and myelosuppression) with internal review (personal communication); and (5) we searched for logic consistent with the decisions inherent in the 3+3 design using two beta distributions for the probability of a DLT (analogous to an optimistic and pessimistic prior) and two decision thresholds for escalation or de-escalation. We challenged those sets of rules consistent with the 3+3 with 2 DLTs out of 7 patients. We found that if 1/6 was an acceptable DLT rate not requiring de-escalation, the decision would never be to de-escalate (see Note 3 below) resulting in 2 DLTs out of 7 or 8 patients constituting the MTD in our decision grid for the IQ 3+3 and the IQ Rolling 6. Note 2: We currently allow the protocol team to evaluate the specifics of the lower level DLT(s), and permit the team to immediately reduce the dose on the patients at the higher dose or allow patients on the higher dose level to complete their evaluation and then decide what action to take. The simulations do not consider such flexible decision-making and only considers those DLTs that occurred on a lower dose after escalating if later de-escalation re-visits that dose. The frequency of two DLTs at a dose below the MTD in the simulations in either the IQ 3+3 or the IQ Rolling 6 is rare (~2%). © 2020 Frankel PH et al. JAMA Network Open. Note 3: R-CODE for 2 of 7 Question comment<- c(" • The Beta distribution is a flexible distribution for the probability of a DLT. • This will be used for the probability of a DLT (p) using parameters a, b, e.g. Beta(a,b) • The expected value of p, E(p) = a/(a+b). • We update the expected value of p with patient data on a dose according to: • E(p) = (a+ #DLTs)/ (a+#Pts+b), (e.g. if a=b=1 with no patients, E(p)=1/2) . • To generate a logical decision engine that agrees with 3+3 based on two priors: A) One prior, beta(a1,b1) that guides whether the dose is safe and we can continue to treat. Based on a threshold (T1) for the expected DLT rate B) One prior, beta(a2,b2) guides whether we are comfortable escalating the dose. Based on a different threshold (T2) for the expected DLT rate. (a1+#DLTS)/(a1+#PTS+b1)>T1? Go Down (a2+#DLTS)/(a2+#PTS+b2)<T2? Go Up Otherwise stay • 55,000,000 random samples of parameters to find two priors beta(a1,b1), beta(a2,b2), and two threshold (T1,T2), that are consistent with the traditional 3+3 rules. We used a general grid search. • Find sets of parameters matched the 3+3 • Apply all different sets of rules to 2 DLTs of 7 patients. (a1+2)/(a1+7+b1)>T1? Go Down ..else (a2+2)/(a2+7+b2)<T2? Go Up …else MTD ") # Use 64-BIT R to allow larger number of simulations set.seed(9) simnum<-55000000 # 55 million #(a1+2)/(a1+7+b1)>T1? Go Down ..else #(a2+2)/(a2+7+b2)<T2? Go Up …else MTD a1=runif(simnum,min=0.00,max=13) # rep(0,simnum) #rbinom(simnum,10,0.2) b1= runif(simnum,min=0.0,max=50) a2=runif(simnum,min=0.0,max=45) b2=runif(simnum,min=0,max=45) t1=runif(simnum,min=0,max=1/2) # 1/6 was key t2= runif(simnum,min=0,max=1/2) # 1/3 © 2020 Frankel PH et al. JAMA Network Open. ###### down<-function(ndlt,tot) {return( ((a1+ndlt)/(a1+tot+b1))>t1 ) } up<-function(ndlt,tot) {return( ((a2+ndlt)/(a2+tot+b2))<t2 ) } # Flawed with 0 DLTS z0<-( down(0,0) | up(0,0) | down(0,1) | up(0,1) | down(0,2) | up(0,2) | down(0,3) | !up(0,3) | down(0,4) | !up(0,4) | down(0,5) |!up(0,5) | down(0,6)| !up(0,6) ) # Flawed with 1 DLT z1<- ( down(1,1) | up(1,1) | down(1,2) | up(1,2) | down(1,3) | up(1,3) | down(1,4) | up(1,4) | down(1,5) | up(1,5) | down(1,6) | !up(1,6) ) # Flawed with 2 DLTs z2<-( !down(2,2) | up(2,2) | !down(2,3) | up(2,3) | !down(2,4) | up(2,4) | !down(2,5) | up(2,5) | !down(2,6) | up(2,6) ) # Flawed with 3 DLTs z3<- (!down(3,3) | up(3,3) | !down(3,4) | up(3,4) | !down(3,5) | up(3,5) | !down(3,6) | up(3,6) ) # Flawed with 4 DLTs z4<- (!down(4,4) | up(4,4) | !down(4,5) | up(4,5) | !down(4,6) | up(4,6) ) # Flawed with either 0, 1, ,2 ,3 or 4 DLTs (not consistent with 3+3) bad<- (z0 | z1 | z2 | z3 | z4) # Gather up data tot<-data.frame(bad, a1, a2, b1, b2, t1, t2, prior1=a1/(a1+b1), prior2=a2/(a2+b2)) # Select parameters that are consistent with the 3+3 tot2<-tot[!tot$bad,] dim(tot2)[1]/simnum # Of paramters consistent with 3+3, answer is 1297 with above conditions and seed dim(tot2)[1] ############################################################################## ####### downcheck<-function(ndlt,tot,a1c,b1c,t1c) {return( ((a1c+ndlt)/(a1c+tot+b1c))>t1c ) } upcheck<-function(ndlt,tot,a2c,b2c,t2c) {return( ((a2c+ndlt)/(a2c+tot+b2c))<t2c) } # Ask what those 1297 sets of parameters would do with 2 DLTs out of 7 patients tot3<-data.frame(tot2, d2of7=downcheck(2,7,tot2$a1,tot2$b1,tot2$t1), u2of7=upcheck(2,7,tot2$a2,tot2$b2,tot2$t2) ) table(tot3$u2of7) # 0/1297 go up © 2020 Frankel PH et al. JAMA Network Open. table(tot3$d2of7) # 959/1297 go down BUT downtot<-tot3[tot3$d2of7,] summary(downtot$t1) # All decisions to go down had the threshold to go down less than 1/6 # If 1/6 is an acceptable rate, the stay option is selected. # Refining the grid to increase the number of parameter sets consistent with the 3+3 does not # Alter the conclusion. © 2020 Frankel PH et al. JAMA Network Open. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JAMA Network Open American Medical Association

Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies

Loading next page...
 
/lp/american-medical-association/model-of-a-queuing-approach-for-patient-accrual-in-phase-1-oncology-4YRixGdxsZ
Publisher
American Medical Association
Copyright
Copyright 2020 Frankel PH et al. JAMA Network Open.
eISSN
2574-3805
DOI
10.1001/jamanetworkopen.2020.4787
Publisher site
See Article on Publisher Site

Abstract

Key Points Question Can the duration of phase 1 IMPORTANCE Phase 1 cancer studies, which guide dose selection for subsequent studies, are studies using either the 3 + 3 or rolling 6 almost 3 times more prevalent than phase 3 studies and have a median study duration considerably designs be substantially reduced longer than 2 years, which constitutes a major component of drug development time. without exceeding the patient risk limits or changing the operating OBJECTIVE To discern a method to reduce the duration of phase 1 studies in adult and pediatric characteristics of the parent design? cancer studies without violating risk limits by better accommodating the accrual and evaluation Findings This decision analytical model process (or queue). found that the modified study designs were associated with reduced expected DESIGN The process modeled, the phase 1 queue (IQ), includes patient interarrival time, screening, study durations. The modified designs and dose-limiting toxicity evaluation. For this proof of principle, the rules of the 3 + 3 and rolling 6 were associated with minimal changes phase 1 designs were modified to improve patient flow through the queue without exceeding the in the number of patients treated and maximum risk permitted in the parent designs. The resulting designs, the IQ 3 + 3 and the IQ rolling the determination of the maximum 6, were each compared with their parent design by simulations in 12 different scenarios. tolerated dose, without changing the operating characteristics or exceeding MAIN OUTCOMES AND MEASURES (1) The time from study opening to determination of the the risk limits of the parent design. maximum tolerated dose (MTD), (2) the number of patients treated to determine the MTD, and (3) the association of the design with the dose selected as the MTD. Meaning Per this analysis, a substantial reduction in the time required to bring RESULTS Based on 800 simulations, for all 12 scenarios considered, the IQ 3 + 3 and the IQ rolling 6 new advances to the clinic can be designs were associated with reduced expected study durations compared with the parent design. accomplished by simple modifications The expected IQ 3 + 3 reduction ranged from 1.6 to 10.4 months (with 3.7 months for the standard of 2 commonly used phase 1 scenario), and the expected reduction associated with IQ rolling 6 ranged from 0.4 to 10.5 months trial designs. (with 3.4 months for the standard scenario). The increase in the mean number of patients treated in the IQ 3 + 3 compared with the 3 + 3 ranged from 0.6 to 3.2 patients. No increase in the number of Supplemental content patients was associated with the IQ rolling 6 compared with the rolling 6 design. The probability of selecting a dose level as the MTD changed by less than 3% for all dose levels and scenarios in both Author affiliations and article information are listed at the end of this article. parent designs. CONCLUSIONS AND RELEVANCE This study found that IQ designs were associated with reduced mean duration of phase 1 studies compared with their parent designs without changing the risk limits or MTD selection operating characteristics. These approaches have been successfully implemented in both hematology and solid tumor phase 1 studies. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 Introduction 1,2 While clinical trial timeliness factors into cost and relevance, phase 1 oncology trials are often underappreciated for their association with the time to translate a therapeutic advance into practice. Unlike most phase 1 studies, which take less than 1 year, phase 1 oncology studies take a median of approximately 32 months after study activation, and the number of phase 1 cancer clinical trials Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 1/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies is consistently almost 3 times the number of phase 3 studies (eTable 1 in the Supplement). In addition, while multiple phase 2 and 3 studies can run in parallel, they must wait for the relevant phase 1 study to complete. Because phase 1 study duration is often slot limited and minimally affected by adding sites, we must look elsewhere for opportunities to alter this early bottleneck that delays therapeutic advances in oncology. Here, we focus on the patient queue and evaluation process to reduce phase 1 study duration. As a proof of principle, we focus on the traditional 3 + 3 phase 1 design and the rolling 6 design often used in pediatric studies. To our knowledge, this represents the most beneficial modification of 2 of the most commonly used designs (1 for adults and 1 for pediatric populations) to reduce expected study duration without affected the operating characteristics. These new designs have been successfully implemented in several completed and ongoing clinical trials. Methods Phase 1 oncology designs typically consider only the first cycle of experimental treatment in both the determination of dose-limiting toxicities (DLTs) and the formal guidelines for dose-escalation decisions. To limit the number of patients at risk for a DLT, most phase 1 designs restrict the number of patients enrolled on the current dose level during the first cycle of experimental therapy. This is our starting point for the study of the phase 1 queue. The activities performed for the purposes of this simulation study are not considered to be human subjects research (per US Department of Health and Human Services regulation under the 45 CFR 46 Common Rule). Figure 1 illustrates the major steps involved in a phase 1 trial that form the basis for the patient queue that are reflected in the simulation tool. After each patient provides consent or any change occurs in any patient’s evaluation status, the protocol team decides the dose level and availability of slots for the accrual of additional patients based on the accumulated data and the protocol- specified phase 1 design. The decision can be to escalate (accrue at the next higher dose level), accrue at the same dose level, deescalate (accrue at the next lower dose level), hold accrual, or end accrual and either declare a maximum tolerated dose (MTD) or declare that the lowest dose level tested is too toxic. The phase 1 design guides these decisions. We modified the 3 + 3 design and rolling 6 design decisions to better accommodate the phase 1 queue. The modifications were evaluated on the time and number of patients required to determine the MTD while constrained to (1) inherit the maximum level of patient risk associated with the parent design, (2) provide at least an equally rigorous assessment of toxicity, (3) maintain the parent design’s operating characteristics with respect to the MTD determination, and (4) revert to the parent design if accrual is slow or the principal investigator chooses to delay securing consent from patients when the queue-based designs permit accrual but the parent design does not. 7,8 The respective queue-based modifications (Table 1 and Table 2) are denoted the IQ 3 + 3 and the IQ rolling 6 designs. At any time, on the current dose level, there is the total number of patients enrolled (eg, promised or taking a slot, excluding those deemed inevaluable for DLT determination), the number of DLTs, and the number of patients evaluable. The number evaluable represents patients who were fully assessed and either had a DLT or did not (termed a pass), whereas the difference between total and evaluable numbers represents patients whose evaluations are pending. The column for the 3 + 3 design can been seen as representing the traditional rules: accrue 3 patients, escalate with 0 DLTs, deescalate with 2 DLTs, and expand to 6 patients with 1 DLT (in which 2 or more DLTs results in dose deescalation and 1 DLT in 6 patients results in dose escalation if the next higher dose is open, and the MTD requires 6 patients treated with 1 DLT at most). Both IQ-based modifications can have up to 8 patients per dose level during the escalation phase. In addition, when deescalating, applying the rule that all patients (up to as many as 10 patients, which occurs in <0.1% of simulations) should not be denied treatment once given a consent form are possible in both IQ designs. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 2/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Figure 1. Representations of Patient Flow Through a Typical Phase 1 Trial A Phase 1 clinical trials simulation: process flow diagram Patient arrives (patients arrive as Screening Screen failure, Consent possible test Yes Yes Is a slot Screen patient exits (screening (patient is accepted candidates per available? failure? duration sampled (patient to leave as a candidate) a stochastic from distribution) model) interarrival time distribution) No No Patient waits Start test Determine (patient waits for (patient to disposition a maximum available slot for Yes tolerable wait current dose level) time sampled from a waiting distribution) Inevaluable DLT Pass Willing to Delay until Delay until Delay until wait further inevaluable DLT test complete No Inevaluable event, DLT event Test complete patient exits (patient stays in (patient stays in Patient exits slot) slot) (patient to leave (patient removed slot and model) from wait list and leaves model) B Screenshot of simulation software reflecting patient flow A, The process accounts for 3 outcomes. Patients who receive protocol-specified deescalation of the rolling 6 design and queue-based modified designs, so all consenting minimum therapy without a dose-limiting toxicity (DLT) pass. Patients with a DLT are patients are treated without delay. Patients go to the waitlist or screening (yellow). If designated DLT. Inevaluable denotes ones who do not meet criteria for pass or DLT, often they exceed maximum wait time or do not meet criteria, they exit. Patients meeting because of inadequate treatment or insufficient follow-up. Disposition is found by criteria take a bed, are treated, and are pending (blue). Patients who pass are green; with randomly sampling times of occurrence of inevaluable and DLT distributions. If both DLTs, red. If inevaluable, they exit. A summary data dashboard for many of the same occur, the disposition is the first event. B, Simulation software showing patient flow. scenario and various design parameters is at right (https://www.flexsim.com/clinical-trials). Each dose level has 2 cohorts of 3 beds, plus 2 extra beds for use in queue-based MAD indicates maximum administered dose; MTD, maximum tolerated dose; TDPS, total, modifications during escalation. There are 2 more beds (gray) for rare instances during DLTs, pending, status (dose above closed). JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 3/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Table 1. Comparison of the 3 + 3 and the Phase 1 Queue 3 + 3 Design Decisions No. on current level Dose level for next patient Dose-limiting b c d e e Row Total Evaluable Toxicity IQ 3+3 3+3 1 0-2 0 0 Same dose level Same dose level 2 3 0 0 Hold accrual Hold accrual 3 1-2 1 0 Same dose level Same dose level 4 3 1 0 Same dose level Hold accrual 5 4 1 0 Hold accrual Not allowed 6 2 2 0 Same dose level Same dose level 7 3 2 0 Same dose level Hold accrual 8 4-5 2 0 Same dose level Not allowed 9 6 2 0 Hold accrual Not allowed f f 10 3 3 0 Escalate Escalate 11 4-6 3-5 0 Escalate Same (up to 6; implies that higher dose cannot be tested further) 12 6 6 0 Escalate (or MTD) MTD 13 1-2 1 1 Same dose level Same dose level 14 3 1 1 Hold accrual Hold accrual 15 2 2 1 Same dose level Same dose level 16 3 2 1 Same dose level Hold accrual 17 4 2 1 Hold accrual Not allowed 18 3-5 3-5 1 Same dose level Same dose level 19 6 3 1 Hold accrual Hold accrual 20 6 4 1 Same dose level Hold accrual 21 6 5 1 Same dose level Hold accrual 22 7 4 1 Hold accrual Not allowed 23 7 5 1 Same dose level Not allowed 24 6-8 6-8 1 Escalate Escalate (patients 7 and 8 not allowed) 25 2-7 2-6 2 Deescalate Deescalate (patient 7 not allowed) 26 7 7 2 MTD Not allowed 27 8 7 2 Hold accrual Not allowed 28 8 8 2 MTD Not allowed g g 29 Any Any 3 Deescalate Deescalate Abbreviations: IQ, phase 1 queue; MTD, maximum tolerated dose. The IQ design and the corresponding parent design are side by side. Rare scenarios are not listed, but the full decision grid 7,8 is available in the software input module available. The number of patients who consented to that dose level, excluding individuals who did not meet screening criteria or patients considered inevaluable with respect to dose-limiting toxicity. The number of patients who provided an answer to the dose-limiting question (yes or no). The number of patients who experienced a dose-limiting toxicity. The number pending is the difference between the total number and the number evaluable. The action to be taken for the next patient for the IQ design and the parent design, respectively. If a patient pending evaluation on a lower dose experiences a dose-limiting toxicity, the principal investigator in consultation with the sponsor may choose to reduce the dose level of any patients currently on a higher dose level, pending review of the adverse event data. If the next higher dose level is not available (there is no higher dose level or the higher dose level was already tested and found to be too toxic), a maximum of 8 patients can be treated at the current dose level and the principal investigator should declare the MTD with 0 or 1 dose-limiting toxicity of 6 (or 0 of 5) patients. For IQ 3 + 3, no more than 4 patients at risk are allowed, with no more than 6 patients at risk for the IQ rolling 6 design. Two dose-limiting toxicities in 7 or 8 patients means that the principal investigator can also declare the MTD (and it is suggested to continued using monitoring rules for the expanded cohort). Current level exceeds the MTD. The MTD is the highest level at which less than 33% of patients had dose-limiting toxicities, with at least 6 patients evaluable. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 4/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies As an example modification, for the IQ 3 + 3 design (Table 1; row 4), enrolling a fourth patient at the same dose level, knowing that 1 of 3 patients has a pass with 2 patients pending, is considered less risky than putting 3 patients at risk without any patient data, as is permitted by the 3 + 3 design, so this modification does not exceed the risk allowed by the 3 + 3 but rather starts a patient through the process in case a patient who started earlier is considered not to meet screening criteria or becomes inevaluable. This is the key principle behind the IQ 3 + 3 design. Additional details and rationale for special IQ 3 + 3 decisions can be found in the eAppendix in the Supplement, including why and when up to 8 patients can be accrued during dose-level escalation. For the IQ rolling 6 design (Table 2), the parent design allows 6 patients to be put at risk but does not allow escalation of the first 3 patients pass when there are patients pending. As a result, the rolling 6 does not achieve its anticipated speed advantage. The IQ rolling 6 allows escalation with 3 or more patients who are treated and fully evaluated with no DLTs, with the additional knowledge that none of the patients whose data are pending or have begun treatment have reported a DLT, which is consistent with the rules for the IQ 3 + 3 design with patients pending. The decision rules have also been modified so the IQ rolling 6 design improves patient flow but does not exceed the maximum risk permitted in the rolling 6 design (eAppendix in the Supplement). Table 2. Comparison of Rolling 6 and Phase 1 Queue Rolling 6 Design Decisions Abbreviations: IQ, phase 1 queue; MTD, maximum tolerated dose. No. on current level Dose level for next patient Dose-limiting The IQ design and the corresponding parent design b c d e e Row Total Evaluaable toxicities IQ rolling 6 Rolling 6 are side by side. Rare scenarios are not listed, but the 1 0-5 0 0 Same dose level Same dose level full decision grid is available in the software input 7,8 2 6 0 0 Hold accrual Hold accrual module available. 3 1-5 1 0 Same dose level Same dose level The number of patients who consented to that dose level, excluding individuals who did not meet 4 6 1 0 Same dose level Hold accrual screening criteria or patients considered inevaluable 5 7 1 0 Hold accrual Not allowed with respect to dose-limiting toxicity. 6 2-5 2 0 Same dose level Same dose level The number of patients who provided an answer to 7 6-7 2 0 Same dose level Hold accrual (patient 7 not the dose-limiting question (yes or no). allowed) 8 8 2 0 Hold accrual Not allowed The number of patients who experienced a dose- limiting toxicity. The number pending is the 9 3 3 0 Escalate Escalate difference between the total number and the 10 4-5 3 0 Escalate Same dose level number evaluable. 11 6 3 0 Escalate Hold accrual The action to be taken for the next patient for the IQ 12 7-8 3 0 Escalate Not allowed design and the parent design, respectively. If a 13 4 4 0 Escalate Escalate patient pending evaluation on a lower dose 14 5 4 0 Escalate Same dose level experiences a dose-limiting toxicity, the principal investigator in consultation with the sponsor may 15 6-8 4 0 Escalate Hold accrual (patients 7 and 8 not allowed) choose to reduce the dose level of any patients f f 16 5 5 0 Escalate (or MTD) Escalate (or MTD) currently on a higher dose level, pending review of f f the adverse event data. 17 6 5 0 Escalate (or MTD) Escalate (or MTD) If the next higher dose level is not available (there is 18 7-8 5 0 Escalate (or MTD) Not allowed no higher dose level or the higher dose level was 19 1-5 1-5 1 Same dose level Same dose level already tested and found to be too toxic), a 20 6 1-3 1 Hold accrual Hold accrual maximum of 8 patients can be treated at the current 21 6 4 1 Same dose level Hold accrual dose level and the principal investigator should 22 6 5 1 Same dose level Hold accrual declare the MTD with 0 or 1 dose-limiting toxicity of 6 (or 0 of 5) patients. For IQ 3 + 3, no more than 4 23 7 4 1 Hold accrual Not allowed patients at risk are allowed, with no more than 6 24 7 5 1 Same dose level Not allowed patients at risk for the IQ rolling 6 design. Two dose- 25 6-8 6-8 1 Escalate Escalate (patients 7 and 8 not f limiting toxicities in 7 or 8 patients means that the allowed) g principal investigator can also declare the MTD (and 26 2-8 2-6 2 Deescalate Deescalate (patients 7 and 8 not it is suggested to continued using monitoring rules allowed) for the expanded cohort). 27 7 7 2 MTD Not allowed Current level exceeds the MTD. The MTD is the 28 8 7 2 Hold accrual Not allowed highest level at which less than 33% of patients had 29 8 8 2 MTD Not allowed dose-limiting toxicities, with at least 6 patients g g 30 Any any 3 Deescalate Deescalate evaluable. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 5/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Using decision tables 1 and 2, the operating characteristics of the 3 + 3 and IQ 3 + 3 designs and the rolling 6 and IQ rolling 6 designs were evaluated for 12 scenarios detailed in Table 3, each motivated by our clinical trial experience. In each scenario, we specify a starting dose level, the lowest and highest dose levels, and other parameters (Table 3). The maximum waiting time was assumed to be 0 in the simulations, representing patients who will not wait if slots are unavailable and instead will be accrued to a different phase 1 study (or treated outside of a study). Scenarios A1 through A3 (Table 3) are based on our experience with phase 1 studies in the California Cancer Consortium, and scenarios A4 through A7 explore the results of changes in the screen failure rate, the interarrival time distribution, the course length, and the dose-toxicity association. For scenarios A1 through A3, we collected data on 14 phase 1 (or dose-finding portions of phase 2) cancer studies that had completed their phase 1 portions between October 2004 and November 2014 in the California Cancer Consortium. Along with the more traditional 3 + 3 designs, these studies include our published phase 1 study that first used a queue-based method and a study using isotonic regression-guided decisions in patients with organ dysfunction. One key parameter, largely independent of the design, that affected the duration of trials is the rate of inevaluability (the frequency with which patients who are treated cannot be assessed for DLTs and subsequent dose- level escalation decisions). In Skolnik et al, the median inevaluability rate for patients in pediatric phase 1 studies in the Children’s Oncology Group was 13.6% (range, 3.4%-22.7%). Our consortium experience with adult solid tumors suggests a higher rate of patients who are inevaluable; in the 14 completed studies reviewed, there were 375 patients with 75 patients inevaluable, representing a 20% overall rate (scenario A1), with considerable variability between studies (3.6% to 44.0% in scenarios A2 and A3, respectively). Scenario A1 (Table 3), our standard example, has 5 dose levels, has a 28-day DLT period (cycle 1), starts at level 2, identifies candidates for the study at a mean of every 10 days, has a 1-month maximum screening period (with a screen failure of 30%), and a 20% inevaluable rate. Reasons for inevaluability in our experience included patient choice; non-dose-associated complications, including rapid disease progression; noncompliance with the protocol; changes in insurance status; drug supply issues; reaction to a poisonous bug bite; and late discovery of disease that rendered the patient ineligible. For another important parameter, the screen failure rate, which is not automatically tracked in the consortium, we reviewed single-institution phase 1 clinical trials at the center City of Hope. The screen failure rate was fairly consistent across the solid tumor studies at approximately 30%, which represented all causes that would result in a patient not receiving treatment after providing consent. This screen failure rate was used for all scenarios other than A4, in which we looked to demonstrate the result of an increase in screen failures, and scenario D, in which we modeled a specific clinical trial with a known screen failure rate. Scenario B was provided to compare the simulation of the IQ designs with its respective parent design for a safety lead-in study. Scenario C1 is based on a study of blinatumomab and lenalidomide (NCT02568553). This study’s experimental treatment was introduced on cycle 2, increasing the number of inevaluable patients (whose data could not be considered for a dose-escalation decision), while simultaneously allowing patients to hold a slot for an extended period. Early in the study, two-thirds of the patients were inevaluable, and combined with the cycle 2 evaluation, this provides the opportunity for a very substantial improvement while using queue-based methods. We amended the study to change from a 3 + 3 design to an IQ 3 + 3 design, which was subsequently approved by the Cancer Therapy Evaluation Program and the central institutional review board based on the simulation work. Scenarios C2 and C3 are variations that show the result of changes in accrual rate or inevaluability rate. Scenario D is based on a study of intraperitoneal chemotherapy (NCT00825201), in which surgery preceding chemotherapy increased the time a patient can occupy a slot; the cycle length was 28 days, but surgery and recovery could delay chemotherapy up to 90 days (screening time). Forty percent of the patients were ineligible per screening criteria, but only 7.5% were inevaluable for DLTs. We estimated the screening distribution and the arrival distribution from the study data. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 6/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 7/13 Table 3. Scenarios Simulated for the 3 + 3, Phase 1 Queue 3 + 3, Rolling 6, and Phase 1 Queue Rolling 6 Scenario A2: Low A3: High A4: Standard B: 21-d C2: Second C3: Second A1: Standard inevaluability, inevaluability, Δscreen A5: Standard A6: Standard A7: Standard safety C1: Second cycle DLT cycle DLT Δin Characteristic phase 1 phase 1 phase 1 failure Δarrival ΔCL Δtoxicity lead-in cycle DLT Δarrival evaluability D: IP phase 1 Starting dose level 2 2 2 2 22223 3 3 1 Lowest possible dose level 1 1 1 1 11111 1 1 1 Highest possible dose level 5 5 5 5 55524 4 4 6 Course length, d 28 28 28 28 28 21 28 21 56 56 56 28 Maximum waiting list time, d 0 0 0 0 00000 0 0 0 Screening duration, d β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,28,1,1) β(0,90,1,1.97) Screen failure probability, % 30 30 30 60 30 30 30 30 30 30 30 40 Probability of patient being 20 3.6 44 20 20 20 20 20 66 66 33 7.5 inevaluable , % Time until patient becomes β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) β(0,CL,1,1) inevaluable, d Dose-limiting toxicity F(0.2, 8.5) F(0.2, 8.5) F(0.2, 8.5) F(0.2, 8.5) F(0.2,8.5) F(0.2, 8.5) F(0.2, 5.5) F(0.2, 5.5) F(0.2, 9.5) F(0.2, 9.5) F(0.2, 9.5) F(0.2, 10.5) probability function Time until dose-limiting β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(0,CL,1.5,1) β(29,CL,1.5,1) β(29,CL,1.5,1) β(29,CL,1.5,1) β(0,CL,1.5,1) toxicity event, d Patient interarrival time, d exp(0,10,1) exp(0,10,1) exp(0,10,1) exp(0,10,1) exp(0,15,1) exp (0,10,1) exp (0,10,1) exp(0,10,1) exp(0,10,1) exp (0,15,1) exp (0,10,1) exp(0,15,1) Abbreviations: β, beta distribution; CL, course length; DLTs, dose-limiting toxicities; exp, exponential distribution F(x,y) = 100 × (0.5 + atan(x×π× [current dose level − y])/π). with the second parameter as the mean; IP, intraperitoneal. JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies For each design (3 + 3, IQ 3 + 3, rolling 6, and IQ rolling 6) scenarios were simulated 800 times, representing different random samplings of patient arrivals, DLT events (yes or no) based on the dose-toxicity DLT probability curve, DLT time (if a DLT occurred), evaluable results and time, and screening results and time. The inclusion of 800 samplings provides high confidence in the results (eg, a standard error <1.8%) while providing adequate numbers to observe unusual events (an occurrence rate of 0.3% will be observed with a probability >90%). Each sampling used the same patients arriving at the same time across the designs, and if the nth patient had a DLT while being treated at level 1 in the rolling 6 design, this same patient treated at level 2 in the IQ rolling 6 design would have a DLT because it is a higher dose level. Calculations were carried out in FlexSim HC version 5.3 (FlexSim Healthcare). The module is freely available online. Results are presented in graphical and tabular forms using means, medians, and ranges to be consistent with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist for transparency on issues associated with analytic methods (eg, skewness), study parameters (eg, values, ranges) and outcomes (eg, mean differences). Results For all scenarios, the IQ 3 + 3 design had shorter expected study durations than the 3 + 3 design, ranging from a reduction of 1.6 to 10.4 months (Figure 2A), and likewise, the IQ rolling 6 design has lower expected study durations than the rolling 6 design, ranging from a reduction of 0.4 months to 10.5 months (Figure 2B). There was a small increase in the mean number of patients treated on the IQ 3 + 3 design (difference in mean number of patients in all scenarios, <3.2 patients; Figure 2C), while the IQ rolling 6 had a smaller mean number of patients than the rolling 6 in 9 of the 12 scenarios, with a difference that did not exceed 3.3 patients (Figure 2D). Detailed results can be found in eTables 3 and 4 in the Supplement. For scenario A1, the typical phase 1 study in the consortium, the expected (mean) duration of the phase 1 study using the traditional 3 + 3 design was 19.5 months (range, 7.1-41.3 months), whereas the expected duration was 15.8 months (range, 5.3-27.0 months) for the IQ 3 + 3 design. This represents an expected reduction of 3.7 months (difference in medians, 3.6 months; a 19% reduction). There is a mean increase of 2.8 patients. In eTable 5 in the Supplement, we extended this mean scenario to 9 dose levels, so the MTD would almost assuredly be reached before the highest allowable dose level. In that setting, the expected duration was reduced by 5.4 months (26.0 vs 20.6 months; a 21% reduction), and we also present additional metrics (expected percentage of patients on each dose level) that confirm that the operating characteristics of the 3 + 3 design are maintained. By reducing inevaluability to 3.6% (scenario A2), the expected duration of both the 3 + 3 and the IQ 3 + 3 designs were shorter compared with scenario A1. The expected times were 16.5 vs 13.9 months, respectively (a 16% difference). When increasing the rate of patients who are inevaluable to 44% (scenario A3), the expected duration of the 3 + 3 and the IQ 3 + 3 designs increased. The reduction because of the IQ-based design became 6.4 months (24%); compared with a reduction of 3.7 months (19%) associated with scenario A1. With all designs, a higher number of patients who are inevaluable tended to reduce the dose level selected as the MTD (eTable 3 and 4 in the Supplement). This is because of the competing events of a patient experiencing a DLT and a patient being deemed inevaluable. As an extreme example, if the DLT evaluation period was very long, all patients would eventually be inevaluable, so the only patients who are evaluable would be those with a DLT before being deemed inevaluable, making all doses appear toxic. In scenarios A4 through A7, we explored the results of (1) increasing screen failures, (2) increasing the availability of patients (reducing the interarrival time), (3) reducing the course length, and (4) increasing toxicity. The IQ designs had uniformly shorter expected durations (reduction of 6.7 months, 4.2 months, 3.7 months, and 3.1 months, respectively). The frequency of the selection of JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 8/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies Figure 2. Box-Whisker Plots of Study Duration for the 3 + 3 and Phase 1 Queue (IQ) 3 + 3 and Rolling 6 and IQ Rolling 6 Designs A Study duration for the 3+3 and IQ 3+3 designs Study design 80 3+3 IQ 3+3 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario B Study duration for the Rolling 6 and IQ Rolling 6 designs Study design Rolling 6 IQ Rolling 6 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario No. of patients who started the 3+3 and IQ 3+3 study designs Study design 3+3 IQ 3+3 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D Scenario D No. of patients who started the Rolling 6 and IQ Rolling 6 study designs Study design Rolling 6 IQ Rolling 6 A, Study duration for the 3 + 3 and IQ 3 + 3 studies. B, Study duration for the rolling 6 and IQ rolling 6 studies. C, Number of patients who started treatment for the 3 + 3 and IQ 3 + 3 designs. D, Number of patients who started treatment for the rolling 6 and IQ rolling 6 A1 A2 A3 A4 A5 A6 A7 B C1 C2 C3 D designs. Graphs are based on 800 simulations per Scenario scenario (Table 3). JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 9/13 Patients starting treatment, No. Patients starting treatment, No. Study duration, mo Study duration, mo JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies the MTD between the IQ design and the parent design differed by a range of less than 0.1% to 3%, and the mean number of DLTs at a dose level greater than the MTD differed by 0.01 to 0.10 patients. For scenario B, which was considered the safety lead-in, even though 77.8% and 77.1% of the simulations selected the starting dose as the MTD (for the 3 + 3 and the IQ 3 + 3, respectively), the expected study duration was reduced from 7.6 months to 6.0 months (a 21% reduction). The expected number of patients differed by 0.6 patients. For scenario C1, the expected study duration was reduced from 34.2 months to 24.5 months, a 9.7-month (28%) reduction. This was expected given the inevaluable rate of 66% (with a 30% screen failure rate) and the amount of time a patient will be pending (up to 84 days). The expected number of patients required increased by 3.2, and the frequency of the MTD selected differed by less than 1%. The results of changing the arrival patterns or inevaluability rate are reflected in C2 and C3. The last scenario, D, demonstrated a lengthier expected study duration of 42.1 months using the 3 + 3 design. This was 31.9 months with the IQ 3 + 3 design. The selection of the MTD differed by 0.5% to 1.5%. For comparisons of the rolling 6 and IQ rolling 6 designs on the same 12 scenarios, a similar pattern emerged. For scenario A1, for example, the reduction in expected study duration for the IQ rolling 6 design is 3.4 months (16.4 vs 13.0 months), with a difference in the proportion of simulations choosing any particular dose as the MTD ranging from 0.1% to 1.1%; the difference in median study duration was 3.9 months. There were no extra patients treated, and in fact, the expected number of patients was reduced by 0.6 patients, reflecting that the rolling 6 design generally requires all patients treated to complete their assessment before escalating (even if 0 DLTs are noted in the first 3 patients), which adds time at that dose level and potentially results in even more patients at that dose level. For extended scenario A1 (eTable 5 in the Supplement), the expected reduction was 4.7 months. Discussion The phase 1 queue depends on the treatment, patient population, and specific language in the protocol. For example, the screen failure rate is dependent on specific eligibility criteria, insurance issues, and the frailty of the patient population. Inevaluability for the purpose of DLT determination depends in part on the length of the evaluation period, the frailty of the patient population, the frequency of rapid disease progression, and other causes of drug discontinuation or unplanned dose reductions that are not associated with adverse events attributed to the dose of the investigational agent (or agents). Patients may also be considered inevaluable for DLT determination if critical tests are missed. We observed in our phase 1 studies that these details can dramatically affect phase 1 study duration and such imperfections are present in every study. We can adapt designs to these imperfections to reduce the study duration while not exceeding the risk permitted in the parent design or affecting the operating characteristics. Because the rolling 6 design was originally developed for special settings, such as pediatrics, after the adult study, we compared the 3 + 3 with the IQ 3 + 3 design and the rolling 6 with the IQ rolling 6 design. In our typical phase 1 study, the IQ 3 + 3 design is expected to save 3.7 months in study duration compared with the 3 + 3 design, and the IQ rolling 6 design 3.4 months compared with the rolling 6 design, allowing phase 2 studies to start months earlier. Two scenarios (C1 or D) demonstrate approximate 10-month reductions in study duration when using the IQ 3 + 3 design. The IQ modifications allow accrual of patients in certain situations when the parent design would not permit this, and in the case of the rolling 6 design, it permits escalation when the parent design would not because of patients who are pending, a key limitation of the original rolling 6 design rules. Several time-saving advantages include (1) the ability to more readily replace patients who do not meet screening criteria or become inevaluable for dose-escalation decisions; (2) if dose JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 10/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies deescalation is required, the possibility that additional patients may have already been treated on the lower dose levels, accelerating the deescalation portion of the study; (3) the IQ rolling 6 design, which permits escalation with 0 of 3 or 0 of 4 patient with a DLT in the presence of additional patients who have started treatment without any DLT to date. Other advantages include situations in which physicians can treat a patient on the highest priority trial rather than hold accrual; especially for the IQ 3 + 3, a further advantage is to provide more information per dose level (if additional patients are treated prior to escalating), which can help select the recommended phase 2 dose. There is minimal difference between the parent and IQ designs in the selection of the MTD. Also, the principal investigator and/or treating physicians are not required to offer a trial to a patient if they wish to wait until the current patients at the current dose level complete cycle 1 evaluation. In that case, the IQ design reverts to the parent design. eTable 2 in the Supplement lists several studies either with completed phase 1 portions (6 studies) or ongoing phase 1 portions (3 studies) using the IQ designs. This includes 2 lymphoma phase 1 trials (including the trial that motivated scenario C1); 2 hematology phase 1 studies, including a phase 1 stem cell transplant study ; 4 breast cancer studies; and an IQ rolling 6 study of escalating doses of radiation therapy to the prostate fossa, in which prior data were judged sufficient to allow 6 patients to accrue, analogous to the pediatric setting. After the first queue-based variation of the 3 + 3 design started accrual in May 2012, the experience of investigators, coordinating centers, and statistical teams with these designs have increased. Their use has increased, and additional refinements have been implemented that are reflected in the rules and simulations presented here. The use of queuing methods to evaluate and optimize the operating characteristics of phase 1 designs or other aspects of clinical research is a nascent field. As with the parent designs, the determination of the MTD with the IQ variations is not usually the end of the dose-finding study. An expansion cohort at the recommended phase 2 dose or MTD is recommended with additional monitoring rules (as noted in the eAppendix in the Supplement for 1 study [NCT02568553]). When appropriate, a randomized dose-ranging phase 2 study evaluating candidate doses may also be suggested. Expanding the queue-based methods beyond the MTD determination is one possible area of future work. Queue-based modifications of DLT-rate targeting phase 1 designs with an implicit or 14,15 explicit queue can also be explored. The direct cost of the IQ 3 + 3 design when compared with the traditional 3 + 3 design comes in the form of a few extra patients. There is no similar cost when converting the rolling 6 to the IQ rolling 6 design. However, the IQ rolling 6 design does allows escalation with patients pending, which is less conservative than the rolling 6 designs. Indirect costs for the IQ designs include more frequent review of the data and more clinical judgment as to whether to add new patients or escalate with patients pending (when permitted) or to hold accrual and revert to the parent design. Limitations We did not model time to acquire data or decision time, which is in control of the data coordinating center and is usually short relative to the other intervals. We also did not implement separate screening-time distributions for patients considered screen failures vs successes or model delays in treatment. Conclusions The IQ 3 + 3 and the IQ rolling 6 designs should be considered as alternatives to the 3 + 3 and rolling 6 designs, respectively. The IQ designs better adapt to the patient queue to reduce study duration without exceeding the risk limits of the parent design or affecting operating characteristics. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 11/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies ARTICLE INFORMATION Accepted for Publication: February 25, 2020. Published: May 13, 2020. doi:10.1001/jamanetworkopen.2020.4787 Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Frankel PH et al. JAMA Network Open. Corresponding Author: Paul H. Frankel, PhD, Division of Biostatistics, Department of Research Information Sciences, City of Hope, Duarte, CA 91010 (pfrankel@coh.org). Author Affiliations: Division of Biostatistics, Department of Research Information Sciences, City of Hope, Duarte, California (Frankel, Longmate); Department of Medical Oncology, City of Hope, Duarte, California (Chung, Newman); Hematology/Oncology, University of California, Davis, Davis (Tuscano); Hematology and Hematopoietic Cell Transplantation, City of Hope, Duarte, California (Siddiqi); Radiation Oncology, City of Hope, Duarte, California (Sampath); Biostatistics Core, Norris Cancer Center, University of Southern California, Los Angeles (Groshen); Developmental Cancer Therapeutics Program, Division of Molecular Pharmacology, City of Hope, Duarte, California (Newman). Author Contributions: Dr Frankel had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Newman is the senior author. Concept and design: Frankel, Longmate, Groshen. Acquisition, analysis, or interpretation of data: Frankel, Chung, Tuscano, Siddiqi, Sampath, Newman. Drafting of the manuscript: Frankel, Groshen, Newman. Critical revision of the manuscript for important intellectual content: Frankel, Chung, Tuscano, Siddiqi, Sampath, Longmate, Groshen. Statistical analysis: Frankel, Siddiqi, Groshen. Obtained funding: Frankel, Newman. Administrative, technical, or material support: Frankel, Chung, Tuscano. Supervision: Frankel, Chung. Conflict of Interest Disclosures: Dr Chung reported personal fees from Celgene, Ipsen, Gritstone, Westwood Bioscience, and Perthera outside the submitted work. Dr Tuscano reported grants from the National Cancer Institute during the conduct of the study. Dr Siddiqi reported personal fees from Pharmacyclics, Janssen, AstraZeneca, Seattle Genetics, Juno Therapeutics, Kite Pharma, and AstraZeneca outside the submitted work. Dr Newman reported grants from the National Institutes of Health during the conduct of the study. No other disclosures were reported. Funding/Support: This work is supported in part by the National Cancer Institute (grants UM1CA186717, U01CA62505, and N01CM-62209) and included work performed in the Biostatistics Core (grant P30CA033572). Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Additional Contributions: The authors thank Stella Khoo, MBA, BS, City of Hope, for collecting the information on the National Cancer Institute/Cancer Therapy Evaluation Program studies; Richard Sposto, PhD, University of Southern California, for his input and simulation code; Cliff King, BS, FlexSim Software Products, for his assistance with the FlexSim simulation tool; and Sandra Thomas, PhD, City of Hope, for manuscript editing. They were not compensated for their contributions. Additional Information: City of Hope, the University of Southern California, and University of California at Davis are National Cancer Institute–Designated Comprehensive Cancer Centers. REFERENCES 1. Abrams JS, Mooney MM, Zwiebel JA, et al. Implementation of timeline reforms speeds initiation of National Cancer Institute-sponsored trials. J Natl Cancer Inst. 2013;105(13):954-959. doi:10.1093/jnci/djt137 2. Doroshow JH. Timely completion of scientifically rigorous cancer clinical trials: an unfulfilled priority. J Clin Oncol. 2013;31(27):3312-3314. doi:10.1200/JCO.2013.51.3192 3. US Food and Drug Administration. Step 3: clinical research. Published January 4, 2008. Accessed April 7, 2020. https://www.fda.gov/patients/drug-development-process/step-3-clinical-research JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 12/13 JAMA Network Open | Statistics and Research Methods Model of a Queuing Approach for Patient Accrual in Phase 1 Oncology Studies 4. Alsumidaie M, Schiemann P. Why are cancer clinical trials increasing in duration? Published 2015. Accessed April 24, 2020. http://www.appliedclinicaltrialsonline.com/why-are-cancer-clinical-trials-increasing-duration 5. Storer BE. Design and analysis of phase I clinical trials. Biometrics. 1989;45(3):925-937. doi:10.2307/2531693 6. Skolnik JM, Barrett JS, Jayaraman B, Patel D, Adamson PC. Shortening the timeline of pediatric phase I trials: the rolling six design. J Clin Oncol. 2008;26(2):190-195. doi:10.1200/JCO.2007.12.7712 7. FlexSim Healthcare. Register to create your free clinical trials FlexSim account. Published 2019. Accessed April 20, 2020. https://healthcare.flexsim.com/clinical-trials 8. Queue. A queueing approach to reduce phase I study duration in oncology. Published 2015. Accessed April 20, 2020. https://oneq.netlify.app 9. Sposto R, Groshen S. A wide-spectrum paired comparison of the properties of the Rolling 6 and 3+3 Phase I study designs. Contemp Clin Trials. 2011;32(5):694-703. doi:10.1016/j.cct.2011.04.009 10. Siddiqi T, Frankel P, Beumer JH, et al. Phase 1 study of the Aurora kinase A inhibitor alisertib (MLN8237) combined with the histone deacetylase (HDAC) inhibitor vorinostat in lymphoid malignancies. Leuk Lymphoma. 2020;61(2):309-317. doi:10.1080/10428194.2019.1672052 11. Stein A, Palmer J, Tsai NC, et al. Phase I trial of total marrow and lymphoid irradiation transplant conditioning in patients with relapsed/refractory acute leukemia. Biol Blood Marrow Transplant. 2017;23(4):618-624. doi:10.1016/ j.bbmt.2017.01.067 12. Sampath S, Frankel P, Vecchio BD, et al. Stereotactic body radiation therapy to the prostate bed: results of a Phase 1 dose-escalation trial. Int J Radiat Oncol Biol Phys. 2020;106(3):537-545. doi:10.1016/j.ijrobp.2019.11.005 13. Ratain MJ. Targeted therapies: redefining the primary objective of phase I oncology trials. Nat Rev Clin Oncol. 2014;11(9):503-504. doi:10.1038/nrclinonc.2014.135 14. Iasonos A, O’Quigley J. Design considerations for dose-expansion cohorts in phase I trials. J Clin Oncol. 2013;31 (31):4014-4021. doi:10.1200/JCO.2012.47.9949 15. Zhou T, Guo W, Ji Y. PoD-TPI: Probability-of-decision toxicity probability interval design to accelerate phase I trials. Published 2019. Accessed April 8, 2020. https://arxiv.org/abs/1904.12981 SUPPLEMENT. eTable 1. Phase I, II and III studies by year. eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods. eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). eAppendix. Design Rationale. JAMA Network Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 (Reprinted) May 13, 2020 13/13 Supplementary Online Content Frankel PH, Chung V, Tuscano J, et al. Model of a queuing approach for patient accrual in phase I oncology studies. JAMA Netw Open. 2020;3(5):e204787. doi:10.1001/jamanetworkopen.2020.4787 eTable 1. Phase I, II and III studies by year. eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods. eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). eAppendix. Design Rationale. This supplementary material has been provided by the authors to give readers additional information about their work. © 2020 Frankel PH et al. JAMA Network Open. eTable 1. Phase I, II and III studies by year: Study Start Phase I Phase II* Phase III 2019 (1/1/2019-10/11/2019) 1143 1552 395 2018 1278 1778 465 2017 1239 1751 429 2016 1208 1573 426 2015 1047 1499 482 *Phase II studies in clinicaltrials.gov can have a Phase I portion (e.g. safety lead-in), and can be Phase II/III studies, so we have focused on the number of pure Phase I studies (Early Phase 1 or Phase I) vs pure Phase III studies in clinicaltrials.gov (cancer, interventional, study start date). eTable 2. Phase I or Safety Lead-in Clinical Trials Designed With Queue-based Methods NCT Title NCT02568553 Le na lid om id e a nd Blin a t um om a b in Tre a t ing Pa t ie nt s Wit h Re la p se d Non - Hod gkin Lym p hom a NCT00576979 Intensity-Modula t e d Ra dia t ion The ra py, Et op osid e , a nd Cyclop hosp ha m id e Followe d By Donor St e m Ce ll Tra nsp la nt in Tre a t ing Pa t ient s Wit h Re la p se d or Re fra ct ory Acut e Lym p hob la st ic Le uke m ia or Acut e Mye loid Leukemia NCT01567709 Alise rt ib in Com b ina t ion Wit h V orin ost a t in Tre a t ing Pa t ie nt s Wit h Re la p se d or Re curre nt Hod gkin Lym p hom a , B-Ce ll Non- Hod gkin Lym p hom a , or Pe rip he ra l T-Cell Lymphoma NCT01041443 5-Fluoro-2' -De oxycyt id ine a nd Te t ra hyd rourid ine in Tre a t ing Pa t ie nt s Wit h Acut e Mye loid Le uke m ia or Mye lod ysp la st ic Synd romes NCT02778685 Pe m brolizu m a b, Le t rozole , a nd Pa lbociclib in Tre a t ing Post m e nop a usa l Pa t ient s Wit h Ne wly Dia gnose d Me t a st a t ic St a ge IV Est roge n Re ce p t or Posit ive Bre a st Ca nce r NCT02648477 Pe m brolizu m a b a nd Doxorubicin Hyd rochlorid e or Ant i-Est roge n The ra p y in Tre a t ing Pa t ie nt s Wit h Trip le-Ne ga t ive or Horm one Re ce p t or-Posit ive Me t a st a t ic Bre a st Ca nce r NCT02971761 Pe m b rolizu m a b a nd Enob osa rm in Tre a t ing Pa t ie nt s Wit h And roge n Re ce p t or Posit ive Me t a st a t ic Trip le Ne ga t ive Bre a st Ca nce r © 2020 Frankel PH et al. JAMA Network Open. NCT01923506 St e re ot a ct ic Bod y Ra d ia t ion The ra p y in Tre a t ing Pa t ie nt s Wit h Prost a t e Ca nce r Aft e r Und e rgoing Surge ry NCT03853707 Ipatasertib a nd Ca rb op la t in Wit h or Wit hout Pa clit a xe l in Tre a t ing Pa t ie nt s Wit h Me t a st a t ic Trip le Ne ga t ive Bre a st Ca nce r eTable 3. Simulations for 3+3 and IQ 3+3 (each based on 800 simulations). Variation #Treated #Treated #Months #Months Mean 3+3 IQ 3+3 DLT rate 3+3 IQ3+3 3+3 IQ3+3 #DLTs % MTD % MTD (based Mean ,Med Mean,Med Mean,Med Mean, Med above At level At level on model) (range) (range) (range) (range) MTD Scenario A1 19.1, 19.0 21.9, 22.0 19.5, 19.4 15.8, 15.8 3+3: 0.87 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (8-37) (7.1-41.3) (5.3-27.0) IQ3: 0.95 1 7.4% 1 7.3% 1 6.7% Phase I 2 8.4% 2 9.3% 2 7.6% 20% Ineval 3 9.9% 3 12.5% 3 9.0% 4 16.3% 4 15.6% 4 10.8% 5 58.0% 5 55.0% 5 13.6% Scenario A2 16.1,16.0 18.6,19.0 16.5,16.5 13.9,14.0 3+3: 0.81 NA <1.0% NA <1.0% NA NA Low Ineval (8-26) (8-29) (6.3-32.7) (5.1-25.3) IQ3: 0.87 1 5.6% 1 6.1% 1 6.7% Phase I 2 7.8% 2 8.9% 2 7.6% 3.6% Ineval 3 9.5% 3 10.0% 3 9.0% 4 15.4% 4 15.9% 4 10.8% 5 61.3% 5 58.5% 5 13.6% Scenario A3 26.0, 26.0 29.0, 29.0 26.6, 26.3 20.2, 20.3 3+3: 1.08 NA 1.0% NA 1.0% NA NA High Ineval (9-54) (9-54) (7.3-59.6) (6.9-40.8) IQ3: 1.13 1 7.6% 1 8.4% 1 6.7% Phase I 2 11.8% 2 11.9% 2 7.6% 44% Ineval 3 13.5% 3 15.0% 3 9.0% 4 17.6% 4 17.4% 4 10.8% 5 48.5% 5 46.4% 5 13.6% Scenario A4 18.8, 19.0 21.1,21.0 30.0,29.4 23.3,23.1 3+3: 0.90 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (7-37) (7.7-63.8) (6.9-45.9) IQ3: 0.89 1 6.3% 1 6.6% 1 6.7% Phase I 2 8.3% 2 8.1% 2 7.6% 3 11.1% 3 12.1% 3 9.0% 60% 4 17.1% 4 15.8% 4 10.8% 5 56.6% 5 56.9% 5 13.6% Scenario A5 19.0, 19.0 21.3, 22.0 23.5, 23.4 19.3, 19.1 3+3: 0.87 NA <1.0% NA <1.0% NA NA Ave Ineval (8-34) (8-37) (8.6-48.5) (5.4-34.6) IQ3: 0.92 1 7.1% 1 6.8% 1 6.7% Phase I 2 8.5% 2 9.4% 2 7.6% 3 10.0% 3 12.6% 3 9.0% 15 day mean) 4 16.3% 4 14.6% 4 10.8% 5 57.9% 5 56.1% 5 13.6% Scenario A6 19.0,19.0 21.4,22.0 18.1, 17.9 14.4, 14.4 3+3: 0.88 NA <1.0% NA <1.0% NA NA Ave Ineval (9-34) (7-36) (6.4-35.1) (4.7-26.4) IQ3: 0.93 1 7.3% 1 7.1% 1 6.7% Phase I 2 8.4% 2 8.6% 2 7.6% 3 9.8% 3 12.3% 3 9.0% days 4 16.5% 4 15.6% 4 10.8% 5 57.8% 5 55.9% 5 13.6% Scenario A7 16.6, 16.0 18.1, 17.0 16.5, 15.9 13.4, 12.9 3+3: 2.1 NA 3.8% NA 4.4% NA NA Ave Ineval (4-31) (4-38) (3.8-36.5) (3.5-28.3) IQ3: 2.2 1 15.4% 1 16.0% 1 10.8% Phase I 2 23.5% 2 24.3% 2 13.6% 3 27.8% 3 29.8% 3 18.0% 4 22.5% 4 20.6% 4 25.9% 5 7.1% 5 5.0% 5 40.3% Scenario B 8.2, 7.0 8.8, 8.0 7.6, 6.9 6.0, 5.5 3+3: 0.53 NA 4.3% NA 4.6% NA NA Safety Lead- (4-21) (4-21) (2.5-20.4) (2.4-17.0) IQ3: 0.56 1 18.0% 1 18.3% 1 10.8% in Phase I 2 77.8% 2 77.1% 2 13.6% Scenario C1 26.7, 26.0 29.9, 29.0 34.2, 33.3 24.5, 23.6 3+3: 0.48 NA <1% NA <1% NA NA nd 2 Cycle with (11-62) (11-72) (11.2-90.2) (9.9-65.3) IQ3: 0.51 1 <1% 1 1.3% 1 5.9% experimental 2 7.8% 2 8.1% 2 6.7% combination 3 13% 3 13% 3 7.6% 4 78.1% 4 77.4% 4 9.0% Scenario C2 26.7, 26.0 29.6, 29.0 39.5, 38.4 29.1, 28.2 3+3: 0.47 NA <1% NA <1% NA NA nd 2 Cycle (11-71) (13-72) (11.4- (12.0-75.0) IQ3: 0.48 1 1% 1 <1% 1 5.9% 100.4) 2 6.5% 2 8.1% 2 6.7% 3 12.3% 3 13.3% 3 7.6% © 2020 Frankel PH et al. JAMA Network Open. 4 79.0% 4 77.8% 4 9.0% Scenario C3 13.9, 13.0 16.5, 16.0 18.2, 17.6 14.6, 14.3 3+3: 0.32 NA <1.0% NA <1.0% NA NA nd 2 Cycle (8-25) (8-32) (9.0-40.0) (8.2-29.1) IQ3: 0.36 1 <1.0% 1 <1.0% 1 5.9% 2 6.5% 2 6.4% 2 6.7% 3 9.0% 3 10.6% 3 7.6% 33% 4 84.4% 4 82.6% 4 9.0% Scenario D 22.5, 23.0 25.3, 26.5 42.1, 42.9 31.9, 32.9 3+3:.71 NA 3.6% NA 3.8% NA NA IP Phase I (2-36) (2-40) (3.5-81.5) (3.5-57.8) IQ3:.75 1 3.5% 1 3.6% 1 5.3% 2 4.6% 2 4.1% 2 5.9% 3 5.9% 3 7.4% 3 6.7% 4 6.5% 4 6.6% 4 7.6% 5 10.9% 5 10.9% 5 9.0% 6 65.0% 6 63.6% 6 10.8% eTable 4. Simulations for Rolling 6 and IQ Rolling 6 (each based on 800 simulations). Variation #Treated #Treated #Months #Months Mean Rolling 6 IQ R6 DLT rate Rolling 6 IQ R6 Rolling 6 IQ R6 #DLTs % MTD % MTD (based Mean,Med Mean,Med Mean,Med Mean, Med above At level At level on model) (range) (range) (range) (range) MTD Scenario A1 23.9, 26.0 23.3, 24.0 16.4,16.9 13.0,13.0 R6 : 0.89 NA <1.0% NA <1.0% NA NA Ave Ineval (8-39) (7-38) (4.9-38.0) (3.1-23.2) IQR: 0.94 1 7.6% 1 7.8% 1 6.7% Phase I 2 9.0% 2 8.9% 2 7.6% 20% Ineval 3 12% 3 13% 3 9.0% 4 14.9% 4 15.4% 4 10.8% 5 56.0% 5 54.9% 5 13.6% Scenario A2 20.5,23.0 20.1,21.0 14.0,14.4 11.4,11.6 R6 : 0.90 NA <1.0% NA <1.0% NA NA Low Ineval (8-28) (7-33) (4.1-26.2) (3.2-19.8) IQR: 0.88 1 6.5% 1 6.5% 1 6.7% Phase I 2 9.3% 2 9.3% 2 7.6% 3.6% Ineval 3 13.1% 3 11.9% 3 9.0% 4 14.9% 4 14.0% 4 10.8% 5 55.8% 5 57.8% 5 13.6% Scenario A3 31.5,33.0 30.0,31.0 21.4,21.7 16.4,16.3 R6 : 1.06 NA 1.0% NA 1.1% NA NA High Ineval (8-59) (9-51) (5.6-50.6) (5.3-31.6) IQR:1.13 1 8.1% 1 7.9% 1 6.7% Phase I 2 14.3% 2 12.3% 2 7.6% 44% Ineval 3 14.0% 3 15.5% 3 9.0% 4 14.3% 4 17.0% 4 10.8% 5 48.4% 5 46.3% 5 13.6% Scenario A4 23.1,24.0 21.8,22.0 25.2,25.3 19.8,19.5 R6 : 0.93 NA <1.0% NA < 1.0% NA NA Ave Ineval (8-39) (7-37) (5.7-56.1) (4.3-42.7) IQR:0.94 1 7.5% 1 7.1% 1 6.7% Phase I 2 8.1% 2 8.6% 2 7.6% 3 13.8% 3 12.9% 3 9.0% 60% 4 15.4% 4 15.8% 4 10.8% 5 54.5% 5 55.3% 5 13.6% Scenario A5 23.2,25.0 22.1,23.0 20.7,21.0 17.1,17.1 R6 : 0.91 NA < 1.0% NA < 1.0% NA NA Ave Ineval (8-37) (9-38) (5.2-38.8) (4.9-34.9) IQR:0.94 1 7.3% 1 6.9% 1 6.7% Phase I 2 8.6% 2 9.4% 2 7.6% 3 12.8% 3 12.6% 3 9.0% 15 day mean) 4 15.4% 4 15.1% 4 10.8% 5 55.5% 5 55.4% 5 13.6% Scenario A6 23.5,25.0 22.5,23.0 15.4,15.7 12.1,12.1 R6 : 0.91 NA < 1.0% NA < 1.0% NA NA Ave Ineval (8-36) (9-38) (4.2-35.0) (3.7-23.2) IQR: 0.91 1 7.8% 1 7.5% 1 6.7% Phase I 2 8.4% 2 8.5% 2 7.6% 3 13.1% 3 12.5% 3 9.0% days 4 14.6% 4 14.5% 4 10.8% 5 55.6% 5 56.8% 5 13.6% Scenario A7 18.0,17.0 18.7,18.0 12.3,11.6 10.7,10.1 R6 : 2.01 NA 3.5% NA 3.9% NA NA Ave Ineval (4-34) (4-37) (3.4-28.3) (2.7-22.9) IQR:2.16 1 18.8% 1 17.4% 1 10.8% Phase I 2 26.1% 2 24.8% 2 13.6% 3 26.8% 3 28.3% 3 18.0% 4 19.5% 4 21.1% 4 25.9% 5 5.4% 5 4.6% 5 40.3% Scenario B 8.3, 7.0 9.0, 8.0 5.5, 4.9 5.1, 4.7 R6 : 0.53 NA 3.3% NA 4.3% NA NA Safety Lead- (4-21) (4-22) (1.6-15.8) (1.6-16.4) IQR: 0.57 1 18.9% 1 18.9% 1 10.8% in Phase I 2 77.9% 2 76.9% 2 13.6% Scenario C1 31.1, 31.0 31.1, 30.0 24.9, 23.9 18.7, 18.0 R6 : 0.48 NA <1.0% NA <1.0% NA NA nd 2 Cycle with (11-71) (13-72) (7.3-57.4) (7.3-46.2) IQR: 0.47 1 <1.0% 1 <1.0% 1 5.9% experimental 2 9.6% 2 7.5% 2 6.7% combination 3 12.0% 3 12.8% 3 7.6% 4 77.5% 4 78.8% 4 9.0% © 2020 Frankel PH et al. JAMA Network Open. Scenario C2 30.7, 30.0 30.0, 29.0 30.4, 29.1 23.9, 23.0 R6 : 0.49 NA <1.0% NA <1.0% NA NA nd 2 Cycle (11-63) (13-73) (8.6-77.3) (8.5-55.8) IQR: 0.47 1 1.0% 1 <1.0% 1 5.9% 2 8.8% 2 7.9% 2 6.7% 3 12.6% 3 12.6% 3 7.6% 4 77.4% 4 78.5% 4 9.0% Scenario C3 16.7, 16.5 17.7, 17.0 13.8, 13.5 11.2, 10.9 R6 : 0.36 NA <1.0% NA <1.0% NA NA nd 2 Cycle (9-27) (9-32) (5.8-29.6) (5.2-21.1) IQR: 0.39 1 <1.0% 1 <1.0% 1 5.9% 2 7.3% 2 7.0% 2 6.7% 3 9.8% 3 11.0% 3 7.6% 33% 4 82.8% 4 81.6% 4 9.0% Scenario D 30.0, 34.0 26.7, 28.0 36.1, 38.6 25.6, 26.6 R6 : 0.78 NA 4.4% NA 4.0% NA NA IP Phase I (2-41) (2-40) (3.1-76.9) (3.1-45.7) IQR: 0.74 1 4.6% 1 4.5% 1 5.3% 2 6.4% 2 4.4% 2 5.9% 3 6.0% 3 7.3% 3 6.7% 4 7.1% 4 6.3% 4 7.6% 5 10.5% 5 10.0% 5 9.0% 6 61.0% 6 63.6% 6 10.8% eTable 5. Simulations Based on Scenario A1 with 4 added higher dose levels (based on 800 simulations). Based on Model #Treated #Treated #Treated #Treated 3+3 IQ 3+3 Rolling 6 IQ R6 Scenario A1 with 3+3 IQ3+3 Rolling 6 IQ R6 % MTD % MTD % MTD % MTD an additional 4 Mean=25.6 Mean=28.4 Mean=31.6 Mean=30.3 At level At level At level At level higher dose levels Median=26.0 Median=28.5 Median=33.0 Median=31.0 Range 8-53 Range 8-58 Range 8-62 Range 7-66 Dose DLT level Rate Average Average Average Average fraction at Fraction at Fraction at Fraction at dose level: dose level: dose level: dose level: *NA NA -- -- -- -- <1.0% <1.0% <1.0% <1.0% 1 6.7% 0.04 0.04 0.04 0.04 7.4% 7.3% 7.6% 7.8% 2 7.6% 0.22 0.24 0.26 0.24 8.4% 9.3% 9.0% 8.9% 3 9.0% 0.19 0.21 0.22 0.22 9.9% 12.4% 12.1% 13.0% 4 10.8% 0.17 0.17 0.17 0.17 14.3% 14.1% 14.8% 13.8% 5 13.6% 0.14 0.14 0.13 0.13 17.3% 18.6% 14.8% 17.8% 6 18.0% 0.11 0.11 0.09 0.10 21.1% 20.9% 22.0% 20.8% 7 25.9% 0.08 0.07 0.06 0.06 16.6% 14.6% 15.6% 14.0% 8 40.3% 0.04 0.02 0.02 0.02 4.7% 2.1% 3.4% 3.8% 9 59.7% 0.01 <0.01 <0.01 <0.01 <1% <1% <1% <1% *NA represents when dose level 1 was above the MTD Study Duration (months) 3+3 IQ3+3 Rolling 6 IQ R6 Study Duration (Mean or Expected) 26.0 20.6 21.5 16.8 Study Duration Median (Range) 25.7 20.7 22.2 16.7 (7.1-63.9) (5.3-42.6) (4.9-45.9) (3.1-36.4) #Started Treatment 25.6 28.4 31.6 30.4 (Mean or Expected) #Started Treatment 26.0 28.5 33 .0 31.0 Median (Range) (8-53) (8-58) (8-62) (7-66) #DLTs above MTD 2.2 2.2 2.0 2.2 (Mean or Expected) #DLTs above MTD 2.0 2.0 2.0 2.0 Median (range) (0-7) (0-6) (0-5) (0-7) eAppendix. Design Rationale: IQ 3+3 Rationale: Row 4 differences were discussed in the manuscript. Additionally, When 2 patients have a “Pass” (fully assessed and evaluable with no DLT) a fifth patient can be accrued without © 2020 Frankel PH et al. JAMA Network Open. exceeding the 3+3 risks. In that situation the worst case scenario in the 3+3 is that third patient has a DLT, in which case, 3 additional patients can be accrued. As a result, with 2 patients without a DLT, we are within the risk limits to accrue up to 4 additional patients, allowing us to get to 6 enrolled, with 2 evaluable each with a “Pass” (rows 7-8). In row 11, with 4-6 treated and 3-5 evaluable with no DLT, the 3+3 would have previously escalated if 0/3, so the 3+3 could only be in that situation if expanding to 6 patients with the dose above closed. That is not automatically the case with the IQ 3+3, however, we would know that there were no DLTs in at least 3 patients, and no DLTs in the pending patients reported to date, so if the above level is open, we can escalate, with a stronger safety signal than with only information on 3 patients, hence reduced risk to future patients. The next major difference (row 16) occurs with three patients accrued (consented and promised a slot), 1 DLT and 1 patient with a “Pass” (or 1 of the 2 evaluable patients with a DLT). The 3+3 holds accrual in that setting, whereas the IQ 3+3 can th accrue a 4 patient. In the standard 3+3, if 1 of 1 patients have a DLT, 2 additional patients can be accrued, so we are within that risk to have 2 patients accrued with 1 of 2 patients with a DLT, again, not exceeding the risks of the 3+3. Risk-based logic also allows the IQ 3+3 to accrue additional patients with 1 DLT, and 4 or 5 patients evaluable and up to 2 patients pending (rows 20, 21). The IQ 3+3 also allows escalation with 1 out of 6 evaluable patients with DLTs and no DLTs reported in pending patients. To consider why eight patients are permitted on the IQ 3+3 consider: If we have 3 patients enrolled, 2 with a “PASS” (meaning fully evaluable with no DLT), and 1 DLT, we can enroll th another 3 patients per the standard 3+3. If the 4 patient is a “PASS”, we are safer with 1 of 4 DLT than 1 of 3, so we are not exceeding the risks to still enroll 3 patients. That means we could th get to 7 patients. If the 5 patient is “PASS”, the same argument can be made, and we get to 8 th patients. However, if a 6 patient is a “PASS” we escalate, so we don’t need more than 8 slots © 2020 Frankel PH et al. JAMA Network Open. during escalation. Also note that if the patient consented and was promised a slot based on a dose level but has not started treatment when an escalation has been declared, that patient is put on the currently accruing (higher) dose level, and this is modelled in the simulations. To address DLTs occurring in pending patients at a lower dose level after escalation we note that information would not have been observed in the 3+3, so any conservative action reduces the risk to future patients when compared to the original 3+3. IQ Rolling 6 Where the original Rolling 6 allows 6 patients to be put at risk at the same time, the IQ Rolling 6 maintains that risk limit or lower, and when 1 patient has a “Pass” or a second patient has a “Pass”, an additional 1 or 2 patients, respectively, can be accrued to maintain 6 patients at risk when there are no DLTs. If 1 patient has a DLT, the Rolling 6 permits accrual of up to 5 additional patients, as does the IQ Rolling 6. If 1 patient has a DLT, the IQ Rolling 6 as presented is not maximally aggressive. When there is a DLT, we revert to the risk rules of the 3+3 for accruing beyond 6 patients due to the conditions for using the Rolling 6 being in question in the presence of a DLT . Additional Details: Note 1: Some decisions in Table 1-2 required special notes. For example, the traditional 3+3 design and the Rolling 6 design do not specify what to do with 2 DLTs in 7 patients or 2 DLTs in 8 patients (row 26 and 28 in Table 1 and rows 27 and 29 in Table 2). We first addressed this situation in the context of the NCI-CTEP sponsored study (NCT02568553, Table S1). Considerations © 2020 Frankel PH et al. JAMA Network Open. included (1) the principal investigator did not want to be obligated to lower the dose if 2 of 7 patients had DLTs and wanted the option to continue to treat at that dose (but not escalate), suggesting that dose as the MTD; (2) additional principal investigators agreed; (3) as recommended , the study had additional monitoring rules for the expansion cohort so that if 3 or more DLTs occurred in the first 12 patients treated at that dose, the recommended Phase 2 dose and continued accrual would be re-evaluated; (4) Children’s Oncology Group has set a precedent allowing the possibility of cohort expansion when 2 DLTs occur in 6 evaluable patients if the two DLTs are of different classes (e.g. hepatotoxicity and myelosuppression) with internal review (personal communication); and (5) we searched for logic consistent with the decisions inherent in the 3+3 design using two beta distributions for the probability of a DLT (analogous to an optimistic and pessimistic prior) and two decision thresholds for escalation or de-escalation. We challenged those sets of rules consistent with the 3+3 with 2 DLTs out of 7 patients. We found that if 1/6 was an acceptable DLT rate not requiring de-escalation, the decision would never be to de-escalate (see Note 3 below) resulting in 2 DLTs out of 7 or 8 patients constituting the MTD in our decision grid for the IQ 3+3 and the IQ Rolling 6. Note 2: We currently allow the protocol team to evaluate the specifics of the lower level DLT(s), and permit the team to immediately reduce the dose on the patients at the higher dose or allow patients on the higher dose level to complete their evaluation and then decide what action to take. The simulations do not consider such flexible decision-making and only considers those DLTs that occurred on a lower dose after escalating if later de-escalation re-visits that dose. The frequency of two DLTs at a dose below the MTD in the simulations in either the IQ 3+3 or the IQ Rolling 6 is rare (~2%). © 2020 Frankel PH et al. JAMA Network Open. Note 3: R-CODE for 2 of 7 Question comment<- c(" • The Beta distribution is a flexible distribution for the probability of a DLT. • This will be used for the probability of a DLT (p) using parameters a, b, e.g. Beta(a,b) • The expected value of p, E(p) = a/(a+b). • We update the expected value of p with patient data on a dose according to: • E(p) = (a+ #DLTs)/ (a+#Pts+b), (e.g. if a=b=1 with no patients, E(p)=1/2) . • To generate a logical decision engine that agrees with 3+3 based on two priors: A) One prior, beta(a1,b1) that guides whether the dose is safe and we can continue to treat. Based on a threshold (T1) for the expected DLT rate B) One prior, beta(a2,b2) guides whether we are comfortable escalating the dose. Based on a different threshold (T2) for the expected DLT rate. (a1+#DLTS)/(a1+#PTS+b1)>T1? Go Down (a2+#DLTS)/(a2+#PTS+b2)<T2? Go Up Otherwise stay • 55,000,000 random samples of parameters to find two priors beta(a1,b1), beta(a2,b2), and two threshold (T1,T2), that are consistent with the traditional 3+3 rules. We used a general grid search. • Find sets of parameters matched the 3+3 • Apply all different sets of rules to 2 DLTs of 7 patients. (a1+2)/(a1+7+b1)>T1? Go Down ..else (a2+2)/(a2+7+b2)<T2? Go Up …else MTD ") # Use 64-BIT R to allow larger number of simulations set.seed(9) simnum<-55000000 # 55 million #(a1+2)/(a1+7+b1)>T1? Go Down ..else #(a2+2)/(a2+7+b2)<T2? Go Up …else MTD a1=runif(simnum,min=0.00,max=13) # rep(0,simnum) #rbinom(simnum,10,0.2) b1= runif(simnum,min=0.0,max=50) a2=runif(simnum,min=0.0,max=45) b2=runif(simnum,min=0,max=45) t1=runif(simnum,min=0,max=1/2) # 1/6 was key t2= runif(simnum,min=0,max=1/2) # 1/3 © 2020 Frankel PH et al. JAMA Network Open. ###### down<-function(ndlt,tot) {return( ((a1+ndlt)/(a1+tot+b1))>t1 ) } up<-function(ndlt,tot) {return( ((a2+ndlt)/(a2+tot+b2))<t2 ) } # Flawed with 0 DLTS z0<-( down(0,0) | up(0,0) | down(0,1) | up(0,1) | down(0,2) | up(0,2) | down(0,3) | !up(0,3) | down(0,4) | !up(0,4) | down(0,5) |!up(0,5) | down(0,6)| !up(0,6) ) # Flawed with 1 DLT z1<- ( down(1,1) | up(1,1) | down(1,2) | up(1,2) | down(1,3) | up(1,3) | down(1,4) | up(1,4) | down(1,5) | up(1,5) | down(1,6) | !up(1,6) ) # Flawed with 2 DLTs z2<-( !down(2,2) | up(2,2) | !down(2,3) | up(2,3) | !down(2,4) | up(2,4) | !down(2,5) | up(2,5) | !down(2,6) | up(2,6) ) # Flawed with 3 DLTs z3<- (!down(3,3) | up(3,3) | !down(3,4) | up(3,4) | !down(3,5) | up(3,5) | !down(3,6) | up(3,6) ) # Flawed with 4 DLTs z4<- (!down(4,4) | up(4,4) | !down(4,5) | up(4,5) | !down(4,6) | up(4,6) ) # Flawed with either 0, 1, ,2 ,3 or 4 DLTs (not consistent with 3+3) bad<- (z0 | z1 | z2 | z3 | z4) # Gather up data tot<-data.frame(bad, a1, a2, b1, b2, t1, t2, prior1=a1/(a1+b1), prior2=a2/(a2+b2)) # Select parameters that are consistent with the 3+3 tot2<-tot[!tot$bad,] dim(tot2)[1]/simnum # Of paramters consistent with 3+3, answer is 1297 with above conditions and seed dim(tot2)[1] ############################################################################## ####### downcheck<-function(ndlt,tot,a1c,b1c,t1c) {return( ((a1c+ndlt)/(a1c+tot+b1c))>t1c ) } upcheck<-function(ndlt,tot,a2c,b2c,t2c) {return( ((a2c+ndlt)/(a2c+tot+b2c))<t2c) } # Ask what those 1297 sets of parameters would do with 2 DLTs out of 7 patients tot3<-data.frame(tot2, d2of7=downcheck(2,7,tot2$a1,tot2$b1,tot2$t1), u2of7=upcheck(2,7,tot2$a2,tot2$b2,tot2$t2) ) table(tot3$u2of7) # 0/1297 go up © 2020 Frankel PH et al. JAMA Network Open. table(tot3$d2of7) # 959/1297 go down BUT downtot<-tot3[tot3$d2of7,] summary(downtot$t1) # All decisions to go down had the threshold to go down less than 1/6 # If 1/6 is an acceptable rate, the stay option is selected. # Refining the grid to increase the number of parameter sets consistent with the 3+3 does not # Alter the conclusion. © 2020 Frankel PH et al. JAMA Network Open.

Journal

JAMA Network OpenAmerican Medical Association

Published: May 13, 2020

References