David M. Vail, DVM, DACVIM (Oncology)
Adaptive Trial Designs and Stopping Rules
Adaptive trial designs allow investigators to modify trials while they are ongoing based on data generated thus far and in some cases, taking into account data generated in other trials or past trials.
Stopping rules, rules that terminate a clinical trial either earlier than originally projected or within a predetermined adaptive trial design, can be applied to randomized Phase II or Phase III trials. Several methods and variations have been extensively reviewed.5,15,23-25 Stopping rules are designed to protect treatment subjects from unsafe drugs, to hasten the general availability of superior drugs as soon as sufficient evidence has been collected, and help ensure the transfer of resources and patients to alternative trials. Trials are stopped for 3 reasons; the investigational treatment is clearly better than control, is clearly worse then the control (either < activity or > toxicity), or the investigations therapy is not likely to be better (so called stopping for futility or futility analysis). The methods by which stopping rules are applied usually involve some type of interim analysis that looks at the data (by a blinded individual) generated so far and makes a determination based on predetermined rules. The interim data is often analyzed for conditional power, which is the probability that the final study result will demonstrate statistical significance in the primary efficacy endpoint, conditional on the current data observed so far and a specific assumption about the pattern of the data to be observed in the remainder of the study. If a study is designed up front to involve conditional power calculations of interim data, the rules for early termination are sometimes referred to as stochastic curtailing.
Bayesian (Continuous Learning) Adaptive Designs
Adaptive trial designs can be used not only to stop trials early, but also to adapt trials with respect to changing the randomization weight to better performing treatment arms, adding new treatment arms or dropping poorly performing arms, or extending accrual beyond the original target when more information is needed. With the availability of advance computational techniques, a new statistical methodology, the Bayesian approach was developed that makes statistical inferences that focus on the probability that a hypothesis is true given the available evidence.26 Traditionally, a frequentist approach to statistics is applied to clinical trials where parameters are fixed and not subject to future probabilities and are very inflexible. In contrast, Bayesian trials use available patient-outcome information, including biomarkers that accumulate data related to outcome (if available and validated) and even historical information or results from other relevant trials. The Bayesian approach uses this information to continually adapt the current trial design based on newly informed probabilities. Bayesian designs are intrinsically adaptive and data-driven, which allows inferences to be less dependent on the original study design. Bayesian approaches can be incorporated into the trial at the beginning, or can be used to monitor clinical trials originally designed with frequentist statistical methods. An example that illustrates the utility of the Bayesian approach involves interim analysis applied to a randomized Phase II trial of neoadjuvant epidermal growth factor receptor 2-positive breast cancer.27 In this trial initially designed to enter 164 patients (based on the frequentist approach to power), a Bayesian approach was used to perform interim analysis after 34 patients were enrolled; 67% of patients in the investigational treatment arm experienced complete responses compared to 25% in the standard treatment arm. The Bayesian predictive probability of statistical significance if 164 patients were accrued, based on the data available from these 34 patients, was calculated to be 95% and the trial was stopped and the drug moved to Phase III early.
Randomized Discontinuation Trials (RDT)
This relatively new Phase II design was proposed for evaluating the efficacy of newer targeted agents that are thought to have disease-stabilizing activity (cytostatic) in contrast to more traditional cytotoxic chemotherapeutics. Several reviews of this trial design are recommended.28 Trials that evaluated growth-inhibiting agents in tumors with a variable natural history seem ideally suited for RDTs as the "no treatment effect" is hard to control for in these cases. In essence, these trials serve to enrich (see later section) and homogenize for those patients that are likely to benefit from the static-agent. RDTs involve a 2-stage trial design where the first stage involves a "run-in" phase where all patients receive the cytostatic agent under investigation. At the end of the run-in phase, assessment of disease response is made. If a response is noted, the subject continues on with the investigational drug, while if progression (or excess toxicity) is noted the subject is removed from trial and allowed to receive alternative treatment. Those patients that meet stable disease criteria enter the second stage of the RDT and are randomized to either continue on with the investigational drug or placebo (the discontinuation arm). Then at predetermined times, follow-up determinations are made. Endpoints in stage 2 of the trial at these follow-up intervals are "stable or better" versus "progression"; that is progression-free rate. Time to event measures could be applied as well (e.g., time to progression), although this takes more time to complete. If a subject progresses in the second stage, the code can be broken and if in the placebo group, the investigational drug can be reinstituted. Therefore, there are 2 ways for RDTs to be stopped; either there is a substantial number of objective responses noted in the run-in phase making a second stage unnecessary or the number of subjects progressing in the second stage differ statistically between the treatment and placebo groups.
It becomes intuitive that the length of the run-in phase is critical to RDTs since if it is too long, some initially responding patients will progress during the late stage of the run-in and be missed (therefore increasing subject numbers) and if too short, insufficient enrichment occurs (not enough time for non-responders to progress) and the randomization might as well have been done at the outset. Therefore it pays to have some preliminary ideas as to the natural history of the disease.
The two major advantages of RDTs are that all subjects receive the drug up front such that every patient is given a chance to respond to the drug--something that is very popular with patients (or companion animal owners) and enrichment of likely responders may increase power and decrease subject numbers. Potential disadvantages of RDTs include the ethics of discontinuation (but the design can allow reinstitution), the potential for a carry-over effect of the drug after discontinuation (unlikely for most targeted agents), and failure to detect short duration activity (but this would likely be a clinically irrelevant duration anyway). RDTs can be improved by combining other modifications of clinical trials such as interim analysis, Bayesian analysis, and using active controls.
For purposes of illustration, an RDT in veterinary medicine that the author has considered would be the investigation of a cytostatic agent in dogs with pulmonary metastatic osteosarcoma. All dogs would enter the run-in phase and receive the cytostatic for 4 weeks and then be evaluated for response. From what we already know about the natural history of osteosarcoma in dogs, the majority (probably 80%) of dogs who did not receive treatment would progress in that time period. Those that were stable at 4 weeks, however, would be randomized to drug continuation or discontinuation (placebo) and followed with monthly reevaluations. This would ensure all dogs had a chance to respond to the drug, enrich the population likely to respond, and a positive result would be either clinical response noted in the run-in phase or a statistical difference in groups in the second stage.
Several trials have combined the different Phases (I/II/III) of clinical trials in the hope of streamlining and accelerating the drug development path as well as decreasing the number of subjects needed by combining data from patients entered in earlier phases with those in later phases with regard to clinical outcome endpoints. These have been referred to as "parallel" or "seamless" Phase I/II or Phase II/III trials and have been reviewed. For I/II combinations, following an initial period of dose-escalation, patients are randomized to different admissible dose levels and Bayesian probabilities are used to adaptively assign patients into groups over time. Doses with lower activity or unacceptable toxicity are eliminated and those with higher activity are expanded. The combined trial is stopped when Bayesian probabilities for safety, activity or futility hit prespecified boundaries. In Phase II/III combinations, Phase II learning is combined into Phase III confirming. That is, data generated for response in Phase II continues into Phase III. Treatment selection in Phase III may be based on more short-term endpoints in the Phase II portion (e.g., surrogate biomarker or response rate) and then all subjects (II and III) are followed for more Phase III-like confirmatory endpoints that take longer to mature (e.g., overall survival and time to progression).
Enrichment involves an intent to select a population of subjects to randomize in a trial that are more homogenous with respect to prognostic and, more importantly, predictive factors. Enrichment is used when the molecular target of the investigational drug is thought to be well known and there is some method to determine which subjects have tumors with the target and which do not. For example, if evaluating a novel compound that targets the epithelial growth factor receptor, immunohistochemistry could be performed on biopsy samples to ensure the presence of the receptor and only biopsies positive for the receptor would be eligible for inclusion in the trial. The classic example cited to illustrate the power of enrichment is the seminal trial evaluating trastuzumab (Her-2 monoclonal antibody) therapy combined with chemotherapy in woman with breast cancer.29 In this trial, 469 patients enriched for Her-2 expressing tumors were needed to show a statistical advantage with combination treatment (1 year OS of 78% versus 67%). It was estimated statistically, that approximately 23,586 patients would have been required to show a similar difference in a population that was not enriched. One must be careful with enrichment, however, as many of the drugs we think of as specifically targeted actually have other targets that we are unaware of (so called "dirty drugs" that have "off-target" effects) and we could miss a population of responders if we enriched only for what is known. An example would be evaluating tumors for responsiveness to imatinib, now know to inhibit 3 tyrosine kinase growth-factor pathways (Kit, PDGF, and Bcr-Abl). If we initially thought imatinib only had activity against Kit-expressing tumors and enriched for them alone, patients with PDGF and Bcr-Abl driven tumors would be excluded from the study and potential responders would be lost.
Rather than demonstrating that a new drug is superior to standard-of-care, it is sometimes desirable to show that a drug is not inferior.30 For example, a competing pharmaceutical company may wish to market a "me-too" drug of the same family that they feel is as effective but has better PK parameters or safety profile; or you may wish to show that the addition of a drug that alleviates toxicity in a standard protocol does not also decrease the anticancer effects of the active agent. These are noninferiority trials. An example in the veterinary literature would be the pyridoxine/liposomal doxorubicin trial where dogs were randomized to receive liposomal doxorubicin /placebo versus liposomal doxorubicin /pyridoxine.31 In addition to determining that pyridoxine helped prevent the dose-limiting toxicity of liposomal doxorubicin (PPES), an analysis was performed to ensure activity (remission duration) was not adversely affected. These trials tend to be expensive trials with large numbers of subjects as the definition of "inferior" must be predetermined and power sufficient to prove a negative.
Crossover trials are a modification of a Phase II or III trial where subjects are randomized to receive one treatment arm or the other, then after a certain trial period, enter a no treatment washout phase. Following washout, they then are crossed over to receive the other treatment. Response endpoints are collected during both treatment phases and compared. This design increases the robustness of power as patients serve as their own controls and therefore paired data analysis can be used. However, it is intuitive that this design is only well suited for chronic diseases that progress slowly (e.g., arthritis, hypertension, diabetes, urinary incontinence) and for drugs that do not have a long carryover effect (longer than the washout period). They are rarely used for cancer therapy trials due to the progressive nature of these diseases.
The fact that the discussion of this component of clinical trials is at the end should not be construed to trivialize it. Rather, this is a critical component of any trial and we as investigators are ethically bound to ensure our clients are informed of the design complexity, the risks (known and unknown) and the benefits (or lack thereof) of any trial they may be considering for their companion animal prior to entry. The 14 components of consent found on the Morris Animal Foundations Web site (www.morrisanimalfoundation.org/reports/grants/ full_proposal/Established_Investigator_guidelines.pdf ) are recommended by the author to be included in any clinical trial consent form.
The application of rigorously controlled, clinical trials is a relatively new concept in veterinary oncology and clinical trial design has been an afterthought in most veterinary medical curriculums. A working understanding of trial design and implementation is important for both trial investigators and the clinicians that read trial reports in order to fully recognize the strengths and weaknesses of the reported data. This is a necessary prerequisite to make appropriate conclusions and ultimately advance the standard-of-care in clinical practice. While much of the current standard-of-care in veterinary oncology is based on retrospective studies or transference from the human literature, a new era of clinical trial awareness, brought on by new consortia and cooperative investigative groups is beginning to change this limitation. The use of controlled, randomized, blinded multicenter trials, testing new cytotoxics and cytostatic agents is now becoming the norm rather than the exception. Ultimately, advanced clinical trial design applied to companion animal populations will advance both veterinary-based practice and inform future human clinical trials that may follow.
(Session 233 & 234)
1. Vail DM. Cancer clinical trials: Development and implementation. Vet Clin North Am Small Anim Pract 37(6):1033-1057, 2007.
2. Kummar S, Gutierrez M, Doroshow JH, Murgo AJ. Drug development in oncology;classical cytotoxics and molecularly targeted agents. Br J Clin Pharmacol 2006;62(1):15-26.
3. Kamb A, Wee S, Lengauer C. Why is cancer drug discovery so difficult. Nature Reviews Drug Discovery 2007;6(2):115-20.
4. Kamb A. What's wrong with our cancer models? Nature Reviews Drug Discovery 2005;4(2):161-5.
5. Whitehead J. Stopping clinical trials by design. Nature Reviews Drug Discovery 2004;3(11):973-7.
6. Booth B, Glassman R, Ma P. Oncology trials. Nature Reviews Drug Discovery 2003;2(8):609-10..
7. Von Hoff DD. There are no bad anticancer agents, only bad clinical trial designs: Twenty-first Richard and Hinda Rosenthal Foundation Award Lecture. Clin Cancer Res 1998;4:1079-86.
8. Kummar S, Kinders R, Rubinstien L, et al. Compressing drug development timelines in oncology using phase '0' trials. Nature Reviews Cancer 2007;7(2):131-9.
9. Potter DM. Phase I studies of chemotherapeutic agents in cancer patients: A review of the designs. J Biopharmaceutical Statistics 2006;16(5):579-604.
10. Acevedo PV, Toppmeyer DL, Rubin EH. Phase I trial design and methodology for anticancer drugs. In:Teicher BA, Andrews PA, editors. Anticancer Drug Development Guide, 2nd edition. Totowa NJ; Humana Press; 2004. p. 351-62.
11. Vail DM. Veterinary co-operative oncology group--common terminology criteria for adverse events following chemotherapy or biological antineoplastic therapy in dogs and cats. Veterinary and Comparative Oncology 2004; 2(4): 194-213.
12. Simon R. Optimal two-stage designs for phase II clinical trial. Control Clin Trials 1989; 10(1):1-10.
13. Michaelis LC, Ratain MJ. Phase II trials published in 2002: A cross-specialty comparison showing significant design differences between oncology trials and other medical secialties. Clin Cancer Res 2007;13(8):2400-5.
14. Gray R, Manola J, Saxman S, et al. Phase II clinical trial design: Methods in translational research from the genitourinary committee at the Eastern Cooperative Oncology Group. Clin Cancer Res 2006;12(7):1966-9.
15. Lee JJ, Feng L. Randomized phase II designs in cancer clinical trials: Current status and future directions. J Clin Oncol 2005;23(19):4450-7.
16. Therasse P, Arbuck SG, Eisenhauer EA, et al. New guidelines to evaluate the response to treatment in solid tumors. JNCI 2000;92(3):205-16.
17. WHO handbook for reporting results of cancer treatment. Geneva (Switzerland): World Health Organizations Offset Publication No. 48; 1979.
18. Poirier VJ, Kurzman ID, Thamm DK, Jeglum A, Chun R, Obradovich JE, O'Brien MG , Rodgers F, Phillips B, Vail DM. Liposome-encapsulated doxorubicin (Doxil R) and doxorubicin in the treatment of vaccine-associated sarcoma in cats. J Vet Intern Med 2002;16:726-731.
19. Dagher RN, Pazdur R. The phase III clinical cancer trial. In:Teicher BA, Andrews PA, editors. Anticancer Drug Development Guide, 2nd edition. Totowa NJ; Humana Press; 2004. p. 401-10.
20. Vail DM, ID Kurzman, PA Glawe, MG O'Brien, R Chun, LD Garrett, JE O'Bradovich, R Fred, C Khanna, GT Colbern, PK Working. Stealth liposomal cisplatin versus carboplatin as adjuvant therapy for spontaneously arising osteosarcoma in the dog: A randomized multicenter clinical trial. Canc Chem Pharm 2002;50:131-136.
21. Montori VM, Guyatt GH. Intention-to-treat principle. Can Med Assoc J 2001;165(10):1339-41.
22. Fergusson D, Aaron SD, Guyatt G, Herbert P. Post-randomization exclusion: the intention to treat principle and excluding patients from analysis. Br Med J 2002;325(7365):652-655.
23. Lee JJ, Lieberman R, Sloan JA, et al. Design considerations for efficient prostate cancer chemoprevention trials. Urology 2001;57(4 suppl 1):206-12.
24. Betensky RA. Conditional power calculations for early acceptance of Ho embedded in sequential tests. Statist Med 1997;16:465-77.
25. Lachin JM. A review of methods for futility stopping based on conditional power. Statist Med 2004;24:2747-64.
26. Berry DA. Bayesian clinical trials. Nature Reviews Drug Discovery 2006;5:27-36.
27. Buzdar AU, Ibrahim NK, Francis D, et al. Significantly higher pathologic complete remission rate after neoadjuvant therapy with trastuzumab, paclitaxel, and epirubicin chemotherapy; results of a randomized trial in human epidermal growth factor receptor 2-positive operable breast cancer. J Clin Oncol 2005;23(16):3676-85.
28. Stadler WM. The randomized discontinuation trial: a phase II design to assess growth-inhibitory agents. Mol Cancer Ther 2007;6(4):1180-5.
29. Slamon DJ, Leyland-Jones B, Shak S, et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. NEJM 2001;344(11):783-92.
30. Tsong Y, Chen WJ. Noninferiority testing beyond simple two-sample comparison. J Biopharm Stat 2007;17(2):289-308.
31. Vail DM, Chun R, Thamm DH, Garrett LD, Cooley AJ, Obradovich JE. Efficacy of pyridoxine to ameliorate the cutaneous toxicity associated with doxorubicin containing pegylated (stealth) liposomes: a randomized, double-blind clinical trial using a canine model. Clin Cancer Res 1998;4:1567-1571.