2024_guidline rule_r201-study design.pptx

aaa

1. “ Study Design- Observational and Experimental ”

2. Question Suppose a loved one is diagnosed with a serious disease You are selecting treatment 3 treatment options: A, B, and C 2 outcomes Treatment success: yes/no Safety event: yes/no

3. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) B (N=100) C (N=100)

4. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% B (N=100) Success: 50% C (N=100) Success: 50%

5. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50%

6. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% Which treatment would you choose?

7. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% Which treatment would you choose? They all have the same success rate.

8. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% Which treatment would you choose? They all have the same success rate. A has the lowest safety event rate.

9. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% Which treatment would you choose? They all have the same success rate. A has the lowest safety event rate. B and C are indistinguishable.

10. RCT Comparing A, B, and C Analysis of Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% Which treatment would you choose? They all have the same success rate. A has the lowest safety event rate. B and C are indistinguishable. Choose A…right?

11. Analysis of Patients : 4 Possible Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% 50 50 50 50 15 15 35 35 Success + - SE + - Success + - Success + -

12. Analysis of Patients : 4 Possible Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% 50 50 50 50 15 15 35 35 Success + - SE + - Success + - Success + -

13. Analysis of Patients : 4 Possible Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% 50 50 50 50 15 15 35 35 Success + - SE + - Success + - Success + -

14. Analysis of Patients : 4 Possible Outcomes A (N=100) Success: 50% Safety event: 30% B (N=100) Success: 50% Safety event: 50% C (N=100) Success: 50% Safety event: 50% 50 50 50 50 15 15 35 35 Success + - SE + - Success + - Success + -

15. Our culture is to use patients to analyze the outcomes.

16. Our culture is to use patients to analyze the outcomes. Shouldn’t we use outcomes to analyze the patients?

17. GE_ CT Scan

18. “It is ironic that we take the same clinical trial approach to evaluate all manner of potentially amazing transformative experimental therapies and yet we don’t experiment with the design of the clinical trial itself.” –Don Berry, MD Anderson 1 1 Berry DA. (2015). Brave New World.... Mol Oncol 9: 951-959. Lindsay A. Renfro 19 / 39

19. If I had asked people what they wanted, I would’ve built a faster horse Henry Ford

20. Why

21. We use logic to prove and ……………….intuition to discover

22. What type of Design to chose depends on: Research question/ objective Time and Resource available Outcome of interest Quality of data from various sources

23. Why There were 30 participants in the phase 2 trial for  Biocon’s  drug formulation itolizumab (normally used to treat psoriasis).   There were 100 participants for  Patanjali’s  trial for Coronil, a herbal concoction claimed as a ‘ cure ’ for Covid-19. The phase 3 trial of  Glenmark’s  favipiravir, an antiviral drug, had 150 patients, whereas phase 3 trials are generally  much larger .. 24

24. Why Overall, the analysis showed that of the 201 trials, 87.6% were based in China (49.8%) or in the USA (37.8%).Only eight products or combinations of products were new. Although randomized trials were more common than non-randomized trials, with 152 having some form of randomization, the majority of the trials in this group were open label with only 55 trials including at least single blinding to help minimize investigator bias 25

25. “It takes less time to do it right than it takes to explain why you did it wrong.” - Henry Waldsworth Longfellow

26. Design The choice of design depends on the goal of the trial Choice also depends on the population, knowledge of the intervention Proper design is critical, analysis cannot rescue improper design

27. Thought provoking… 28

28. Back Bone of Study Design Patient centric Comparison Randomisation Placebo 29

29. 30

30. Design should be patient centric It’s a team effort All stake holders to be involved 31

31. The need for control groups in clinical studies recognized, but not widely accepted until 1950s No comparison groups needed when results dramatic: Penicillin for pneumococcal pneumonia Rabies vaccine Use of proper control group necessary due to: Natural history of most diseases Variability of a patient's response to intervention Foundation of Design

32. Why Compare? COMPARATIVE STUDIES

33. Purpose of Control Group To allow discrimination of patient outcomes caused by experimental intervention from those caused by other factors Natural progression of disease Observer/patient expectations Other treatment Fair comparisons Necessary to be informative

34. Considerations in Choice of Control Group Available standard therapies Adequacy of the control evidence for the chosen design Ethical considerations

35. 542-03-# 36 Significance of Control Group Inference drawn from the trial Ethical acceptability of the trial Degree to which bias is minimized Type of subjects Kind of endpoints that can be studied Credibility of the results Acceptability of the results by regulatory authorities Other features of the trial, its conduct, and interpretation

36. 37

37. 38

38. Use of Placebo Control The “placebo effect” is well documented Could be No treatment + placebo Standard care + placebo Matched placebos are necessary so patients and investigators cannot decode the treatment E.g. Vitamin C trial for common cold Placebo was used, but was distinguishable Many on placebo dropped out of study Those who knew they were on vitamin C reported fewer cold symptoms and duration than those on vitamin who didn't know

39. 40

40. 41 There is a Pharmacy in all of us.

41. 542-03-# 42 Randomized Control Clinical Trial Reference: Byar et al. (1976) New England Journal of Medicine Patients assigned at random to either treatment(s) or control Considered to be “ Gold Standard”

42. 542-03-# 43 Disadvantages of Randomized Control Clinical Trial 1. Generalizable Results? Subjects may not represent general patient population – volunteer effect 2. Recruitment Twice as many new patients 3. Acceptability of Randomization Process Some physicians will refuse Some patients will refuse 4. Administrative Complexity

43. 542-03-# 44 Bias of Non-RCT’s Example - Peto (1979) Biomedicine Trials of anticoagulant therapy Design #Patients P<0.05 Observed Effect 18 Historical 900 15/18 50% 8 Concurrent 3000 5/8 50% 6 Randomized 3000 1/6 20% Biases False positives Magnitude of effect

44. 542-03-# 45 Ethics of Randomization Statistician/clinical trialist must sell benefits of randomization Ethics ⇒ MD should do what he thinks is best for his patient Two MD's might ethically treat same patient quite differently Chalmers & Shaw (1970) Annals New York Academy of Science 1. If MD "knows" best treatment, should not participate in trial 2. If in doubt, randomization gives each patient equal chance to receive one of therapies (i.e. best) 3. More ethical way of practicing medicine

45. Age of Bionics 46

46. Overall, phase III failure rates are 45 per cent or more across therapeutic categories, and rise to around 66 per cent for new cancer drugs. "One of the main reasons for pivotal trial failures in cancer to date is that they have enrolled the wrong patient population," 

47. 48 The key to all my research woes! Designs where I can do whatever I want, whenever I want to (ethically) answer my research questions, The "good" designs that statisticians have been selfishly keeping to themselves all this time! "An adaptive design is defined as a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial." (FDA 2018 Adaptive Designs for Clinical Trials Guidance Document). What are adaptive designs ?

48. 49 Group sequential designs (i.e., interim analyses). Adaptations to sample size (i.e., sample size re-estimation based on interim results to preserve power). Adaptations to the patient population (i.e., adaptive enrichment). Adaptations to treatment arm selection (i.e., adding or terminating arms). Adaptations to patient allocation (i.e., adaptive randomization). Adaptations to endpoint selection. Adaptations to multiple design features (combining multiple features above). FDA Adaptive Elements

49. 50  Hypothetical Scenario:   • You design and power a study on a research topic with limited prior information (i.e., there is uncertainty in your sample size calculation assumptions).   • As the study is being conducted, the observed treatment effect is smaller than expected, but still clinically meaningful. • You design and power a study on a research topic with limited prior information (i.e., there is uncertainty in your sample size calculation assumptions).   • As the study is being conducted, the observed treatment effect is smaller than expected, but still clinically meaningful.   • If we maintain the planned sample size, we may be underpowered to detect this difference. Why Adapt the Sample Size?

50. 51 Using interim estimates we can address the prior uncertainty about the treatment effect size These can be blinded or unblinded, however they involve different statistical approaches and the FDA Guidance focuses primarily on the unblinded context FDA recommends steps should be taken to limit personnel with detailed knowledge to maintain trial integrity It can be challenging if the re-estimation suggests the need for a much larger sample size Sample Size Re-Estimation

51. 52 Study powered for com event rates of 5.1% vs. ( Word ? ) study arms → 10,900 patients for 85% power and two-sided a=0.05. Unblinded sample size re estimation planned after 70% enrolled. At the interim analysis, an early stopping efficacy boundary was crossed but DSMB decided to continue the trial as planned (i.e., no sample size increase). Trail Example Effect of platelet inhibitation with cangrelor during PCI on Ischemic Events

52. 53  Hypothetical Scenario:   • You expect the treatment effect to be greater in a certain targeted subset of the trial population:   • Do we enroll only the targeted subpopulation? Why "Enrich" the Patient Population?

53. 54

54. 55 Want information about both the targeted and non-targeted subpopulations Uncertain about treatment effect in non-targeted subpopulation (i.e., perhaps the treatment is as effective or less effective but still clinically meaningful) Can provide greater power relative to a fixed sample design without enrichment (i.e., if we restrict enrolment we have more subpopulation observations versus having equivalent power) Reasons for Population Enrichment

55. 56 Seamless study designs combine multiple phases of a study into one trial. e.g., Phase II and Phase III combined to include both treatment selection and confirmation in one trial. Interim analyses used to determine what continues from Phase II portion of the study to Phase III. Advantages include reducing overall study size, shorter development time, more long term safety information. Disadvantages include logistical challenges and issues maintaining statistical properties. Seamless Designs

56. 57 Compare Treatment 1 and 2 after Phase II and drop least effective arm. Then compare efficacy after Phase III between the SOC and continued treatment using all data from Phases II and III.   Seamless Phase iI/IIi design   One Version of Seamless Phase II/III Designs

57. 58 MAMS can drop ineffective arms early on at an interim analysis. Promising arms seamlessly continue to a (confirmatory) Phase III trial. One disadvantage is that you can only compare "Novel" arms to the Control arm (to maintain the type I error and power). Multi- Arm Multi-Stage

58. 59 The probability of the next treatment assignment is altered on the basis of the previous assignments in order to achieve better balance (i.e., biased coin, minimization procedures). Considerations: How to implement (central entity vs. local entities) Multiple treatments What is considered a lack of balance What covariates to use for balance Main advantage: opportunity to balance composition of treatment groups on several characteristics without stratification Baseline (Covariate) Adaptive Randomization

59. 60 Assignment probabilities are modified based on observed responses or outcomes. The motivation is to allocate as many patients as possible to the "best" treatment arm. Recent research has identified that outcome adaptive randomization may result in randomization to the inferior arm, concerns about sample size imbalance (leading to reduced power), and challenges where time effects are present. Response/Outcome Adaptive Randomization

60. 61 Zelen's 1969 Play the Winner Design (2 arm study):   Assign 1st participant to either arm with equal probability Observe success/failure in arm Depending on outcome:   Observed success leads to use increasing the probability of the successful treatment being assigned for the next participant. Observed failure leads to a decreased probability.   Disadvantages are that sample size/power is challenging to calculate priori and you need to know the previous response before randomizing the next-individual(although you-could-update in blocks). Response Adaptive Randomization Example

61. Master Protocol Designs 62 Master Protocol to study multiple therapies, Multiple diseases or both. Master Protocols Traditionally we have conducted separate standalone studies for at most a few interventions in targeted populations, however these are becoming increasingly expensive and prohibitive Precision medicine and the need for flexible designs to consider multiple drugs, diseases, populations, or combinations of these are needed Master protocols provide a unifying framework that use one master protocol for a study that is designed to answer multiple questions

62. MP Innovation Woodcock and LaVange describe the many areas of innovation that can be found in master protocols Areas of Innovation   Infrastructure Common screening platform for biomarker identification Governance  Steering committee  Adjudication committee  Data monitoring committee Central institutional review board   Trial networks and clinical centers Processes Randomization Data and safety capture and management Quality control oversight   Trial Design Adaptive randomization and other adaptive design features Longitudinal modelling to determine probabilities of success or failure Shared control patients Natural history cohort Bio maker Qualification

63. General Type of Master Protocols 64 Type of Trial Objective Umbrella To study multiple targeted therapies in the context of a single disease Basket To study a single targeted therapy in the context of multiple diseases or disease subtypes. Platform To study multiple targeted therapies in the context of a single disease in a perpetual manner, with therapies allowed to enter or leave the platform on the basis of a decision algorithm

64. 65

65. 66

66. 67

67. 68

68. EGFR inhibitors (e.g., cetuximab, panitumumab) 1 KRAS-WT metastatic colorectal cancer Trastuzumab 2 HER-2 positive metastatic breast cancer Vemurafenib 3 BRAF-V600E-mutant melanoma Erlotinib 4 / crizotinib 5 EGFR-mutated / ALK-mutated lung cancer 1 Jonker et al. (2007). New Engl J Med 357: 2040-2048. 2 Slamon DJ et al. (2001). New Engl J Med 344: 783-792. 3 Chapman et al. (2011). N Engl J Med 364: 2507-2516. 4 Zhou et al (2011). Lancet Oncol 12: 735-742 5 Shaw et al (2013). New Engl J Med 368: 2385-2394. 69 / 39 Lindsay A. Renfro Recent FDA (USA) Approvals

69. Advantage Relatively small sample size Increased “hit rate” by enrolling patients with rare molecular features across tumor types Offer an array of novel therapeutic agents to a broad group of patients who may benefit Dis Advantage Prognostic heterogeneity across tumor types Single arm sub-studies generally require a tumor response rate endpoint (with a high bar) Challenging to define historical controls across diseases For this reason, PFS endpoint (though often relevant) usually not primary and is challenging to assess Multiple testing (though separate studies face the same issue) Lindsay A. Renfro Basket Trials

70. Advantage Relatively improved prognostic homogeneity (all patients from same tumor group) Any observed benefit may be more readily attributed to the marker Particularly true when randomization against a control treatment occurs Even more true when marker-negative patients concurrently randomized to same treatments Treatment-by-marker interaction may be computed Dis Advantage Larger size, particularly when sub-trials are randomized Longer duration Difficulty enrolling rare molecular subtypes of a single tumor type Susceptibility to changes in the “treatment landscape” during the trial E.g., introduction of a new standard of care (may change control arm) Umbrella Trials

71. 72 Example: Lung-MAP Lung-MAP (SWOG S1400) Patients with previously-treated advanced squamous cell lung cancer Initially 3 parallel randomized phase II/III sub-trials for targeted therapy vs. SOC (docetaxel) Goal: 500-1,000 patients screened per year Contains 4th cohort: non-match study for patients not eligible for target cohorts Phase II endpoint: PFS 68-124 patients per sub-study Phase III endpoint: overall survival (OS) with phase II patients contributing 272-336 patients per sub-study No cross-cohort comparisons Initially, non-match patients randomized to anti-PD-L1 immunotherapy vs. SOC

72. 73 Lung-MAP Schema (Original )

73. 74 Lung-MAP Updates / Challenges One initial cohort (c-MET-positive) closed early for toxicity March 2015: FDA approved nivolumab in same patient population Control arm (docetaxel) no longer the standard of care Lung-MAP re-opened with modifications: 1 Control arm dropped in phase II → single arm experimental therapy only New objectives in non-match arm (single vs. combo immunotherapy) Oct 2017: 1,400 patients registered Nov 2017: Non-match sub-study for PD-1/PD-L1-resistant patients 1 http://www.lung-map.org

74. 75 Demonstrated the feasibility of the umbrella design to advance personalized treatment of NSCLC Different responses by mutation type and status: Umbrella Example: Conclusion

75. 76 Basket and Umbrella Trials: Practical and Statistical Issues Practical Issues: ) New collaboration paradigm ) Logistics far beyond a single trial ) Trials must adapt to external changes over years, decades ) Unforeseen screening challenges may affect feasibility Statistical Issues: ) Effect size vs. sample size ) Whether to include an all-marker-negative subgroup ) Classification of patients with multiple markers or genetic mutations

76. 77 Conclusions Basket and umbrella trials → potential solution to challenges of precision medicine Expected increase in popularity as larger, traditional trials become less feasible Need for improved statistical methodology to address design challenges, e.g., rare molecular subtypes

77. 78 Designs can be non comparative or comparative If comparative, you may have a common control group or multiple control groups depending on design Designs can include adaptive elements or not Designs can be exploratory or confirmatory LOTS of flexibility Master Protocols

78. 79 Design of a Randomized Controlled Trial for Ebola Virus Disease Medical Countermeasures: PREVAIL II, the Ebola MCM Study   Lori E. Dodd,' Michael A. Proschan.' Jacqueline Neuhaus. Joseph S. Koopmeiners," James Neaton.' John D. Beigel. Kevin Barrett." Henry Clifford Lane, and Richard T. Davey Jr! National Institute of Allergy and Infectious Diseases, Bethesda, and Leidos Biomedical esearch, Frederick National Laborator for Cancer Research Manland and Division of Biostatistics, School of Public Health University of Minnesota Minneapolis Platform Trial Example

79. 80 No known candidate therabies or vaccines for Ebola. Outcome was 28-day mortality. Design sequentially considered multiple treatments within a single trial to most effectively identify beneficial therapeutics. Used a Bayesian design with frequent interim monitoring (starting after 12 participant outcomes observed, 6 per arm). Used Haybittle-Peto style boundaries for interim monitoring based on the posterior probability of the experimental treatment being better than the current standard of care. Platform Example: PREVAIL ||

80. 81 Due to connects with time effects, only concurrent controls were used in analyses (i.e., only information within each segment).

81. 82 Use method incorporation segment when "exchangeable” (i.e., concurrent data) incorporate segments when formation rolled potentially use non-   Adaptively randomize to maintain information balance between 0 SOC and Experimental arms   Extension to incorporate past Information

82. 83 Should I Consider Adaptive Designs ? Advantages   Improved flexibility. More efficient use of resources (financial and administrative). Greater statistical power possible. Ability to answer broader questions that may be refined as the trial progresses. Challenges   Advanced analytic methods needed to avoid type I errors. Gains in efficiency have trade offs with other trial components. Logistics to maintain trial conduct and integrity. Adaptations may be limited by clinical/scientific constraints.

83. 84 The FDA requires that all trials maintain desired frequentist operate characteristics, including Bayesian trials:   Power Type I error rate   Evaluating trial operating characteristics generally involves extensive simulation studies (i.e., this is how you calculate power, the target sample size, etc.)   Prior specification can be challenging (e.g., conjugate priors, informative priors, vague priors, etc.)   Design Considerations

84. "We have set up a system where everyone can learn faster and, together, we can dramatically reduce the amount of time and the cost to bring those drugs to market that can make a difference in whether women live or die." "Times are changing, and at the highest levels in pharmaceutical companies there is a recognition that cancer drugs aren't going to work for all patients. That means they have to understand who will benefit, and which drugs will work in combination. "Comparing them in the same sandbox is the only way to go,"

85. "It's a real departure from the 'close your eyes, open them at the end' approach of double-blind studies, often followed by the statement: 'Oh rats, I wish I'd have looked where I was going!'"

86. 11/26/2020 87

87. Humans share 98.8% of their DNA with chimpanzees. But thankfully, despite this, the differences lie in how these genes are used .