Lab 04 - Debugging and Profiling

Learning goals

  • Functions, loops, and vectorized programming
  • Writing and simulation a function from a statistical paper
  • Debugging, profiling, and improving efficiency

Lab task

In Lab 1, we developed a function that enrolls 40 participants equally across four arms (one control and three different treatment arms). Using a Beta-Binomial posterior distribution for each arm (with a \(Beta(\alpha_t=0.35,\beta_t=0.65)\)), we estimated \(Pr(p_t > p_0)\).

Today, we will build upon this.

Imagine a trial that will enroll \(N=228\) participants with a control arm and three treatment arms. The trial will be considered a success if at least one treatment is better than control. That is, if \(\max{Pr(p_t>p_0)} > \delta_d\), where \(\delta_d\) depends upon the design.

Consider a trial that uses Equal Randomization (ER) and another that uses Response Adaptive Randomization (RAR) that updates every 40 participants.

The RAR updates the treatment allocation using the following rules:

  • For \(t \in 1, 2, 3\): \(V_t = Pr(p_t \text{ is best across all arms})\)
  • For \(t = 0\): \(V_0 = min\{\sum V_t \frac{(n_t + 1)}{(n_0 + 1)}, max(V_1, V_2, V_3) \}\)

\(V_0, V_1, V_2,\) and \(V_3\) are renormalized to sum to 1 and are allocation probabilities.

For lab:

  1. How will you “modularize” and “vectorize” your solution?

  2. Write a function for each study design to simulate one trial.

  • N = 228 with interim analyses after every 40th participant starting at 40.
  • Use equal allocation for first 40 patients for both designs.
  • Assume a setting where treatment effect is 0.35 for each study arm (the null scenario). (But allow flexibility in function for other treatment effects).
  • \(\alpha_t = 0.35\) for all \(t\) and \(\beta_t = 0.65\) for all arms.
  • Use the following \(\delta\) thresholds to determine a successful trial:
    • Design ER, \(\delta_{ER} = 0.9912\)
    • Design RAR, \(\delta_{RAR} = 0.9892\)

For simplicity, have your function return a list of at least the following output:

  1. The probability that the best treatment arm is better than control.
  2. The number of patients assigned to each treatment arm.

If you have more time

  • Replicate the design many (10K) times. Calculate the Type I error.
  • Find \(\delta\) for each design (supposing you didn’t already know it).
  • Replicate the study design assuming treatment effects of
    • \(p_0 = p_1 = p_2 = 0.35\)
    • \(p_3 = 0.65\)