Phase II single-arm two-stage designs are typically used to determine if a drug is sufficiently efficacious to move on a randomized phase III trial. Sometimes, we would like to design a trial to examine whether the drug is safe due to a significant adverse event or toxicity. These are often called toxicity or tolerability studies instead of safety studies and they occur frequently. Results from a sample of subjects are then compared to an adverse event rate from historical controls. If the drug appears to have a fewer number of patients with a toxic response than expected, the trial proceeds to stage 2 for further research; otherwise, the study terminates at stage 1.

Drug trials estimate the true success rate p of a drug. These are efficacy studies and are conducted in one stage or two stages. In Simon’s two-stage efficacy studies (Simon, 1989), a minimal efficacy response rate p₀ is pre-specified in the null hypothesis versus a user-specified higher rate p₁ in the alternative hypothesis. Given type I and II error rates, there are four variables to optimize the design problem: the patient number n₁ to recruit in stage 1, the total number of patients n required in the entire trial, the rejection number r₁ in stage 1 and the cumulative rejection number r in the entire trial. If the estimated response rate is smaller than p₀, the treatment is deemed not efficacious and the trial stops in the first stage. Otherwise, it proceeds to the second stage, where the number of additional patients and the number of additional responders have to be determined to arrive at a decision for the trial.

These optimization problems are typically solved by a greedy search over the set of constrained positive integers (Kim and Wong, 2017). There are two optimality criteria for Simon’s two-stage designs: a minimax criterion that minimizes the maximum sample size and an optimal design that minimizes the expected sample (Simon, 1989). Since Simon’s landmark paper was proposed, many variations of the strategies have been proposed for phase II designs. Green and Dahlberg (1992) investigated a two-stage design for multicenter trials when the attained sample size is not the planned one. Mander and Thompson (2010) and Mander et al. (2012) proposed novel designs that are optimal under the alternative hypothesis. Wason et al. (2011) proposed reducing the sample size for phase II trials with a continuous outcome, and Kwak and Jung (2014) considered a two-stage adaptive optimal design for single arm trials with right-censored survival time. Ensign et al. (1994), Chen (1997), and Chen and Shan (2007) proposed phase II designs with three stages. Under the Bayesian framework, Thall and Simon (1994) proposed posterior probability to account for uncertainty regarding the response rates of the null and alternative hypotheses. Lee and Liu (2008) developed an efficient and flexible design using Bayesian predictive probability and the minimax criterion. Cai et al. (2014) constructed a phase II trial design for continuous monitoring when delayed responses are present using multiple imputation. Zhou et al. (2017) recently proposed a flexible Bayesian optimal phase II (BOP2) design to handle simple and complex endpoints such as binary, ordinal, nested, and co-primary endpoints.

Several phase II single-arm two-stage designs are available for toxicity studies, but those studies are designed to monitor toxicity as a secondary or a co-primary endpoint to determine if a drug or treatment is efficacious; see, Bryant and Day (1995), Conaway and Petroni (1996), Ray and Rai (2011), and references. Therefore, those designs are not directly applicable for safety studies where the primary endpoint is the proportion of adverse events or toxicity. The motivation for this work is that there are no phase II single-arm safety studies that utilize Simon’s two-stage designs. There are two possible reasons for this: (i) there are allusions that this is possible but the theoretical justifications have not been worked out, and (ii) there is no software package to generate a two-stage design for a safety study. PASS 15 (NCSS, LLC), a widely used power and sample size calculation commercial software package, has an option for Simon’s two-stage designs under the ‘Proportions: One Proportion: Group-Sequential: Two-Stage Phase II Clinical Trials’ category, but cannot handle safety cases where a null proportion p₀ is larger than an alternative proportion p₁ (i.e., p₀ > p₁). Similarly, STATA 15.0 (StataCorp, LLC) and nQuery 8.0 (StatSols, Ltd.) only have a code to generate a Simon’s two-stage design for an efficacy trial but not for a safety trial.

As an example of a safety study, Rugo et al. (2017) used a one-sample proportion test to estimate the required sample size for a phase II single-arm study to assess the incidence of grade 2 or worse everolimus-related stomatitis in women when dexamethasone mouthwash (SWISH) is used. On the basis of historical controls, the incidence rate of grade 2 or worse stomatitis without SWISH was considered to be 0.33 (i.e., the null hypothesis is p₀ = 0.33). The alternative hypothesis is that the use of SWISH would lead to an absolute reduction of 0.13 (i.e., p₁ = 0.20). A direct calculation shows that the estimated required sample size was 73 for a one-sided 5% sized test with 80% power using a one-sample proportion test. If two-stage designs are applied to this trial using our proposed approaches, the estimated sample sizes are (r₁, n₁, r, n) = (18, 26, 63, 85) and (50, 67, 54, 72) for the optimal and minimax designs, respectively. Therefore, they could have an opportunity to evaluate the study early at the end of the first stage using either minimax or optimal designs that could also save 3 patients compared to their original design if the minimax design was employed. We provide details on how to apply a two-stage design to this SWISH trial in Section 3.2.

We now develop a theory to construct a two-stage safety trial following Simon’s original two-stage design. In particular, we provide analytical formulas parallel to those for an efficacy study and show how a two-stage safety study can be found directly from Simon’s two-stage design. We also compare single-stage and two-stage designs for safety studies through simulation studies and apply our results to construct a phase II two-stage design for a real application (Rugo et al., 2017). The user can implement our phase II single- and two-stage designs for a single-arm trial using our R package called gen2stage available at http://cansur.wayne.edu/.

Throughout, let b(x, n, p) and B(x, n, p) be the probability mass function (pmf) and the cumulative distribution function (cdf), respectively, of a random variable X that follows the binomial distribution with parameters n and p, i.e., X ~ Binom(n, p). We now describe the theoretical relationships between designs for safety studies and those for efficacy studies.

2.1. Single-stage design

A single-stage design is estimated using Fleming’s single-stage procedure with the exact binomial distribution (Fleming, 1982; A’Hern, 2001). The true rate (or proportion) is denoted by p. The standard rate is assumed to be p₀ under the null hypothesis (i.e., H₀ : p = p₀) and the new treatment can use further research if the null hypothesis is rejected in favor of the alternative hypothesis, where the rate is p₁ (i.e., H₁ : p = p₁). For a safety study, the toxicity (or adverse event) rate is the primary endpoint. We wish to test the toxicity rate is p₀, which is the maximum allowed; consequently, we reject the null hypothesis and terminate the study if there are too many toxic responses among the first cohort of patients. In the alternative hypothesis, the target toxicity rate is p₁ with p₀ > p₁. Then the single-stage design aims to find a good design by testing H₀ : p = p₀ vs. H₁ : p = p₁ with predetermined p₀, p₁, type I error rate α and type II error rate β, respectively. The single-stage design for the efficacy or safety studies is performed as:

• Begin by recruiting n patients of the single-arm phase II design and observe the number of the responses or toxicity, x, out of n patients.

• When p₀ < p₁ (for efficacy), the null hypothesis is H₀ : p = p₀ and the alternative hypothesis H₁ : p = p₁ with p₁ > p₀. The decision rule is to fail to reject the null hypothesis H₀ if the total number x of responses is less than r (i.e., x ≤ r).

• When p₀ > p₁ (for safety), the null hypothesis is H₀ : p = p₀ and the alternative hypothesis H₁ : p = p₁ with p₁ < p₀. The decision rule is to fail to reject the null hypothesis H₀ if the total number x of subjects experiencing toxicity is larger than r (i.e., x ≥ r).

The single-stage design has two parameters θ = (r, n) to be optimized. Given error rates α and β, their optimal values are found by a greedy search. If the true rate is p, the probability of failing to reject H₀ is given by

where B is the cdf of a binomial distribution with parameters n and p. Note that we hereafter use the subscripts of 1 and 2 to indicate the probability of failing to reject H₀ for the single-stage and the two-stage designs, respectively. To find an appropriate design, we first identify a set Θ̃ that contains all θ that satisfies the two user-specified error constraints:

The optimal choice of θ̂ in Θ̃ may be determined by the following optimality criterion for both efficacy and safety cases:

where n_θ is the required sample size corresponding to θ = (r, n).

2.2. Two-stage design

The two-stage design evaluates the trial endpoint at each stage and allows the trial to proceed to the second stage only if one rejects the null hypothesis at the end of the first stage. The required sample sizes for the first and second stages are denoted by n₁ and n₂, respectively, where the total required sample size n = n₁ + n₂. Using similar notation used in the single-stage design, the two-stage design for efficacy or toxicity tests H₀ : p = p₀ versus H₁ : p = p₁ with p₀ < p₁ or H₁ : p = p₁ with p₁ < p₀, respectively, as:

• Step I: Begin by recruiting n₁ patients in the first stage of the trial and observe the number of the responses or toxicity, x, out of n₁ patients.

• Step II:

  • When p₀ < p₁ (for efficacy), if x ≤ r₁, stop the trial and fail to reject H₀ (i.e., p ≤ p₀); if x > r₁, power the study at (1 − β) for p = p₁ and enter n₂ = n − n₁ additional patients into the study. Reject the alternative hypothesis H₁ (i.e., p ≥ p₁) if the total number of responses ≤ r out of n patients.

  • When p₀ > p₁ (for safety), if x ≥ r₁, stop the trial and fail to reject H₀ (i.e., p ≥ p₀); if x < r₁, power the study at (1 − β) for p = p₁ and enter n₂ = n − n₁ additional patients into the study. Reject the alternative hypothesis H₁ (i.e., p ≤ p₁) if the total number of toxicity ≥ r out of n patients.

The two-stage designs for efficacy or safety have four parameters θ = (r₁, n₁, r, n) to optimize given error rate constraints. The probability of failing to reject H₀ when the true rate is p is given by

where b and B are the pmf and cdf of a binomial distribution with n = n₁ + n₂. Furthermore, the probability of early termination (PET) at the end of the first stage is

where the subscripts P and S hereafter indicate the designs for testing H₀ : p = p₀ versus H₁ : p = p₁ with p₀ < p₁ for efficacy and designs for testing H₀ : p = p₀ versus H₁ : p = p₁ with p₀ > p₁ for safety, respectively. The expected sample size with p as the response probability is then

The sought two-stage design for evaluating toxicity is to find a design θ̂ ∈ Θ̃, where Θ̃ contains all θ that satisfies two natural error constraints:

The goodness of θ̂ may be determined by one of the following two optimality criteria:

where n_θ is the required sample size corresponding to θ = (r₁, n₁, r, n).

2.3. Theoretical results

We describe several relationships between the designs for efficacy, p₀ < p₁, and those for safety, p₀ > p₁. These relationships enable us to find one or two stage optimal designs for toxicity studies for testing p = p₀(> p₁) versus H₁ : p = p₁ from optimal designs for efficacy studies to test p = p₀(< p₁) versus H₁ : p = p₁. Suppose a random variable X follows the binomial distribution with parameters n and p, i.e., X ~ Binom(n, p). An important result which we will use repeatedly and without further mention is b(x, n, p) = b(n − x, n, 1 − p), where x, n ∈ N, x ≤ n, and p ∈ [0, 1].

We now discuss several useful relationships between decision rules to test toxicity and efficacy rates with justifications.

For a single-stage design, if θ̂ = (r, n) is the optimal single-stage design for p₀ < p₁, then ϕ̂ = (n − r, n) is the optimal single-stage design for 1 − p₀ > 1 − p₁and vice versa.

Consider a two-stage design with θ = (r₁, n₁, r, n). Then P₂(θ|p) = S₂(ϕ|1 − p), where ϕ = (n₁ − r₁, n₁, n − r, n).

Proof

Using Theorem 1, calculations of the PET and the expected sample size for toxicity studies are similar to those for efficacy studies. It can be shown that for a two-stage design with θ = (r₁, n₁, r, n), we have PET(p|θ) = PET(1 − p|ϕ) and E_P(N|p, θ) = E_S (N|1 − p, ϕ), where PET is the probability of early termination after the first stage and ϕ = (n₁ −r₁, n₁, n − r, n). Therefore, on the basis of Theorem 2 and Corollary 2, two-stage designs for p₀ > p₁ can be estimated by finding the two-stage designs for p₀ < p₁, respectively.

If θ̂ = (r₁, n₁, r, n) is the optimal two-stage design for p₀ < p₁, then ϕ̂ = (n₁ − r₁, n₁, n − r, n) is the optimal two-stage design for 1 − p₀ > 1 − p₁and vice versa. This relationship also holds for the minimax two-stage design.

Proof

Since θ̂ is the optimal two-stage design,

However, by Theorem 2, finding the set of feasible solutions Θ̂ that satisfies P₂(θ|p₀) ≥ 1 − α and P₂(θ|p₁) ≤ β is equivalent to finding a set of feasible solutions ϕ̂ that satisfies S₂(θ|p₀) ≥ 1 − α and S₂(θ|p₁) ≤ β. That is, θ = (r₁, n₁, r, n) ∈ Θ̂ if and only if ϕ = (n₁ − r₁, n₁, n − r, n) ∈ ϕ̂, resulting that max_θ∈Θ̂ {n₁ + B(n₁ − r₁ − 1, n₁, 1 − p)n₂} = max_ϕ∈ϕ̂ {n₁ + B(n₁ − r₁ − 1, n₁, 1 − p)n₂}. Therefore, by Equation (2.8), ϕ̂ = arg max_ϕ∈ϕ̂E_S (N|1 − p₀, ϕ), showing that ϕ̂ = (n₁ − r₁, n₁, n − r, n) is the optimal design for 1 − p₀ > 1 − p₁.

When θ̂ is the minimax two-stage design, by Theorem 2, we have min_θ∈Θ̂n_θ = min_ϕ∈ϕ̂n_ϕ; therefore, ϕ̂ = arg min_ϕ∈ϕ̂n_ϕ by Equation (2.9), showing that ϕ̂ = (n₁ − r₁, n₁, n − r, n) is the minimax design for 1 − p₀ > 1 − p₁

It follows from Theorem 2 and Corollary 2. The two-stage design for testing toxicity when p₀ > p₁ can be obtained by finding the two-stage designs for p₀ < p₁ and conversely.

Here is an example of a two-stage design for testing toxicity.

3.1. Simulation studies

We use simulation studies to show optimal single- and two-stage designs for toxicity studies using various design parameters where p₀ > p₁. Our simulation covers two sets of type I (α) and type II (β) error rates with (α, β) ∈ {(0.05, 0.20), (0.10, 0.20)} and seven sets of target toxicity rates with 0.3 ≤ p₀ ≤ 0.9 and p₀ − p₁ = 0.2. The estimation procedure is the same as the original Simon’s two-stage design (Simon, 1989).

Table 1 shows single-stage and two-stage designs when α = 0.05 and β = 0.2. We observe that the required total sample sizes (n) of the single-stage design are larger than the minimax two-stage design; however, the single-stage design has the smaller required total sample sizes than the optimal two-stage design except when p₀ = 0.3. As expected, the optimal two-stage design has a smaller expected total sample size under the null hypothesis E(N|p₀) than the minimax two-stage design. Interestingly, the single-stage design has smaller estimates of β than the two-stage designs, implying that β is underestimated under the single-stage design. Two-stage designs have advantages over single-stage designs because they allow the early evaluation of the plausibility of the null hypothesis and can save up to 3 patients compared to the single-stage design. It is noteworthy that the critical values, r₁ and r, are the minimum number of adverse events or toxicity leading to a rejection of a null hypothesis different from the efficacy designs. As an example, suppose 0.4 = p₀ > p₁ = 0.2 and the minimax design is used. We then terminate the study at the end of stage 1 if 5 or more toxicities occur out of 13 patients for excess toxicities. Otherwise, the study proceeds to the second stage and enrolls additional 22 patients. If 10 or more toxicities occur out of 35 patients at the end of the second stage, we conclude that the experimental therapy has excess toxicities. Otherwise, the experimental therapy is considered for further investigation in subsequent trials.

Table 2 displays the simulated single-stage and two-stage designs when α = 0.10 and β = 0.2. Similarly, as observed when α = 0.05 in Table 1, the required total sample sizes (n) for the minimax two-stage designs are always smaller than the single-stage designs. However, the optimal two-stage designs require the same or larger total sample sizes than single-stage designs except when p₀ = 0.3. One interesting case is when p₀ = 0.9 and all designs have the same required total sample size (n = 18) and the same rejection boundary (r = 15). Moreover, the rejection boundary (r₁ = 7) and the required sample size (n₁ = 7) at stage 1 are the same across the two two-stage designs, meaning that, if all 7 patients experience toxicities at the end of the first stage, we terminate the study for excessive toxicities.

3.2. Stomatitis study

We revisit the stomatitis study conducted by Rugo et al. (2017). The purpose of that study was to assess the incidence of grade 2 or worse everolimus-related stomatitis in women who used dexamethasone mouthwash (SWISH). The study assumed that the incidence rate of grade 2 or worse stomatitis without SWISH was 0.33 (i.e., p₀ = 0.33) based on historical controls and then proposed that the use of SWISH would lead to an absolute reduction of 0.13 (i.e., p₁ = 0.20). We now use Corollaries 1 and 2 to construct a single-stage design and a two-stage design under the hypotheses that 0.67 = 1 − 0.33 = p₀ < p₁ = 1 − 0.20 = 0.80 for specified type I and II error rates.

Suppose α = 0.05 and β = 0.20. Since the optimal exact single-stage design with 0.67 = p₀ < p₁ = 0.80 is (r, n) = (55, 73), the optimal exact single-stage design with 0.33 = p₀ > p₁ = 0.20 becomes (r, n) = (18, 73) and the estimated α̂ = 0.047 and β̂ = 0.196. By Corollary 1, The minimax and optimal two-stage designs with 0.67 = p₀ < p₁ = 0.80 are (r₁, n₁, r, n) = (50, 67, 54, 72) and (18, 26, 63, 85), respectively. It follows that, by Corollary 2, the minimax and optimal two-stage designs with 0.33 = p₀ > p₁ = 0.20 become (r₁, n₁, r, n) = (50, 67, 54, 72) and (18, 26, 63, 85), respectively. The estimated α̂ and β̂ are 0.049 and 0.200 for the minimax design and 0.050 and 0.196 for the optimal design. Compared to the single-stage design, the two-stage designs have the flexibility to stop the trial early when there is evidence of excess adverse events. Consequently, patients can benefit from the two-stage designs by not having them exposed unnecessarily to treatments with undue adverse events. Therefore, employing the optimal two-stage design can assess the incidence of adverse events early as a form of an interim analysis at the end of stage 1 and protect patients. For example, if 18 patients experienced grade 2 or worse everolimus-related stomatitis out of 26 patients at the end of stage 1, we can terminate the study for excessive adverse events and do not have to wait till the trial concludes.

The primary objective of a phase II single-arm study is often to assess safety and/or tolerability of a certain drug or treatment by the incidence of adverse events or toxicity. A phase II single-arm safety study aims to show that the rate of an adverse event is lower in the experimental therapy than that in the historical control (i.e., p₀ > p₁). In contrast, a phase II single-arm efficacy study aims to show that the rate of having a responder is higher in experimental therapy than in the historical control (i.e., p₀ < p₁). This is one of the major differences between safety and efficacy studies.

Two-stage designs are widely used in phase II single-arm efficacy studies because of the flexibility to stop early due to futility and avoid the unnecessary exposure of patients to ineffective therapies. However, there appears to be no phase II single-arm safety studies that employ two-stage designs. This means that current safety studies have no opportunity to stop a trial early due to futility. One possible explanation may be a lack of theoretical justification and dedicated software. Our work shows that the traditional Simon’s two-stage designs to evaluate efficacy in a single one-arm trial can also be used for a one or two-stage safety trial with p₀ > p₁. We presented their analytical formulas and established relationships between the two types of designs and investigators can now easily find single-stage or two-stage designs for a safety study using available designs for an efficacy study.

Jung et al. (2001) proposed a graphical approach to find a suboptimal two-stage design that is a trade-off between optimal and the minimax designs. This approach graphically searches for the suboptimal design solely based on the required total sample size (n) and the expected null sample size E(N|p₀). It is not difficult to observe that Corollary 2 shows that optimal and minimax two-stage designs for safety hold for the same n and E(N|p₀) as those for efficacy, implying that Corollary 2 is valid for suboptimal two-stage designs for safety and efficacy. Therefore, the graphical approach can be applied to find two-stage safety designs to find a suboptimal two-stage design.

It is noteworthy that the Bayesian optimal phase II (BOP2) design (Zhou et al., 2017) can monitor the toxicity endpoint through the binary toxicity endpoint at http://www.trialdesign.org/. However, in order to implement the BOP2 design, a user needs to provide preselected sample sizes for the first and second stages, while the proposed approach requires a desirable power. The BOP2 design computes the achieved power based on fixed sample sizes (i.e., a post-hoc power calculation), while our proposed approach is to estimate the sample sizes based on a pre-specified power (i.e., the prior sample size estimation).

To facilitate practitioners to use our single-stage and two-stage designs for a phase II single-arm study, including Jung et al.’s graphical approaches, we implement our computer codes in an R package gen2stage. Using the gen2stage package, investigators can generate single-stage or two-stage designs for an efficacy study or a toxicity study whether we are concerned with p₀ < p₁ or p₀ > p₁.

The Biostatistics Core is supported, in part, by NIH Center Grant P30 CA022453 to the Karmanos Cancer Institute at Wayne State University. WKK was partially supported by NIH Grant R01GM1076 39. The content is solely the responsibility of the authors and does not necessarily represent the official views of NIH.