We propose Sequential Patient Recruitment Monitoring (SPRM), a new monitoring procedure for patient recruitment in a clinical trial. Based on the sequential probability ratio test using improved stopping boundaries by Woodroofe, the method allows for continuous monitoring of the rate of enrollment. It gives an early warning when the recruitment is unlikely to achieve the target enrollment. The packet data approach combined with the Central Limit Theorem makes the method robust to the distribution of the recruitment entry pattern. A straightforward application of the counting process framework can be used to estimate the probability to achieve the target enrollment under the assumption that the current trend continues. The required extension of the recruitment period can also be derived for a given confidence level. SPRM is a new, continuous patient recruitment monitoring tool that provides an opportunity for corrective action in a timely manner. It is suitable for the modern, centralized data management environment and requires minimal effort to maintain. We illustrate this method using real data from two well-known, multicenter, phase III clinical trials.
In a clinical trial, successful recruitment of patients is important. Since inadequate recruitment results in reduced sample size and may weaken the validity of the scientific findings, maintaining a steady stream of patient accrual is crucial. In fact, in a large clinical trial involving multiple sites, a periodic report of recruitment statistics is an integral part of clinical trial management.
At the design stage the project time and desired sample size are fixed to achieve the power of the statistical test associated with the primary endpoint. Throughout the trial, investigators monitor the enrollment process to see whether the number of subjects is projected to reach the goal. According to Lasagna’s law (van der Wouden
There are numerous reasons that the patient enrollment does not go as planned. No matter what causes the delay in the recruitment process, efficient monitoring tools give researchers a window of opportunity to detect and address the problem in a timely manner.
Currently there are a number of methods for patient recruitment monitoring. A simple but popular method uses ad-hoc rules that consist of a set of fixed proportions. At any given time during the recruitment period, recruitment is deemed “on the way” if the cumulative number of participating patients is at least 75% of the target number, prorated at that point in time. For example, suppose the recruitment plan is to randomize 400 patients in a 24-month period. Then the target enrollment at the six-month mark is 100 randomized patients. If the current patient count is at least 75, it is considered satisfactory. This heuristic approach reflects that there is a substantial variation in actual recruitment with the expectation that the recruitment will pick up pace later on.
In other approaches, prorated target numbers serve as the basis for recruitment indices. See Corregano
There are recruitment monitoring tools based on probability models and stochastic processes. Carter (2004) used the Poisson distribution to model the distribution of the accrual period. Zhang and Lai (2011) used fractional Brown motion, while Zhang and Long (2010) used a non-homogeneous Poisson process and cubic B-spline to model the accrual distribution. Anisimov and Fedorov (2007) applied the Poisson-gamma model to multi-center clinical trials, and Gajewski
These stochastic monitoring tools are a significant improvement over deterministic approaches, but they often share the same weakness: the stochastic models often require theoretical assumptions to be held during the entire recruitment process. In practice, these assumptions are difficult to verify prior to actual recruitment. Besides, according to Rengerink (2014), 11% of clinical trials changed the inclusion criteria to make it easier to meet the recruitment goal. Other protocol modifications that affect the sample size and enrollment period are also not uncommon, and it is not easy to incorporate such changes into existing deterministic or stochastic models.
In a different approach, Haidich and Ioannidis (2001) point out that the enrollment in the first month or two is strongly correlated with subsequent accrual. They argue that early enrollment records provide strong evidence for the feasibility of required accrual. However, such an assessment is subjective by nature and cannot be easily formulated to provide an actionable monitoring tool. In a large clinical trial, multiple sites start recruitment at different times and accrual rates so the initial assessment is more complicated than in a single center trial. Moreover, patient recruitment is often nonlinear, and feedback and interventions are quite common.
In this paper, we propose Sequential Patient Recruitment Monitoring (SPRM), a new monitoring method based on the sequential probability ratio test. The sequential test was originally developed during the Second World War for military applications (Wald, 1947). Compared to other statistical tests with the same type I and type II errors, it minimizes the expected sample size. SPRM inherits the desirable properties of this fully sequential approach, thus suitable for continuous monitoring. SPRM uses boundaries by Woodroofe (1982). Based on nonlinear renewal theory, these boundaries are an improvement over Wald boundaries.
By organizing recruitment entry times into a series of packet data, i.e., non-overlapping groups of observations, SPRM requires only minimal assumptions on the accrual distribution. Consequently, it is applicable to a wide range of recruitment scenarios.
The monitoring method allows for timely feedback on whether enrollment is going as planned. If the enrollment rate falls significantly below a specified level during monitoring, it triggers a warning. At this point, SPRM can be used to estimate (1) the size of shortfall, (2) the probability of reaching the target enrollment, and (3) possible extension of recruitment period, based on the assumption that the recent trend that triggered the warning will continue. The monitoring method can also suggest a required enrollment rate for the remainder of the term to reach the target with high probability. This information could be useful for boosting enrollment by giving incentives to existing sites or adding new sites. At reset, hypotheses are updated to reflect these decisions and SPRM resumes monitoring, admitting a new set of patient recruitment data.
However, our approach does not call for a drastic change in clinical trial design or sample size in an adaptive fashion. In fact, SPRM assumes that the enrollment goal is fixed by the original design. The purpose of SPRM is more modest in that its primary function is to inform the users whether that goal is realistic, based on the evidence from data.
In the next section we formally introduce SPRM. In Section 3, we illustrate the method using actual recruitment data from two well-known, phase III clinical trials. In Section 4, we discuss the key features, limitations and recommendations.
Consider a clinical trial with target sample size
where
For
and
where
Let
Then the log-likelihood ratio test statistic ℓ
Let
Table 1 lists
Under SPRM, after a decision is made by either accepting or rejecting the null hypothesis, a new test is proposed with updated hypotheses and a new parameter
where
Wald boundaries for the sequential test (Wald, 1947) have been around for a long time, and they are popular due to simplicity and ease of use. However, these stopping boundaries are asymptotically incorrect by a constant factor, as shown in Theorem 3.1, Woodroofe (1982). Consequently, Wald boundaries produce a wider interval than necessary. The test based on Wald boundaries tends to be somewhat conservative in that the empirical error probabilities are smaller than the nominal values. One of the practical implications is that it takes a longer wait until a decision is made, and it may miss the warning signal entirely. For comparison, Table 2 lists the sample mean and sample standard deviation of the optimal stopping time for the two boundaries when the data were generated from
Woodroofe boundaries for the normal distribution assume the standard deviation is known. In practice, this is rarely the case so it should be estimated from data. Rather than starting monitoring from the very beginning, we recommend using a portion of the initial enrollment data for the parameter estimation before the formal monitoring starts. In a large, multi-center clinical trial, individual centers often start at different times and enrollment rates. So usually it takes awhile for the recruiting process to be stabilized, and allowing a lead-in time makes practical sense in this respect.
It is desirable to strike a balance between more precision for estimation and a shorter wait until monitoring starts. With lack of systematic approach to suggest an optimal starting time, we suggest the first 10 to 15% of the target enrollment to be used for initial estimation. This ad-hoc number is based on simulation studies and our field experience.
Let
where
If the null hypothesis is rejected, it is an indication that the accrual process is significantly underperforming. When this happens, a quantity of interest is the probability of reaching the target enrollment by the end of the recruitment period, under the assumption that the current trend will hold for the remainder of the term.
To estimate this probability, first recall that
where
At the time of rejection, one may also be interested in the enrollment rate needed to recruit the remaining patients with probability
Note that the first term on the right-hand side of
In practice, it may be easier to think in terms of the average number of new recruits per day. In such cases, the desired average daily rate is
After the null hypothesis is rejected, the accrual rate for the remaining period should be increased to reach the target recruitment goal. This may be difficult to accomplish due to several reasons, e.g., rarity of the disease or stringent inclusion/exclusion criteria. Then an extension of the recruitment period could be an option.
Suppose after the first rejection, one wishes to determine how much extension is needed to reach the target recruitment
The performance of
Another way to increase recruitment is to add more recruitment sites without extending the accrual time. Suppose the accrual rate remains approximately the same for the remaining duration. In
We can use
In this section we illustrate the sequential monitoring method using enrollment data from two phase III, multi-center clinical trials. For both cases, we used the first 15% of the entry data for the initial setup and estimation. We also used 5% Type I and II errors,
ENhancing recovery in coronary heart disease (ENRICHD) (Berkman
At the second rejection on day 504, one may wish to estimate the probability of reaching the target enrollment at the end of the period. To do so, we use the formula (
Suppose an extension of accrual period is considered at that point. Utilizing formula (
Beta-blocker Heart Attack Trial (BHAT) (Byington, 1984) was a double-blind, randomized, placebo-controlled multi-center clinical trial where 30 U.S. sites and 1 Canadian site participated. The purpose of the study was to investigate whether long-term propranolol therapy significantly reduces all-cause mortality for patients with acute MI. The original enrollment plan for the trial was to recruit 4,200 patients in 27 months. Randomization started in June 1978 and ended in October 1980. The actual number of enrolled patients was 3,837 or 91.4% of the original plan.
In this illustration, the formal monitoring commenced five months after recruitment started. In Figure 2, between the sixth month and one year mark, SPRM signaled six times (marked by solid circles) to indicate the recruitment was on target to meet the 4,200 patient goal, if the trend continued. Starting at the two year mark, however, SPRM gave three warning signals in succession (marked by solid triangles). These are a strong indication that the recruitment would likely fall short of the target enrollment.
The estimated probability of reaching the target enrollment, at the time of the last acceptance (day 375) is 0.508. Such a seemingly low probability follows the fact that the nominal target rate concerns the average inter-arrival time of successive recruitment. In a clinical trial where the success of reaching the target is crucial, one may desire a higher probability, say 0.95 instead. Since the nominal target rate is 53 new recruitment per 10 days and the adjustment factor is estimated 1.07, the desired daily rate is 56 patients to ensure success with a 95% chance. The desired target rate at the specified probability could serve as a practical guideline during the recruitment period.
Statistical patient monitoring methods often assume a certain form for the underlying distribution of the time between successive recruitments. In practice, this is difficult to know ahead or even during the trial because the recruitment patterns rely on individual clinical trials, characteristic of the disease, severity, inclusion/exclusion criteria among others. Besides, a misspecification of the underlying distribution often leads to poor inference.
One way to solve the problem is to use the sequence of sample means calculated from the series of non-overlapping packets of enrollment data. Suppose the observations are iid with a common distribution with finite mean and variance. For an adequate packet size
An obvious downside of this approach is that it reduces the effective sample size by the factor of
In the illustrations, we chose 0.75 for the design parameter
There are other limitations in the current implementation. We recommended an ad-hoc range of 10–15% of planned recruitment for the timing of the initial estimation of the standard deviation, a nuisance parameter. A more formal procedure would be desirable to determine the optimal start time for monitoring. In the current approach, we did not consider patient dropouts. This is because dropouts are expected in the course of a clinical trial so it is usually taken into account when the initial recruitment goal is determined. However, unexpected excessive dropouts have a negative impact on recruitment so it may well be incorporated into the monitoring process.
To estimate the shortfall at the end of the accrual period, we used the parameter under alternative hypothesis when the test rejected the null hypothesis. Another possible approach is to use the estimate for the average daily accrual rate using data collected up to that point. In such a case, it may require bias correction because the estimation is made after the sequential test. This is a well-known fact in sequential inference and there are standard bias correction methods available. We did not take the approach here, but it would be an interesting comparison to our method.
The repeated significance test may have a theoretical edge over our current approach when the parameters are unknown. However, one could also argue that the goal of SPRM is to give a simple but practical guideline for stakeholders in clinical trials.
In spite of these limitations, SPRM is a useful addition to the toolbox that is available to the sponsor, study investigators and DCC. SPRM can help stakeholders make an informed and timely decision regarding patient recruitment in a clinical trial.
The authors wish to thank Drs. Nancy Geller and Eric Leifer at the Office of Biostatistics Research, NHLBI for reading the early draft and providing insightful comments.
The views expressed in this article are those of the authors and do not represent the views of NHLBI, NIH, OSEHRA, or the University of North Carolina.
Woodroofe boundaries
0.2 | 0.89 | 2.88 |
0.4 | 0.79 | 2.76 |
0.6 | 0.71 | 2.65 |
0.8 | 0.63 | 2.53 |
1.0 | 0.56 | 2.42 |
1.2 | 0.50 | 2.30 |
1.4 | 0.45 | 2.19 |
1.6 | 0.40 | 2.08 |
Optimal stopping time
sd( | sd( | |||
---|---|---|---|---|
0.2 | 137.51 | 99.63 | 144.21 | 103.54 |
0.4 | 33.83 | 24.08 | 37.26 | 26.30 |
0.6 | 15.28 | 10.65 | 17.52 | 11.90 |
0.8 | 8.71 | 6.09 | 10.34 | 6.99 |
1.0 | 5.74 | 3.97 | 7.06 | 4.67 |
1.2 | 4.02 | 2.71 | 5.14 | 3.31 |
1.4 | 3.06 | 2.02 | 3.99 | 2.52 |
1.6 | 2.42 | 1.58 | 3.23 | 2.00 |
Adjustment factor
100 | 300 | 500 | 700 | 900 | 1100 | 1900 | |
---|---|---|---|---|---|---|---|
0.5 | 1.08 | 1.05 | 1.04 | 1.03 | 1.03 | 1.02 | 1.02 |
1.0 | 1.16 | 1.09 | 1.07 | 1.06 | 1.05 | 1.05 | 1.04 |
1.5 | 1.25 | 1.14 | 1.11 | 1.09 | 1.08 | 1.07 | 1.06 |
2.0 | 1.33 | 1.19 | 1.15 | 1.12 | 1.11 | 1.10 | 1.07 |
Additional recruitment time (days)
ART | sd(ART) | sd( | Power | |||
---|---|---|---|---|---|---|
1,000 | 133.8 | 5.6 | 1025 | 30.2 | 60.1 | 0.93 |
2,000 | 130.5 | 2.8 | 2036 | 43.4 | 30.0 | 0.92 |
3,000 | 129.0 | 1.9 | 3046 | 53.8 | 20.0 | 0.92 |
4,000 | 128.0 | 1.4 | 4052 | 63.0 | 15.0 | 0.92 |