Goodness-of-fit techniques are an important problem in statistical analysis. In survival and reliability experiments, censored data occur frequently. As much as nonparametric methods are widely used, the parametric approach also plays an important role in survival analysis. Hence we need to take into account a testing problem for censored data as well as complete data. Usually the goodness-of-fit tests for complete data are adapted to censored cases according to censoring types. Regarding censoring types, the most common and simplest censoring schemes are type I or type II censoring. In this paper, we deal with random censoring. It is more general censoring type and occurs frequently in medical studies. For several censoring types and general theory of survival analysis, we refer Lee and Wang (2003), and Tableman and Kim (2004).
As for random censoring, Koziol and Green (1976), Koziol (1980), and Nair (1981) modified test statistics based on the empirical distribution function (EDF) or weighted empirical process. In Koziol and Green (1976), the Cramér-von Mises statistic is generalized to randomly censored data using the Kaplan-Meier product limit estimator of the distribution function. Chen (1984) studied a correlation statistic for randomly censored data.
In this paper, we study test statistics for normal distributions with unknown location and scale parameters when data are randomly censored. When lifetimes are assumed to follow a lognormal distribution, the logarithm of the lifetimes should follow a normal distribution, and the inference for normal is needed. The lognormal distribution is useful when a hazard rate is increasing at first and then decreasing. We consider the test statistics for normality based on EDF such as the Kolmogorov-Smirnov statistic, the Koziol-Green statistic, and the Anderson-Darling statistic. A more general study about the EDF statistics is given in Stephens (1986), and several topics for testing normality are also studied in Thode (2002).
The Cramer-von Mises statistic and the Anderson-Darling statistic are the quadratic statistics based on EDF. The main difference between two statistics is the weight function in the quadratic form. As it is mentioned, the Koziol-Green statistic is a generalized version of the Cramer-von Mises statistic for randomly censored data. This paper is to generalize the Anderson-Darling statistic to randomly censored data. The Kaplan-Meier estimator is used as in Koziol and Green (1976). The newly defined statistic is applied to test normality with unknown parameters. Kim (2012, 2017) studied the EDF statistic for randomly censored exponential and Weibull distribution, respectively, when some parameters are unknown.
In Section 2, we provide test statistics based on EDF. In Section 3, simulation study and power comparisons are presented. An example is also provided. In Section 4, we mention some concluding remarks.
First, we summarize the goodness-of-fit test statistics based on EDF when data are a complete random sample. Let
with a completely specified distribution function
The EDF statistics for goodness-of-fit tests measure the difference between
A second well known and wide class of statistics are the quadratic statistics. The Cramér-von Mises statistic
and the Anderson-Darling statistic
are the most popular quadratic statistics.
We use the probability integral transformation,
Now let us consider censored data cases. Let
We write
Since the censored data do not have the full knowledge of the EDF, we use the product limit estimator
to estimate
for simplicity. By Michael and Schucany (1986), the
and it reduces to (
As in the complete sample, we still need to use the probability integral transformation to compute the statistics. Hence we define the product limit estimator
as
Using (2.8) and (2.6), the Kolmogorov-Smirnov statistic based on the EDF in (2.2) could be generalized to randomly censored data by
where
with
Koziol and Green (1976) generalized the Cramér-von Mises statistic in (2.3) to
for randomly censored data. It measures the discrepancy between
with
In this paper, let us think about a generalization of the Anderson-Darling statistic to randomly censored data. In this case, the statistic in (2.4) becomes
for
are not integrable on [0, 1], we define
for fixed 0 <
Since
Let us consider the case
When we assume the distribution of
and consider
We consider the following set of remission times for two groups of acute leukemia patients. In this clinical trial, one group of 21 patients received a treatment called 6-mercaptopurine (6-MP), and the other group of 21 patients received a placebo. Each patient was randomized to receive a treatment or a placebo; the study ended after one year. The data set originally comes from Freireich
Treatment | 6, | 6, | 6, | 7, | 10, | 13, | 16, | 22, | 23, | |||
6+, | 9+, | 10+, | 11+, | 17+, | 19+, | 20+, | 25+, | 32+, | 32+, | 34+, | 35+ | |
Placebo | 1, | 1, | 2, | 2, | 3, | 4, | 4, | 5, | 5, | |||
8, | 8, | 8, | 8, | 11, | 11, | 12, | 12, | 15, | 17, | 22, | 23 |
By looking at Figure 1, the straight lines fit quite closely to both groups. It indicates a normal distribution appears to give fairly good fit; however, the way of judging may be subjective. First, let us examine the treatment group. To test that a lognormal distribution fits the data, we need to estimate unknown parameters. The MLEs of the parameters are
They are computed by the S-plus function
The
The value is almost the same when we use the Weibull distribution (Lee and Wang, 2003, Chapter 3).
As for the placebo group, the data are complete. Hence we can compute either
The
A simulation study is conducted to give the null distributions of the test statistics
We use the random censorship model proposed in Koziol and Green (1976) to control the censored ratio. It is
where
under this model. It is the expected proportion of the censored observations. In Table 1 to Table 3,
As we explained in Section 1, the EDF statistics we introduced are distribution free under the simple null hypothesis, since we can use the probability integral transformation. However they depend on the distribution tested when unknown parameters are estimated (Stephens, 1986). If we investigate the null distribution of
In Table 2 and Table 3,
Next, we examine the power of the test statistics. Table 4 and Table 5 provide the power of the statistics at the significance level
exponential distribution with pdf
gamma distribution, Gamma(
Weibull distribution, Weibull(
log-logistic distribution with pdf
log double exponential, log-DE, DE with pdf
half-logistic distribution with
Note that the
In this paper, we have studied goodness-of-fit test statistics for normal distributions with an unknown location and scale parameter when data are randomly censored. In this case the distributional assumption for lifetime itself is lognormal distributions. We take into account test statistics based on EDF statistics such as the Kolmogorov-Smirnov statistic, the Koziol-Green statistic, and the Anderson-Darling statistic. We have generalized the Anderson-Darling statistic to randomly censored data, and found a computational form. We have used the Kaplan-Meier product limit as it was done in Koziol and Green (1976).
Based on the simulation studies, the generalized Anderson-Darling statistic has shown the best power among EDF statistics that we have considered under almost all alternatives. The power results are consistent with complete sample cases in normal distributions as it is mentioned in D’Agostino (1986).
This work was supported by a 2019 Hongik University Research Fund.
Q-Q plots of the log(remission times) for a normal distribution. The left plot is for the treatment group, and the right plot is for the placebo.
Upper tail percentage points of the test statistic
0.01 | 0.025 | 0.05 | 0.10 | 0.15 | 0.25 | 0.5 | ||
---|---|---|---|---|---|---|---|---|
20 | 0.6 | 3.23 | 2.90 | 2.62 | 2.25 | 2.02 | 1.70 | 1.27 |
0.5 | 2.54 | 2.24 | 1.95 | 1.67 | 1.50 | 1.28 | 0.98 | |
0.4 | 1.88 | 1.63 | 1.43 | 1.24 | 1.13 | 1.00 | 0.81 | |
0.2 | 1.13 | 1.04 | 0.96 | 0.88 | 0.83 | 0.76 | 0.64 | |
30 | 0.6 | 3.44 | 3.06 | 2.73 | 2.37 | 2.13 | 1.79 | 1.35 |
0.5 | 2.54 | 2.24 | 1.97 | 1.68 | 1.52 | 1.31 | 1.02 | |
0.4 | 1.83 | 1.62 | 1.43 | 1.24 | 1.15 | 1.02 | 0.83 | |
0.2 | 1.13 | 1.03 | 0.96 | 0.88 | 0.83 | 0.76 | 0.65 | |
40 | 0.6 | 3.52 | 3.13 | 2.82 | 2.44 | 2.19 | 1.88 | 1.41 |
0.5 | 2.59 | 2.24 | 1.97 | 1.71 | 1.54 | 1.34 | 1.05 | |
0.4 | 1.82 | 1.60 | 1.42 | 1.25 | 1.14 | 1.02 | 0.84 | |
0.2 | 1.11 | 1.03 | 0.95 | 0.88 | 0.82 | 0.75 | 0.65 | |
50 | 0.6 | 3.58 | 3.24 | 2.89 | 2.51 | 2.27 | 1.95 | 1.47 |
0.5 | 2.56 | 2.22 | 1.97 | 1.70 | 1.56 | 1.35 | 1.07 | |
0.4 | 1.84 | 1.60 | 1.42 | 1.24 | 1.14 | 1.02 | 0.84 | |
0.2 | 1.11 | 1.03 | 0.95 | 0.87 | 0..83 | 0.76 | 0.65 | |
100 | 0.6 | 3.86 | 3.46 | 3.09 | 2.71 | 2.45 | 2.13 | 1.63 |
0.5 | 2.59 | 2.28 | 2.01 | 1.78 | 1.62 | 1.42 | 1.13 | |
0.4 | 1.75 | 1.51 | 1.37 | 1.22 | 1.14 | 1.03 | 0.86 | |
0.2 | 1.11 | 1.02 | 0.96 | 0.88 | 0.83 | 0.77 | 0.66 |
Upper tail percentage points of the test statistic
0.01 | 0.025 | 0.05 | 0.10 | 0.15 | 0.25 | 0.5 | ||
---|---|---|---|---|---|---|---|---|
20 | 0.6 | 2.52 | 1.85 | 1.37 | 0.90 | 0.68 | 0.46 | 0.25 |
0.5 | 1.25 | 0.88 | 0.61 | 0.42 | 0.34 | 0.25 | 0.15 | |
0.4 | 0.56 | 0.39 | 0.31 | 0.23 | 0.19 | 0.15 | 0.10 | |
0.2 | 0.22 | 0.18 | 0.15 | 0.13 | 0.11 | 0.09 | 0.06 | |
0.0 | 0.17 | 0.14 | 0.12 | 0.10 | 0.09 | 0.07 | 0.05 | |
30 | 0.6 | 2.51 | 1.78 | 1.31 | 0.88 | 0.68 | 0.46 | 0.25 |
0.5 | 1.05 | 0.75 | 0.55 | 0.39 | 0.31 | 0.24 | 0.15 | |
0.4 | 0.45 | 0.35 | 0.28 | 0.22 | 0.19 | 0.15 | 0.10 | |
0.2 | 0.22 | 0.18 | 0.15 | 0.12 | 0.11 | 0.09 | 0.06 | |
40 | 0.6 | 2.32 | 1.68 | 1.24 | 0.85 | 0.66 | 0.47 | 0.26 |
0.5 | 0.99 | 0.67 | 0.51 | 0.37 | 0.31 | 0.23 | 0.15 | |
0.4 | 0.43 | 0.33 | 0.27 | 0.21 | 0.18 | 0.14 | 0.10 | |
0.2 | 0.21 | 0.17 | 0.15 | 0.12 | 0.11 | 0.09 | 0.06 | |
50 | 0.6 | 2.19 | 1.66 | 1.22 | 0.84 | 0.67 | 0.48 | 0.27 |
0.5 | 0.88 | 0.62 | 0.47 | 0.35 | 0.29 | 0.23 | 0.15 | |
0.4 | 0.41 | 0.31 | 0.26 | 0.20 | 0.17 | 0.14 | 0.10 | |
0.2 | 0.21 | 0.17 | 0.15 | 0.12 | 0.11 | 0.09 | 0.06 | |
100 | 0.6 | 2.00 | 1.48 | 1.09 | 0.79 | 0.64 | 0.48 | 0.29 |
0.5 | 0.70 | 0.53 | 0.41 | 0.33 | 0.28 | 0.22 | 0.15 | |
0.4 | 0.33 | 0.27 | 0.23 | 0.18 | 0.16 | 0.13 | 0.09 | |
0.2 | 0.20 | 0.17 | 0.15 | 0.12 | 0.10 | 0.09 | 0.06 | |
0.0 | 0.17 | 0.15 | 0.13 | 0.10 | 0.09 | 0.07 | 0.05 |
Upper tail percentage points of the test statistic
0.01 | 0.025 | 0.05 | 0.10 | 0.15 | 0.25 | 0.5 | ||
---|---|---|---|---|---|---|---|---|
20 | 0.6 | 7.05 | 5.09 | 3.96 | 3.11 | 2.65 | 2.07 | 1.32 |
0.5 | 3.79 | 2.97 | 2.42 | 1.94 | 1.67 | 1.33 | 0.90 | |
0.4 | 2.47 | 1.94 | 1.63 | 1.34 | 1.16 | 0.95 | 0.67 | |
0.2 | 1.40 | 1.14 | 0.97 | 0.82 | 0.73 | 0.61 | 0.44 | |
0.0 | 0.99 | 0.83 | 0.71 | 0.60 | 0.53 | 0.45 | 0.32 | |
30 | 0.6 | 6.75 | 5.42 | 4.34 | 3.33 | 2.84 | 2.27 | 1.48 |
0.5 | 3.90 | 3.15 | 2.55 | 2.01 | 1.75 | 1.41 | 0.95 | |
0.4 | 2.54 | 2.04 | 1.68 | 1.37 | 1.20 | 0.99 | 0.71 | |
0.2 | 1.41 | 1.16 | 0.99 | 0.81 | 0.72 | 0.61 | 0.44 | |
40 | 0.6 | 6.82 | 5.43 | 4.30 | 3.48 | 3.01 | 2.40 | 1.60 |
0.5 | 4.03 | 3.09 | 2.61 | 2.11 | 1.83 | 1.50 | 1.02 | |
0.4 | 2.68 | 2.10 | 1.73 | 1.41 | 1.23 | 1.00 | 0.71 | |
0.2 | 1.36 | 1.13 | 0.97 | 0.81 | 0.71 | 0.60 | 0.43 | |
50 | 0.6 | 7.01 | 5.46 | 4.48 | 3.66 | 3.19 | 2.58 | 1.71 |
0.5 | 3.87 | 3.18 | 2.66 | 2.12 | 1.85 | 1.50 | 1.04 | |
0.4 | 2.68 | 2.09 | 1.75 | 1.40 | 1.23 | 1.02 | 0.73 | |
0.2 | 1.38 | 1.15 | 0.97 | 0.81 | 0.72 | 0.61 | 0.44 | |
100 | 0.6 | 7.78 | 6.22 | 5.20 | 4.32 | 3.76 | 3.05 | 2.09 |
0.5 | 4.64 | 3.61 | 2.95 | 2.36 | 2.07 | 1.70 | 1.18 | |
0.4 | 2.89 | 2.18 | 1.76 | 1.46 | 1.27 | 1.05 | 0.76 | |
0.2 | 1.34 | 1.12 | 0.97 | 0.82 | 0.72 | 0.61 | 0.45 | |
0.0 | 1.00 | 0.86 | 0.74 | 0.62 | 0.55 | 0.46 | 0.33 |
Power comparison of
Distribution | Censoring ratio ( |
|||
---|---|---|---|---|
log-normal | 0.6 | 0.10 | 0.10 | 0.10 |
0.5 | 0.11 | 0.11 | 0.10 | |
0.4 | 0.11 | 0.10 | 0.11 | |
0.2 | 0.11 | 0.10 | 0.10 | |
exponential | 0.6 | 0.15 | 0.16 | 0.37 |
0.5 | 0.25 | 0.30 | 0.44 | |
0.4 | 0.31 | 0.38 | 0.47 | |
0.2 | 0.47 | 0.55 | 0.59 | |
Gamma(0.5) | 0.6 | 0.18 | 0.21 | 0.51 |
0.5 | 0.35 | 0.43 | 0.64 | |
0.4 | 0.50 | 0.60 | 0.71 | |
0.2 | 0.71 | 0.78 | 0.82 | |
Gamma(2) | 0.6 | 0.12 | 0.13 | 0.24 |
0.5 | 0.17 | 0.19 | 0.28 | |
0.4 | 0.20 | 0.25 | 0.30 | |
0.2 | 0.29 | 0.34 | 0.36 | |
Weibull(0.5) | 0.6 | 0.16 | 0.17 | 0.37 |
0.5 | 0.24 | 0.29 | 0.42 | |
0.4 | 0.32 | 0.40 | 0.49 | |
0.2 | 0.45 | 0.54 | 0.59 | |
Weibull(2) | 0.6 | 0.16 | 0.17 | 0.36 |
0.5 | 0.23 | 0.29 | 0.43 | |
0.4 | 0.32 | 0.39 | 0.49 | |
0.2 | 0.47 | 0.55 | 0.59 | |
log-logistic | 0.6 | 0.11 | 0.11 | 0.16 |
0.5 | 0.13 | 0.14 | 0.16 | |
0.4 | 0.13 | 0.15 | 0.17 | |
0.2 | 0.16 | 0.19 | 0.21 | |
log-DE | 0.6 | 0.14 | 0.16 | 0.32 |
0.5 | 0.19 | 0.28 | 0.34 | |
0.4 | 0.23 | 0.37 | 0.37 | |
0.2 | 0.44 | 0.52 | 0.51 | |
half-logistic | 0.6 | 0.17 | 0.20 | 0.47 |
0.5 | 0.30 | 0.38 | 0.55 | |
0.4 | 0.44 | 0.54 | 0.64 | |
0.2 | 0.59 | 0.69 | 0.72 |
Power comparison of
Distribution | Censoring ratio ( |
|||
---|---|---|---|---|
log-normal | 0.6 | 0.10 | 0.10 | 0.10 |
0.5 | 0.10 | 0.09 | 0.10 | |
0.4 | 0.11 | 0.10 | 0.11 | |
0.2 | 0.10 | 0.10 | 0.09 | |
exponential | 0.6 | 0.23 | 0.31 | 0.57 |
0.5 | 0.42 | 0.55 | 0.68 | |
0.4 | 0.62 | 0.71 | 0.78 | |
0.2 | 0.74 | 0.82 | 0.87 | |
Gamma(0.5) | 0.6 | 0.36 | 0.49 | 0.81 |
0.5 | 0.65 | 0.79 | 0.89 | |
0.4 | 0.87 | 0.92 | 0.95 | |
0.2 | 0.95 | 0.97 | 0.99 | |
Gamma(2) | 0.6 | 0.16 | 0.20 | 0.37 |
0.5 | 0.24 | 0.33 | 0.43 | |
0.4 | 0.38 | 0.46 | 0.51 | |
0.2 | 0.45 | 0.55 | 0.61 | |
Weibull(0.5) | 0.6 | 0.24 | 0.32 | 0.59 |
0.5 | 0.42 | 0.56 | 0.69 | |
0.4 | 0.61 | 0.70 | 0.76 | |
0.2 | 0.73 | 0.81 | 0.87 | |
Weibull(2) | 0.6 | 0.23 | 0.31 | 0.60 |
0.5 | 0.42 | 0.55 | 0.69 | |
0.4 | 0.61 | 0.71 | 0.76 | |
0.2 | 0.73 | 0.82 | 0.87 | |
log-logistic | 0.6 | 0.12 | 0.14 | 0.19 |
0.5 | 0.12 | 0.16 | 0.18 | |
0.4 | 0.15 | 0.20 | 0.20 | |
0.2 | 0.20 | 0.24 | 0.27 | |
log-DE | 0.6 | 0.18 | 0.31 | 0.45 |
0.5 | 0.25 | 0.50 | 0.48 | |
0.4 | 0.41 | 0.65 | 0.58 | |
0.2 | 0.69 | 0.77 | 0.76 | |
half-logistic | 0.6 | 0.31 | 0.43 | 0.73 |
0.5 | 0.56 | 0.72 | 0.84 | |
0.4 | 0.78 | 0.86 | 0.90 | |
0.2 | 0.86 | 0.92 | 0.95 |