In prognostic studies, it happens a substantial portion of patients can be event-free, which is denoted as a cured group or a risk-free group. For evaluating the effect of covariates both on the cure rate and on the failure time of susceptible (uncured) patients, several models have been proposed. Among them, the mixture model is expressed of a logistic model for the cure rate and a proportional hazard regression model for susceptible patients (Kuk and Chen, 1992; Maller and Zhou, 1996). In the context of a cure rate model, two issues related with a predictive accuracy have been considered. The first one is to predict who is cured and the second is to predict the survival probabilities of uncured subjects based on the markers. Both issues can be dealt by extending the classical discriminative accuracy measures such as the ROC curve and C-index.
The ROC curve has been the most frequently applied measure by providing both a graph and an AUC value. There are two probabilities to construct the curve; a sensitivity is defined as the probability of having a higher marker value among a case group (true positive rate; TPR) and a specificity is defined as the probability of having a lower marker value among a control one (true negative rate; TNR), respectively. These probabilities have been changed according to the threshold value of a marker and are displayed as the ROC curve where plots sensitivity against one minus specificity over all possible thresholds. The predictive performance of a marker can be evaluated with the AUC (area under curve) where a higher AUC value indicates a better performance.
For survival data, the time to event as the response variable has been observed during follow-up and changed over time which results in a time-dependent AUC denoted by AUC(
the corresponding ROC curve is given by ROC
which is interpreted as the probability that for two randomly chosen subjects, one experiencing the event prior to
For a cure rate model, most estimators of the AUC have been focused on the cure probability of a prediction model. Asano
In this study, our interest is to suggest the time-dependent AUC estimator based on the inverse probability censoring weight (IPCW) technique, when a censoring is related with a covariate. The rest of this article is organized as follows. In Section 2, we introduce the notations and propose three types of time-dependent AUCs. In Section 3, the finite sample performance of suggested methods is evaluated through simulation studies. Application of the suggested methods to a melanoma dataset is presented in Section 4 and several discussions are given in Section 5.
In the context of survival data, the time to event is not always observed due to the censoring related with diverse observation schemes. Furthermore, in the presence of nonsusceptible (cured) patients, the time to event is denoted as
where
For modelling the susceptible rate of a subject
Under a PH model assumption, the conditional hazard model of a susceptible group is written as
then
For evaluating the prediction model of a cure rate data, a risk score
Beyene
is estimated
where the survival function
For a same problem, Wang and Wang (2020) directly implemented the estimated survival function as follows,
where they applied a smoothing technique to obtain AUC
In general survival data, a right censoring causes a biased sampling when a censoring distribution is related with a certain subpopulation which is sometimes modelled with a vector of covariates. Inverse probability of censoring weighting (IPCW) technique has been originally proposed to adjust for dependent censoring (Robins, 1993; Robins and Finkelstein, 2000). Under a competing risk data, it has been adopted to reflect the effect of the subpopulation with competing event (Fine and Gray, 1999) and also applied to the discriminative measures such as C-index (Uno
In this paper, we propose a class of IPCW estimators of time-dependent AUC(
The first estimator is to incorporate IPCW into Beyene’s method (
Blanche
For the control group of a dynamic specificity in (1.2), two versions are presented. The first version of control group, the event-free subjects at
Then the corresponding time-dependent AUC(
For the variance estimation, the bootstrap samples are generated and the confidence intervals are obtained from their standard deviations.
In this section, the performance of the suggested estimators is evaluated with three situations; (i) light censoring; 35% (cure-rate: 15%), (ii) medium censoring; 55% (cure-rate: 30%) and (iii) heavy censoring; 70%(cure-rate: 50%), respectively. The difference of these censoring rates is inclined to the amount of cure rates. For reflecting the effect of a covariate on cure rate, a failure time and a censoring distribution, a covriate
To compare the performance of several IPCW estimators of AUC(
300 datasets are generated with two sample sizes
Table 1 shows the biases(standard deviations) of suggested methods when a censoring distribution is independent of covariate. All estimates have similar results and seem to be unbiased. However, AUC
Table 3 shows the coverage probabilities (CP) and the standard errors obtained using 50 bootstrap samples at
We analyzed a malignant melanoma dataset which is available in the R package MASS. The dataset consists of 205 patients whose tumors were completely removed together with the skin within a distance of about 2.5cm around it at the operation. The study started in the period 1962–1977 and all patients have been followed for checking disease progression and survival until 1977. Among 205 patients, only 57 patients died of melanoma, 14 one died from other causes and the remaining were alive. In this study, the death from other causes is regarded as a censoring (censoring rate 72%). The time scale is days since operation and four covariates
Table 5 shows the nine AUC(
In this paper, we applied the IPCW approach to estimate time-dependent AUC for cure rate models when a censoring distribution is related with covariates. Simulation results show that the proposed procedures work well for covariate dependent censoring and a large censoring rate. However, Uno’s method AUC
At melanoma data analysis, nine AUC(t) values are similar at the 1 and 4 year and the suggested ones have smaller values than non-IPCW AUC as time increases. In particular, covariate-dependent versions have large variations which bring the wider confidence intervals.
As another discriminative measure, a concordance index or C-index is defined as the proportion of concordant pairs where a patient with an early event time is likely to have a higher marker. Asano and Hirakawa (2017) proposed the C-index reflecting the patients’ cure status estimated the cure rate model. A time-dependent C-index
As another interesting topic, dynamic prediction models have been studied with joint model and landmark approach when a cure rate model includes longitudinal covariates (Rizopulos
Bias(sd) of AUC(
(Cure, Cen) | AUC | AUC | AUC | AUC | AUC | AUC | ||
---|---|---|---|---|---|---|---|---|
(0.15, 0.35) | 200 | 0.0001 | 0.0001 | 0.0001 | 0.0001 | 0.020 | 0.0001 | |
(0.053) | (0.052) | (0.028) | (0.052) | (0.057) | (0.053) | |||
0.001 | 0.001 | 0.003 | 0.002 | 0.035 | 0.001 | |||
(0.041) | (0.040) | (0.029) | (0.040) | (0.046) | (0.041) | |||
400 | 0.0001 | 0.0001 | 0.0002 | 0.0001 | 0.031 | 0.0001 | ||
(0.037) | (0.036) | (0.022) | (0.036) | (0.040) | (0.037) | |||
0.002 | 0.001 | 0.001 | 0.001 | 0.032 | 0.002 | |||
(0.028) | (0.028) | (0.022) | (0.027) | (0.032) | (0.028) | |||
(0.30, 0.55) | 200 | 0.001 | 0.001 | 0.0031 | 0.001 | 0.0581 | 0.0001 | |
(0.050) | (0.052) | (0.037) | (0.052) | (0.059) | (0.053) | |||
0.001 | 0.001 | 0.003 | 0.002 | 0.071 | 0.000 | |||
(0.040) | (0.043) | (0.038) | (0.043) | (0.047) | (0.045) | |||
400 | 0.001 | 0.0001 | 0.001 | 0.0001 | 0.054 | 0.001 | ||
(0.038) | (0.037) | (0.022) | (0.037) | (0.044) | (0.038) | |||
0.002 | 0.0001 | 0.0001 | 0.0001 | 0.0617 | 0.001 | |||
(0.032) | (0.032) | (0.021) | (0.032) | (0.040) | (0.029) | |||
(0.50, 0.70) | 200 | 0.008 | 0.011 | 0.017 | 0.011 | 0.112 | 0.010 | |
(0.049) | (0.057) | (0.048) | (0.058) | (0.062) | (0.058) | |||
0.005 | 0.003 | 0.000 | 0.000 | 0.097 | 0.002 | |||
(0.038) | (0.055) | (0.052) | (0.055) | (0.049) | (0.056) | |||
400 | 0.012 | 0.0011 | 0.0008 | 0.0006 | 0.101 | 0.0010 | ||
(0.038) | (0.032) | (0.024) | (0.031) | (0.041) | (0.033) | |||
0.003 | 0.003 | 0.005 | 0.005 | 0.104 | 0.003 | |||
(0.029) | (0.027) | (0.024) | (0.027) | (0.034) | (0.028) |
cure: cure rate; cp: censoring rate;
Bias(sd) of AUC(
(Cure, cen) | AUC | AUC | AUC | AUC | AUC | AUC | ||
---|---|---|---|---|---|---|---|---|
(0.15, 0.35) | 200 | 0.020 | 0.012 | 0.010 | 0.012 | 0.013 | 0.013 | |
(0.049) | (0.047) | (0.031) | (0.048) | (0.055) | (0.050) | |||
0.037 | 0.020 | 0.014 | 0.016 | 0.004 | 0.001 | |||
(0.042) | (0.041) | (0.032) | (0.043) | (0.049) | (0.045) | |||
400 | 0.021 | 0.032 | 0.012 | 0.012 | 0.014 | 0.001 | ||
(0.038) | (0.037) | (0.022) | (0.039) | (0.042) | (0.039) | |||
0.036 | 0.020 | 0.019 | 0.016 | 0.005 | 0.005 | |||
(0.029) | (0.029) | (0.023) | (0.04) | (0.041) | (0.040) | |||
(0.30, 0.55) | 200 | 0.034 | 0.019 | 0.013 | 0.017 | 0.028 | 0.001 | |
(0.052) | (0.050) | (0.037) | (0.052) | (0.059) | (0.051) | |||
0.056 | 0.028 | 0.025 | 0.020 | 0.008 | 0.008 | |||
(0.042) | (0.042) | (0.037) | (0.050) | (0.056) | (0.053) | |||
400 | 0.033 | 0.018 | 0.016 | 0.015 | 0.032 | 0.003 | ||
(0.037) | (0.035) | (0.024) | (0.038) | (0.043) | (0.039) | |||
0.057 | 0.031 | 0.030 | 0.019 | 0.013 | 0.000 | |||
(0.031) | (0.029) | (0.024) | (0.053) | (0.061) | (0.060) | |||
(0.50, 0.70) | 200 | 0.039 | 0.017 | 0.012 | 0.015 | 0.056 | 0.005 | |
(0.053) | (0.062) | (0.054) | (0.065) | (0.070) | (0.067) | |||
0.070 | 0.030 | 0.027 | 0.012 | 0.030 | 0.002 | |||
(0.041) | (0.058) | (0.057) | (0.075) | (0.082) | (0.077) | |||
400 | 0.036 | 0.018 | 0.013 | 0.018 | 0.061 | 0.003 | ||
(0.034) | (0.035) | (0.028) | (0.036) | (0.044) | (0.036) | |||
0.067 | 0.032 | 0.030 | 0.012 | 0.038 | 0.002 | |||
(0.029) | (0.028) | (0.020) | (0.057) | (0.070) | (0.061) |
Cure: cure rate; cen: censoring rate;
Coverage probability of AUC(
Method | Est | SD | SE | CP | Est | SD | SE | CP |
---|---|---|---|---|---|---|---|---|
AUC | 0.706 | 0.051 | 0.050 | 0.929 | 0.722 | 0.042 | 0.040 | 0.948 |
AUC | 0.707 | 0.054 | 0.051 | 0.935 | 0.723 | 0.048 | 0.043 | 0.967 |
AUC | 0.702 | 0.040 | 0.031 | 0.962 | 0.722 | 0.041 | 0.031 | 0.967 |
0.706 | 0.054 | 0.047 | 0.935 | 0.721 | 0.047 | 0.036 | 0.967 | |
0.643 | 0.059 | 0.060 | 0.801 | 0.659 | 0.049 | 0.047 | 0.775 | |
0.701 | 0.056 | 0.052 | 0.917 | 0.719 | 0.049 | 0.041 | 0.961 | |
Method | Est | SD | SE | CP | Est | SD | SE | CP |
AUC | 0.720 | 0.055 | 0.051 | 0.890 | 0.75 | 0.044 | 0.042 | 0.680 |
AUC | 0.708 | 0.056 | 0.057 | 0.910 | 0.727 | 0.056 | 0.051 | 0.830 |
AUC | 0.705 | 0.049 | 0.045 | 0.860 | 0.723 | 0.052 | 0.047 | 0.830 |
AUC | 0.707 | 0.056 | 0.060 | 0.900 | 0.718 | 0.062 | 0.064 | 0.900 |
AUC | 0.660 | 0.057 | 0.065 | 0.940 | 0.692 | 0.059 | 0.068 | 0.970 |
AUC | 0.689 | 0.056 | 0.061 | 0.930 | 0.693 | 0.058 | 0.064 | 0.930 |
Summary of regression models of Melanoma dataset.
Cure model | Censoring distribution | |||||
---|---|---|---|---|---|---|
Cov | Susceptible rate | Latency distribution | Cox PH | |||
Est(se) | Est(se) | Est(se) | ||||
Intercept | −2.33(0.929) | 0.012 | ||||
Sex | 0.288(0.582) | 0.621 | 0.569(.532) | 0.284 | −0.041(0.178) | 0.818 |
Log (thick) | 0.042(0.095) | 0.659 | 0.874(0.287) | 0.002 | −0.206(0.094) | 0.028 |
Ulcer | 1.323(0.641) | 0.039 | 0.086(0.536) | 0.872 | 0.196(0.197) | 0.321 |
Age | 0.019(0.017) | 0.253 | −0.008(0.013) | 0.557 | 0.026(0.006) | <0.0001 |
Estimation of AUC values and 95% CI at
Method | ||||||
---|---|---|---|---|---|---|
Est(se) | 95% CI | Est(se) | 95% CI | Est(se) | 95% CI | |
AUC | 0.904 | (0.814,0.994) | 0.824 | (0.746,0.902) | 0.772 | (0.628,0.816) |
AUC | 0.868 | (0.768,0.968) | 0.812 | (0.706,0.918) | 0.737 | (0.643,0.831) |
AUC | 0.789 | (0.685,0.893) | 0.780 | (0.610,0.806) | 0.731 | (0.619,0.848) |
covariate-independent censoring: | ||||||
0.887 | (0.785,989) | 0.812 | (0.710,0.914) | 0.725 | (0.615,0.835) | |
0.839 | (0.729,0.949) | 0.755 | (0.669,0.841) | 0.641 | (0.527,0.755) | |
0.889 | (0.787,0.991) | 0.810 | (0.706,0.914) | 0.738 | (0.634,0.842) | |
covariate-dependent censoring: | ||||||
AUC | 0.887 | (0.773,1.000) | 0.812 | (0.675,0.949) | 0.672 | (0.428,0.915) |
AUC | 0.838 | (0.715,0.971) | 0.755 | (0.669,0.841) | 0.583 | (0.338,0.828) |
AUC | 0.888 | (0.774,1.000) | 0.809 | (0.674,0.944) | 0.701 | (0.485,0.917) |