It is obvious that one of the most important objectives in building a time series model is to forecast its future values as accurate as the model can. The better forecasting performance of the model would be the property for the better model to possess. For instance, the accurate incidence forecasting of infectious disease is critical for early prevention and for better strategy development in public health surveillance field (Cardinal
Disease incidence time series can be handled using ad hoc integer-valued time series models (instead of ARMA) since the non-negative integer-valued forecasts as well as forecast limits are obtained. The integer-valued time series models are rigorously studied and applied over past three decades. Various integer-valued models with conditional Poisson distribution (Cardinal
Due to non-linearity in most integer-valued time series models, there exist no closed-form formulas to compute multi-step ahead forecasts of nonlinear models (Tsay, 2010). Both Monte Carlo simulation and bootstrap methods are popular to compute nonlinear forecasts (Kock and Teräsvirta, 2010). Taieb
In this paper we study a parametric bootstrap method for forecasting integer-valued-threshold autoregressive conditional heterosckedastic (INTARCH) model investigated by Kim
In this section, model specifications are mainly based on Kim
With replacing (2.1) by a negative binomial distribution in order to accommodate over-dispersion structure in the data, the first-order integer-valued negative binomial ARCH (NB-INARCH(1)) model (Zhu, 2011; Yoon and Hwang, 2015) is defined as
The NB-INARCH(1) can be extended to the following threshold-asymmetric model as a generalization of (2.3).
An one-step ahead forecast
There are many ways to evaluate the forecasting performance of a time series model. The methods of forecasting evaluation described in this section are mostly based on Section 4.3 of Tsay (2010). Directional measures and magnitude measures are discussed. A directional measure considers the future direction (up or down) implied by the model. For instance, whether next week’s cholera cases will go up and down, or keep staying is an example of directional forecasts that are of practical interest. Magnitude measures are a function of the actual and predicted values of the time series and is used to assess accuracy of the predicted values, and relative ratio of magnitude measures between models is used for the model comparison.
Tsay (2010) described the directional measure as a performance evaluation measure. By considering many zeros in the integer-valued time series as well as no directional changes due to count time series data, a 3×3 contingency table is newly suggested which summarizes the number of “hits” or “misses” of the model in predicting upwards, downwards, and no movements of
Actual | Predicted | |||
---|---|---|---|---|
up (> 0) | stay (= 0) | down (< 0) | Total | |
UP (> 0) | ||||
STAY (= 0) | ||||
DOWN (< 0) | ||||
Total |
where
Four statistics are used to measure the performance of point forecasts from each model. They are the mean squared error (MSE), mean absolute error (MAE), mean absolute scaled error (MASE), and relative mean absolute error (relMAE). The first two measures (MSE and MAE) depending on the scale of data are commonly used accuracy measures when comparing different methods applied to the same dataset, while MASE proposed by Hynman and Koehler (2006) is useful when comparing across data that have a different scale by scaling mean absolute difference between consecutive actual observations. The MASE is also alternatively useful when absolute percentage error is not computed due to division by zero observation in the data. The relMAE (Hynman and Koehler, 2006) is the ratio of the MAE between the comparator and referenced models. For one-step ahead forecasts, these measures are defined as
It will be instructive to consider the case in public health surveillance. As vaccination is one of most cost-effective intervention that contributes to healthcare system (Andre
Actual | Predicted | ||
---|---|---|---|
up | stay | down | |
UP | 0 | C(stay|UP) | C(down|UP) |
STAY | C(up|STAY) | 0 | C(down|STAY) |
DOWN | C(up|DOWN) | C(stay|DOWN) | 0 |
Actual | Predicted | |||
---|---|---|---|---|
up | stay | down | ||
UP | P(up|UP) | P(stay|UP) | P(down|UP) | P(UP) |
STAY | P(up|STAY) | P(stay|STAY) | P(down|STAY) | P(STAY) |
DOWN | P(up|DOWN) | P(stay|DOWN) | P(down|DOWN) | P(DOWN) |
In this section, as a real data application, we illustrate INTARCH and NB-INTARCH models which are defined in Section 2, and evaluate forecasting performance by calculating diverse measures de scribed in Section 3. We consider time series of weekly cholera cases from Matlab in Bangladesh, consisting of 1,513 observations starting from 1st-week of 1988 to 52nd-week of 2016. The data prior to 1988 were excluded because there was an oral cholera vaccine trial in early 1985 that offered direct and indirect protection to people of the area (Ali
The sample autocorrelation and partial autocorrelation function (not listed here) of the series imply that the first-order integer-valued autoregressive model is a reasonable candidate model for the data.
In earlier research, Ali
For the threshold models, two threshold variables (
In the tables and figures to follow, the grand mean and the local mean are designated respectively by the notation * and +.
Diverse measures described in Section 3 are summarized in Tables 1 to 4 based on 300 one-step ahead forecasts in forecasting subsample. As shown in Tables 1 and 2, the models with negative binomial distribution predicted more number of “hits” in predicting upwards, while the models with Poisson distribution predicted more numbers of “hits” in predicting no movements (stay). All the models predicted nearly the same number of “hits” in predicting downwards. Due to better predicting ability in no movements, the models with Poisson distribution had highest number of “hits” in predicting direction, but negative binomial model showed higher predicting ability if we consider higher importance of “hits” in predicting upwards over the others by giving weights, for instance, 0.5, 0.25, and 0.25 on “hits” in predicting upwards, staying, and downwards, respectively. Table 3 displays performance measures of point forecast. The models with negative binomial distribution had lower MSE, MAE, and MASE over the Poisson models. The lowest MSE was calculated in NB1-INTARCH with the local mean threshold defined in (4.1), which gives also the lowest MAE and MASE. Among the models with negative binomial distribution, the NB1-INTARCH model with local mean threshold showed slightly higher accuracy than the others. The relMAE showed negative binomial models showed better performances (relMAE
In this paper, we have discussed the parametric bootstrap method of forecasting threshold INARCH models with conditional Poisson and negative binomial distributions. Overall, the negative binomial threshold ARCH (NB-INTARCH) model with local threshold variable showed relatively better performances compared to the others. Since the threshold models have certain advantages in integer valued time series, it will be of interest to make applications to time series of infectious disease incidence in public health surveillance. Further, in order to accommodate seasonal circulation, sanitation improvement, vaccination, and climate changes, intervention analysis in the context of INTARCH would make an interesting future study. This is now under investigation and will be addressed elsewhere.
We thank the two reviewers for constructive comments which led to a substantial improvement in the revised version. The real data analysis in this paper is based on data collected and shared by the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b) from an original study which was conducted with support from the governments of Bangladesh, Canada, Sweden and UK. SY Hwang’s work was supported by a grant from the National Research Foundation of Korea (NRF-2018R1A2B2004157).
Weekly cholera cases series and histogram: a) entire period and 2nd-subsample for one-step ahead forecasts and evaluation (filled area, time = 1214 to 1513), b) histogram of cholera cases.
Actual observations and one-step ahead forecasts (sample median as predicted value): a) INARCH(1), b) INTARCH(1) with grand mean threshold, c) INTARCH(1) with local mean threshold, d) NB1-INARCH(1), e) NB1-INTARCH(1) with grand mean threshold, f) NB1-INTARCH(1) with local mean threshold, g) NB2-INARCH(1), h) NB2-INTARCH(1) with grand mean threshold, i) NB2-INTARCH(1) with local mean threshold. Actual series and forecasts in dashed-black color and red color, respectively.
Contingency table of directional changes
Actual | Predicted | Total | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
INARCH | INTARCH* | INTARCH+ | ||||||||
up | stay | down | up | stay | down | up | stay | down | ||
UP | 9 | 57 | 16 | 9 | 54 | 19 | 13 | 51 | 18 | 82 |
STAY | 6 | 113 | 20 | 6 | 113 | 20 | 8 | 102 | 29 | 139 |
DOWN | 41 | 23 | 15 | 42 | 23 | 14 | 44 | 21 | 14 | 79 |
Total | 56 | 193 | 51 | 57 | 190 | 53 | 65 | 174 | 61 | 300 |
Actual | Predicted | Total | ||||||||
NB1-INARCH | NB1-INTARCH* | NB1-INTARCH+ | ||||||||
up | stay | down | up | stay | down | up | stay | down | ||
UP | 18 | 35 | 29 | 18 | 35 | 29 | 18 | 36 | 28 | 82 |
STAY | 7 | 96 | 36 | 7 | 96 | 36 | 7 | 96 | 36 | 139 |
DOWN | 56 | 9 | 14 | 56 | 9 | 14 | 56 | 9 | 14 | 79 |
Total | 81 | 140 | 79 | 81 | 140 | 79 | 81 | 141 | 78 | 300 |
Actual | Predicted | Total | ||||||||
NB2-INARCH | NB2-INTARCH* | NB2-INTARCH+ | ||||||||
up | stay | down | up | stay | down | up | stay | down | ||
UP | 16 | 40 | 26 | 17 | 41 | 24 | 14 | 44 | 24 | 82 |
STAY | 11 | 102 | 26 | 12 | 99 | 28 | 9 | 102 | 28 | 139 |
DOWN | 41 | 24 | 14 | 41 | 24 | 14 | 44 | 20 | 15 | 79 |
Total | 68 | 166 | 66 | 70 | 164 | 66 | 67 | 166 | 67 | 300 |
Percentage of hits
Measures | Direction of predicted values | ||||||||
---|---|---|---|---|---|---|---|---|---|
INARCH | INTARCH* | INTARCH+ | |||||||
up | stay | down | up | stay | down | up | stay | down | |
Percent of “hits” in each direction | 11% | 81% | 19% | 11% | 81% | 18% | 16% | 73% | 18% |
Average percent of “hits” | 46% | 45% | 43% | ||||||
Weighted average percent of “hits” | 27% | 27% | 29% | ||||||
Measures | Direction of predicted values | ||||||||
NB1-INARCH | NB1-INTARCH* | NB1-INTARCH+ | |||||||
up | stay | down | up | stay | down | up | stay | down | |
Percent of “hits” in each direction | 22% | 69% | 22% | 22% | 69% | 22% | 22% | 69% | 22% |
Average percent of “hits” | 43% | 43% | 43% | ||||||
Weighted average percent of “hits” | 30% | 30% | 30% | ||||||
Measures | Direction of predicted values | ||||||||
NB2-INARCH | NB2-INTARCH* | NB2-INTARCH+ | |||||||
up | stay | down | up | stay | down | up | stay | down | |
Percent of “hits” in each direction | 20% | 73% | 22% | 21% | 71% | 22% | 17% | 73% | 23% |
Average percent of “hits” | 44% | 43% | 44% | ||||||
Weighted average percent of “hits” | 29% | 30% | 29% |
The weighted average percent of “hits” is calculated with weights 0.5, 0.25, and 0.25 to each “hits” of upwards, stay, and downwards direction.
Magnitude measures
Measures | Models | ||
---|---|---|---|
INARCH | INTARCH* | INTARCH+ | |
Mean squared error (MSE) | 3.36 | 3.40 | 3.25 |
Mean absolute error (MAE) | 1.26 | 1.28 | 1.27 |
Mean absolute scaled error (MASE) | 1.23 | 1.25 | 1.24 |
Relative MAE against Poisson model ^{1)} | Ref | Ref | Ref |
Relative MAE against no threshold model ^{2)} | Ref | 1.01 | 0.99 |
Measures | Models | ||
NB1-INARCH | NB1-INTARCH* | NB1-INTARCH+ | |
Mean squared error (MSE) | 3.05 | 3.01 | 2.93 |
Mean absolute error (MAE) | 0.99 | 0.98 | 0.96 |
Mean absolute scaled error (MASE) | 0.97 | 0.96 | 0.94 |
Relative MAE against Poisson model ^{1)} | 0.78 | 0.77 | 0.76 |
Relative MAE against no threshold model ^{2)} | Ref | 0.99 | 0.98 |
Measures | Models | ||
NB2-INARCH | NB2-INTARCH* | NB2-INTARCH+ | |
Mean squared error (MSE) | 2.96 | 2.96 | 2.95 |
Mean absolute error (MAE) | 1.11 | 1.10 | 1.06 |
Mean absolute scaled error (MASE) | 1.08 | 1.07 | 1.03 |
Relative MAE against Poisson model ^{1)} | 0.88 | 0.86 | 0.83 |
Relative MAE against no threshold model ^{2)} | Ref | 0.99 | 0.96 |
^{1)}Relative MAE of each NB model against referenced Poisson model;
^{2)}Relative MAE of each threshold model against referenced threshold model. Ref stands for the referenced model.
Expected cost of misclassification
Measures | Models | ||||
---|---|---|---|---|---|
INARC H | INTAR CH* | INTAR CH+ | NB1-INARC H | NB1-INTAR CH* | |
TPM | 0.5433 | 0.5467 | 0.5700 | 0.5722 | 0.5733 |
ECM | 3.3467 | 3.3600 | 3.3267 | 3.2733 | 3.2733 |
Measures | Models | ||||
NB1-INTAR CH+ | NB2-INARC H | NB2-INTAR CH* | NB2-INTAR CH+ | ||
TPM | 0.5733 | 0.5600 | 0.5667 | 0.5633 | |
ECM | 3.2733 | 3.2267 | 3.2200 | 3.2933 |