^{a}Department of Statistics, Daejeon University, Korea; ^{b}Department of Statistics, Kyungpook National University, Korea
Correspondence to:^{1}Department of Statistics, Kyungpook National University, 80 Daehakro, Bukgu, Daegu 41566, Korea. E-mail: kim.1252@knu.ac.kr
Received April 6, 2018; Revised May 15, 2018; Accepted May 15, 2018.
Abstract
The Bayesian approach is a suitable alternative in constructing appropriate models for observed record values because the number of these values is small. This paper provides an objective Bayesian analysis method for upper record values arising from the Rayleigh distribution. For the objective Bayesian analysis, the Fisher information matrix for unknown parameters is derived in terms of the second derivative of the log-likelihood function by using Leibniz’s rule; subsequently, objective priors are provided, resulting in proper posterior distributions. We examine if these priors are the PMPs. In a simulation study, inference results under the provided priors are compared through Monte Carlo simulations. Through real data analysis, we reveal a limitation of the appropriate confidence interval based on the maximum likelihood estimator for the scale parameter and evaluate the models under the provided priors.
Observations of survival times of objects, precipitation levels, Olympic records, or daily stock prices greater than the existing respective records, are called the upper record values. This concept was introduced by Chandler (1952). Let {X_{1}, …, X_{n}} be a sequence of independent and identically distributed random variables with the cumulative distribution function (CDF) and probability density function (PDF). Then, we can say that x_{j} is an upper record value if x_{j} > x_{i} for every i < j, and the record time sequence {U(k), k ∈ ℕ} is denoted as
The statistical inference based on the record values has limitations due to small sample sizes, although the modelling for small samples is an important issue in statistical application. In addition, the likelihood function for unknown parameters and the predictive likelihood function respectively provided by Arnold et al. (1998) and Basak and Balakrishnan (2003) can yield inappropriate inference results for small sample sizes, as they lead to the likelihood equation in the maximum likelihood method. To overcome these limitations, Wang et al. (2015) proposed a new inference method dependent on pivotal quantities in the family of proportional reversed hazard distributions based on the record values. Wang and Ye (2015) provided bias-corrected estimators and exact confidence intervals (CIs) for unknown parameters of the Weibull distribution based on the upper record values. The Bayesian approach can be a useful alternative for small sample sizes if one has sufficient prior information. Jaheen (2003) developed a Bayesian inference under a subjective prior for unknown parameters of the Gompertz distribution based on upper record values. Madi and Raqab (2004) provided a subjective Bayesian inference to predict the future upper record values based on the observed upper record values from the Pareto distribution.
However, subjective Bayesian approaches cannot be properly used in situations in which little or no prior information is available. In this case, the Bayesian inference can rely on the noninformative or objective priors. The most widely used noninformative priors are the Jeffreys prior (Jeffreys, 1961) and the reference prior (Bernardo, 1979; Berger and Bernardo, 1989, 1992). In addition, the probability matching prior (PMP) introduced by Welch and Peers (1963) has gained recent popularity due to its frequentist properties.
This article provides an objective Bayesian approach based on noninformative priors to estimate the unknown parameters of the two-parameter Rayleigh distribution with the CDF
where μ is the location parameter and σ is the scale parameter. The Rayleigh distribution was first considered by Rayleigh (1880) as the distribution of the amplitude resulting from the addition of harmonic oscillations. This distribution has since been applied in many fields such as communication engineering and electro vacuum devices (Polovko, 1968; Dyer and Whisenand, 1973). Another important characteristic of this distribution is its failure rate function is an increasing linear function of time. Therefore, some authors employed this distribution to construct a statistical model fitting real data. Raqab and Madi (2002) discussed the predictive distribution of the total testing time up to a certain failure in a future sample, as well as the remaining testing time until all the items in the original sample have failed when doubly censored data are observed. Wu et al. (2006) derived the Bayes estimator of the scale parameter and the Bayes predictors of future observations when progressively Type-II censored data are observed. Kim and Han (2009) derived the Bayes estimator of the scale parameter and the reliability function based on multiply Type-II censored data. Lee et al. (2011) constructed a Bayes estimator of the lifetime performance of products and proposed a Bayesian test to assess this performance when progressively Type-II censored data are observed. Soliman and Al-Aboud (2008) provided a subjective Bayesian inference method for the scale parameter and the reliability and failure rate functions based on the record values. Seo and Kim (2017) provided a noninformative prior with partial information to estimate unknown parameters and predict the future upper record values.
This article focuses on inference based on the objective priors to avoid the risk from inappropriate prior information and reduce the effort in obtaining sufficient prior information. To develop the method based on the objective priors, it needs to obtain a closed form of the Fisher information matrix for unknown parameters (μ,σ), as the popular objective priors such as the Jeffreys and reference priors; in addition, the PMPs are obtained based on the Fisher information matrix. We provide the Fisher information matrix for (μ,σ) in terms of the second derivative of the log-likelihood using Leibniz’s rule and develop an objective Bayesian analysis method.
The rest of this paper is organized as follows. Section 2 provides the Fisher information matrix in terms of the second derivative of the log-likelihood and preferred objective priors (the Jeffreys and reference priors and the second-order PMP) for unknown parameters (μ,σ). In the following section we assess an objective Bayesian approach based on the provided priors. Section 4 assesses the proposed objective Bayesian analysis method through the Monte Carlo simulations and applies the method on a set of survival data for lung cancer patients. Section 5 concludes this article.
2. Objective priors
This section provides the Fisher information matrix for unknown parameters (μ,σ) of the Rayleigh distribution based on the upper record values for deriving the objective priors and then proposes objective Bayesian models under the derived priors.
Let X_{U}_{(}_{i}_{)} be the i^{th} upper record value from a PDF with an unknown parameter θ. Then, the Fisher information for θ is given by
is the marginal density function of X_{U}_{(}_{i}_{)} provided in Ahsanullah (1995). Under certain regularity conditions, the Fisher information (2.1) is given by
which has computational convenience compared with (2.1). To employ the Fisher information (2.3), the interchangeability of differentiation and integration operators for θ is a necessary condition. In some cases, the operations of differentiation and integration do not hold provided that the support of a probability distribution depends on an unknown parameter, for example, in the Laplace distribution (Burkschat and Cramer, 2012) with the location parameter and uniform distribution on (0, θ) for some θ > 0 (Romano and Siegel, 1986). The following results represent that integration and differential operators are interchangeable; however, the support of the two-parameter Rayleigh distribution depends on the location parameter μ.
Proposition 1
Let X_{U}_{(}_{i}_{)}be the i^{th} upper record value from a two-parameter Rayleigh distribution. Then,
where F_{XU(i)}(·) is the CDF of X_{U}_{(}_{i}_{)}. Therefore, the relationship in (2.4) holds. The relationship in (2.5) can be proved in the same way.
Let X_{U}_{(}_{i}_{)}, …, X_{U}_{(}_{k}_{)} be the upper record values from the Rayleigh distribution with the PDF (2.2). Then, the likelihood function based on X_{U}_{(}_{i}_{)}, …, X_{U}_{(}_{k}_{)} is given by
Then, the first term is zero by Remark 2 and the second term is also zero because − log (1 − F(x_{U}_{(}_{i}_{)}) has the standard exponential distribution. Therefore
where ψ(·) is the digamma function and C is Euler’s constant. Based on the Fisher information matrix (2.16) and the asymptotic normality of the maximum likelihood estimator (MLE), we can obtain the approximate 100(1 − α)% CIs based on the MLEs μ̂ and σ̂, which maximize the likelihood function (2.10) for μ and σ as
where Z_{α}_{/2} denotes the upper α/2 point of the standard normal distribution and Var (μ̂) and Var (σ̂) are the diagonal elements of the asymptotic variance-covariance matrix of the MLEs obtained by inverting the Fisher information matrix (2.16). The approximate CIs (2.17) can have a negative lower bound even though the support of σ is positive; however, the proposed Bayesian method in the subsequent subsections can overcome this limitation.
Based on the Fisher information (2.16), we provide the objective priors (the Jeffreys and reference priors and the second-order PMP) here.
The Jeffreys prior is proportional to the square root of the determinant of the Fisher information. Therefore, the Jeffreys prior for (μ,σ) is given by
Note that the Jeffreys prior may lead to some undesirable frequentist properties in the presence of nuisance parameters (Bernardo and Smith, 1994). The following theorem provides a reference prior for (μ,σ) and examines the frequentist properties of the provided priors by observing if they satisfy the second PMP criteria.
This is proved by using the algorithm provided by Berger and Bernardo (1989). We first give a proof procedure when μ is the parameter of interest. Define the conditional reference prior for σ given μ as
where I_{ij} is the (i, j) entry of the Fisher information (2.16). Choose a sequence of compact sets Ω_{i} = (d_{1}_{i}, d_{2}_{i}) × (d_{3}_{i}, d_{4}_{i}) for (μ,σ) such that d_{1}_{i}, d_{3}_{i} → 0, d_{2}_{i} → x_{U}_{(1)}, and d_{4}_{i} →∞ as i→∞. Then, the normalizing constant K_{1}_{i}(μ) is given by
where 1_{Ω} denotes the indicator function on Ω. Therefore, when μ is the parameter of interest, the marginal reference prior for μ and the reference prior (μ,σ) are respectively given by
where σ_{0} is any fixed point. Note that the reference priors have the same form regardless of the parameters of interest. Therefore, the notation π_{R}(μ,σ) is used for both reference priors. This completes the proof.
Theorem 2
The second-order PMP has the form of 1/σ. Therefore, the reference prior (2.19) is the second-order PMP, while the Jeffreys prior (2.18) is not.
Proof
The formula for finding the second-order PMP for the multi-parameter case is provided in Peers (1965). When μ is the parameter of interest, the second-order PMP should satisfy the following partial differential equation:
By Remark 2, the resulting posterior is the same as that of Seo and Kim (2017). For comparison, we re-write the results with those based on the reference prior (2.19)
Seo and Kim (2017) proved that the posterior distribution (3.3) is proper by showing that the normalizing constant (3.4) is integrable for μ and σ. In the same way, we prove that the posterior distribution (3.1) is proper.
By integrating out σ from the normalizing constant (3.2), we have
However, a Markov chain Monte Carlo (MCMC) technique should be applied to generate the MCMC samples from the marginal posterior distributions since marginal posterior distributions (3.5) and (3.6) cannot be reduced analytically to any well-known distribution. Seo and Kim (2017) considered the uniform distribution on (0, x_{U}_{(1)}) as a proposed distribution in the Metropolis-Hastings algorithm and obtained satisfactory results. The Metropolis-Hastings algorithm is applied to generate MCMC samples μ_{i} (i = 1, …, N) from the marginal posterior distributions (3.5) and (3.6).
Theorem 4
The marginal posterior distributions for σ under the Jeffreys prior (2.18) and the reference prior (2.19) are, respectively,
The conditional posterior density function (3.7) is the PDF of the square root inverse gamma distribution with the scale parameter k + 1/2 and the shape parameter (x_{U}_{(}_{k}_{)} − μ)^{2}, and the conditional posterior density function (3.8) is the PDF of the square root inverse gamma distribution with the scale parameter k and the shape parameter (x_{U}_{(}_{k}_{)} − μ)^{2}.
By Remark 3, the MCMC samples σ_{i} (i = 1, …, N) can be generated from the corresponding square root inverse gamma distribution as soon as the MCMC samples μ_{i} (i = 1, …, N) are generated from the marginal posterior distribution for μ. Then, the Bayes estimators of μ and σ under the squared error loss function (SELF) are obtained respectively as
where M is the number of burn-in samples. The subscript B under the Jeffreys prior (2.18) and the reference prior (2.19) is substituted by JB and RB, respectively. The highest posterior density (HPD) credible intervals (CrIs) for μ and σ are constructed by the method provided in Chen and Shao (1998).
4. Application
This section assesses how the proposed analysis method is valid through Monte Carlo simulations and real data analysis.
4.1. Simulation study
This subsection reports the mean squared errors (MSE) and biases of the proposed estimators, and the coverage probabilities (CPs) and average lengths (ALs) for the proposed intervals at the 0.95 level to assess their validity. The upper record values are generated from two-parameter Rayleigh distribution with μ = 0.5 and σ = 1 for different k = 5(2)15. All results based on 1,000 simulations are displayed in Figures 1 and 2.
From Figures 1 and 2, we can see that the Bayes estimators under the reference prior (2.19) show the best performance in terms of the MSE and bias. In addition, the HPD CrIs under the reference prior (2.19) are well matched to their corresponding nominal levels. The HPD CrIs under the Jeffreys prior (2.18) and the approximate CIs based on the MLEs have lower CPs than the corresponding nominal levels, but the CPs approach the nominal levels as the size k increases. For ALs, the HPD CrIs under the proposed priors (2.18) and (2.19) have smaller ALs than approximate CIs based on the MLEs do, and the ALs of the HPD CrIs under the two priors have little difference. The results indicate that the proposed objective Bayesian method is superior to the corresponding maximum likelihood counterpart in terms of frequentist properties.
4.2. Real data
In this subsection, we analyze a real data set that represents survival times in days for a group of lung cancer patients, as provided in Lawless (1982):
which have been analyzed by some authors. Soliman and Al-Aboud (2008) showed that the Rayleigh distribution with the scale parameter fits in the analysis of the observed record data. Seo and Kim (2017) applied an objective Bayesian method under the reference prior with partial information to the observed record data and showed that the proposed Bayesian model fits the observed record data well. We focus on comparing the Bayesian models under the Jeffreys prior (2.18) and reference prior (2.19) here. Tables 1 and 2 report numerical results and posterior probabilities (PPs) of the HPD CrIs as well as estimation results of unknown parameters based on the observed upper record values. As mentioned in Remark 2, the reference prior (2.19) has the same form as the reference prior with partial information provided in Seo and Kim (2017), and it has been proved in their study that the Markov chains under the provided prior mix well and converge to the stationary distribution very quickly. Therefore, we do not report the results for the validity of the generated MCMC samples.
Tables 1 and 2 show that the Bayes estimates based on the generated MCMC samples and the numerical results are very close to each other. In addition, the 95% HPD CrIs satisfy their PPs well. It is worth noting that the lower bound of the approximate 95% CI based on the MLE σ̂ has a negative value in Table 2, although the support of σ is positive. This result can be sufficient because the approximate CI for σ is obtained by the asymptotic normality of the MLE. Therefore, it is natural to choose the Bayesian inference.
The quality of models under the derived priors can be evaluated through posterior predictive checking. The data drawn from the fitted model, namely replications, should look similar to observed data if the model is adequate. Let X^{rep} be a replication from a fitted model. Then, the Bayesian predictive density function of X^{rep} under a prior distribution π(θ) is given by
where f_{Xrep} (x^{rep}) is the marginal density function of X^{rep}. Let ${X}^{\text{rep}}\equiv {X}_{U(i)}^{\text{rep}}$ be the replication from the model under the Jeffreys prior (2.18). Then, the MCMC sample ${X}_{U(i)}^{\text{rep}(j)}$ is obtained from the marginal density function ${f}_{{X}_{U(i)}^{\text{rep}}}({x}_{U(i)}^{\text{rep}})$, with μ_{j} and σ_{j} generated from the joint posterior distribution (3.1). Therefore, the replications of the observed upper record values are given by
The replications from the model under the reference prior (2.19) can be obtained similarly. These replications are reported in Table 3. As is conducted in Seo and Kim (2017), we evaluate the Bayesian models through four discrepancy statistics:
Under the provided priors (2.18) and (2.19), we present the histograms and kernel densities of the discrepancy statistics in Figures 3–6.
Table 3 shows that the replications under the Jeffreys prior (2.18) are closer to the observed upper record values than the replications under the reference prior (2.19) are. Figures 3–6 show little difference between the models under the priors (2.18) and (2.19) for D_{1}. In addition, the model under the Jeffreys prior (2.18) shows better performance than that under the reference prior (2.19) for D_{2}. In contrast, the model under the reference prior (2.19) shows better performance than that under the Jeffreys prior (2.18) for D_{3} and D_{4}. However, their differences are not significant.
5. Conclusions
This paper provides an objective Bayesian analysis method based on the objective priors (the Jeffreys and reference priors, and the second-order PMP) for unknown parameters of the two-parameter Rayleigh distribution when the upper record values are observed. To obtain the objective priors, we derived the Fisher information matrix for unknown parameters in terms of the second derivative of the log-likelihood function using Leibniz’s rule. In the simulation study, we showed that the model under the reference prior (2.19) is superior to that under the Jeffreys prior (2.18) and the corresponding maximum likelihood counterpart in terms of frequentist properties. In addition, we showed the limitation of the approximate CI based on the MLE through real data analysis. Based on these results, we recommend the objective Bayesian method under the reference prior (2.19) in the absence of prior information.
Fig. 3. (a) Histogram and kernel density of D_{1} under the Jeffreys prior () and (b) Histogram and kernel density of D_{1} under the reference prior ().
Fig. 4. (a) Histogram and kernel density of D_{2} under the Jeffreys prior () and (b) Histogram and kernel density of D_{2} under the reference prior ().
Fig. 5. (a) Histogram and kernel density of D_{3} under the Jeffreys prior () and (b) Histogram and kernel density of D_{3} under the reference prior ().
Fig. 6. (a) Histogram and kernel density of D_{4} under the Jeffreys prior () and (b) Histogram and kernel density of D_{4} under the reference prior ().
TABLES
Table 1
Estimates and the corresponding 95% CIs and HPD CrIs for μ
Replications of the observed upper record values under the provided priors
i
1
2
3
4
5
π_{J} (μ,σ)
7.75
9.52
10.85
11.96
12.93
π_{R}(μ,σ)
7.76
9.71
11.17
12.39
13.46
References
Ahsanullah, M (1995). Record Statistics. New York: Nova Science Publishers
Arnold, BC, Balakrishnan, N, and Nagaraja, HN (1998). Records. New York: Wiley
Basak, P, and Balakrishnan, N (2003). Maximum likelihood prediction of future record statistics. Mathematical and Statistical Methods in Reliability. 7, 159-175.
Berger, JO, and Bernardo, JM (1989). Estimating a product of means: Bayesian analysis with reference priors. Journal of the American Statistical Association. 84, 200-207.
Berger, JO, and Bernardo, JM (1992). On the development of reference priors (with discussion). Bayesian statistics IV, Bernardo, JM, ed: Array, pp. 35-60
Bernardo, JM (1979). Reference posterior distributions for Bayesian inference (with discussion). Journal of the Royal Statistical Society. Series B. 41, 113-147.
Bernardo, JM, and Smith, AFM (1994). Bayesian Theory. Chichester: Wiley
Burkschat, M, and Cramer, E (2012). Fisher information in generalized order statistics. Statistics. 46, 719-743.
Chen, MH, and Shao, QM (1998). Monte Carlo estimation of Bayesian credible and HPD intervals. Journal of Computational and Graphical Statistics. 8, 69-92.
Chandler, KN (1952). The distribution and frequency of record values. Journal of the Royal Statistical Society. Series B. 14, 220-228.
Dyer, DD, and Whisenand, CW (1973). Best linear unbiased estimator of the parameter of the Rayleigh distribution - part I: small sample theory for censored order statistics. IEEE Transactions on Reliability. 22, 27-34.
Jaheen, ZF (2003). A Bayesian analysis of record statistics from the Gompertz model. Applied Mathematics and Computation. 145, 307-320.
Jeffreys, H (1961). Theory of Probability and Inference. London: Cambridge University Press
Kim, C, and Han, K (2009). Estimation of the scale parameter of the Rayleigh distribution with multiply type-II censored sample. Journal of Statistical Computation and Simulation. 79, 965-976.
Lee, WC, Wu, JW, Hong, ML, Lin, LS, and Chan, RL (2011). Assessing the lifetime performance index of Rayleigh products based on the Bayesian estimation under progressive type II right censored samples. Journal of Computational and Applied Mathematics. 235, 1676-1688.
Lawless, JF (1982). Statistical Model & Methods for Lifetime Data. New York: Wiley
Madi, MT, and Raqab, MZ (2004). Bayesian prediction of temperature records using the Pareto model. Environmetrics. 15, 701-710.
Peers, HW (1965). On confidence sets and Bayesian probability points in the case of several parameters. Journal of the Royal Statistical Society: Series B. 27, 9-16.
Polovko, AM (1968). Fundamentals of Reliability Theory. New York: Academic Press
Rayleigh, L (1880). On the Resultant of a large Number of Vibrations of the same Pitch and of arbitrary Phase. Philosophical Magazine and Journal of Science. 10, 73-78.
Raqab, MZ, and Madi, MT (2002). Bayesian prediction of the total time on test using doubly censored Rayleigh data. Journal of Statistical Computation and Simulation. 72, 781-789.
Romano, JP, and Siegel, AF (1986). Counterexamples in Probability and Statistics: Wadsworth and Brooks/Cole
Seo, JI, and Kim, Y (2017). Objective Bayesian analysis based on upper record values from two-parameter Rayleigh distribution with partial information. Journal of Applied Statistics. 44, 2222-2237.
Soliman, AA, and Al-Aboud, FM (2008). Bayesian inference using record values from Rayleigh model with application. European Journal of Operational Research. 185, 659-672.
Wu, SJ, Chen, DH, and Chen, ST (2006). Bayesian inference for Rayleigh distribution under progressive censored sample. Applied Stochastic Models in Business and Industry. 22, 269-279.
Wang, BX, and Ye, ZS (2015). Inference on the Weibull distribution based on record values. Computational Statistics and Data Analysis. 83, 26-36.
Wang, BX, Yu, K, and Coolen, FPA (2015). Interval estimation for proportional reversed hazard family based on lower record values. Statistics and Probability Letters. 98, 115-122.
Welch, BL, and Peers, HW (1963). On formulae for confidence points based on integrals of weighted likelihoods. Journal of the Royal Statistical Society. Series B. 35, 318-329.