In this survey, we use the normal linear model to demonstrate the use of the linear Bayes method. The superiorities of linear Bayes estimator (LBE) over the classical UMVUE and MLE are established in terms of the mean squared error matrix (MSEM) criterion. Compared with the usual Bayes estimator (obtained by the MCMC method) the proposed LBE is simple and easy to use with numerical results presented to illustrate its performance. We also examine the applications of linear Bayes method to some other distributions including two-parameter exponential family, uniform distribution and inverse Gaussian distribution, and finally make some remarks.
The linear Bayes method was originally proposed by Hartigan (1969), which suggests that in Bayesian statistics one can replace a completely specified prior distribution by an assumption with just a few moments of the distribution. It has been subsequently discussed by Rao (1973) from linear optimization viewpoint. Lamotte (1978) later develops a class of linear estimators, called Bayes linear estimators, by searching, among all linear estimators that have least average total mean squared error. Goldstein (1983) considers the problem of modifying the linear Bayes estimator for the mean of a distribution of unknown form using a sample variance estimate. Heiligers (1993) studies the relationship between linear Bayes estimation and minimax estimation in linear models with partial parameter restrictions. Hoffmann (1996) proposes a well-described subclass of Bayes linear estimators for the unknown parameter vector in the linear regression model with ellipsoidal parameter constraints and obtains a necessary and sufficient condition to ensure that the considered Bayes linear estimators improves the least squared estimator over the whole ellipsoid regardless of the selected generalized risk function. In the framework of empirical Bayes, Samaniego and Vestrup (1999) and Pensky and Ni (2000) respectively construct linear empirical Bayes estimators and establish their superiorities over standard and traditional estimators. In application fields, Busby
In this paper, along the same line as in Wang and Singh (2014), we use the normal linear model as an example to demonstrate how to apply a linear Bayes method to simultaneously estimate all the parameters involved in the model and elaborate advantages and potential disadvantages.
Let
This model is called the normal linear model as defined by Arnold (1980) and adopted by many other authors as well. Let
Define
From the Bayesian viewpoint, note that in most cases past experience about the parameters
where
where
In the following, enlightened by Rao (1973), we employ the linear Bayes method to propose a linear Bayes estimator (LBE) for the parameters
The survey is organized as follows: In Section 2 we define the LBE for the parameter vector
Throughout this paper, for two nonnegative definite matrices
Denote
Put
where align and
Thus, we have the following conclusion.
From the constraint
Hence
For given
where
Substituting (
which yields
Together with
In the definition of
Note that
where we use
Since
Denote
However,
Comparing (
The proof of Theorem 2 is completed.
Moreover, note that the MLE of
Thus,
We rewrite
where
Hence, in order to establish the MSEM superiority of
for
Denote Cov^{−1}(
Thus, to prove (
or equivalently to show that
for
Set Δ = Var(
Further, using
where
Note that Δ = Var(
and accordingly
where we use the facts that
Hence, Theorem 3 has been proved.
For the two-parameter exponential family given by
where
Under the assumption that the prior
Let
Similarly, using the assumption that the prior
Let
where
where
To illustrate Theorem 2 and Theorem 3 we investigate the case of two-dimensional normal linear model, i.e.
where we assume that (
Define the percentages of improvement of
For different sample size
As stated in Theorems 2 and 3, since the above priors belong to the family (
For the model (
Suppose
Denote
and given
Note that the posterior density of (
where
However, it is almost impossible to calculate
Note that the posterior conditional densities of
where
The Gibbs sampler was originally developed by Geman and Geman (1984) as applied to image processing and the analysis of Gibbs distributions on a lattice. It is brought into mainstream statistics through the articles of Gelfand and Smith (1990) and Gelfand
Step 1. Choose the initial values of
Step 2. Generate
Step 3. Repeat Step 2 for
Step 4. Calculate the Bayes estimator of
Note that under the above priors, it is readily seen that
In the following table we first calculate the values of
The above numerical comparisons indicate two trends, one is that for the same prior, ||
Two cases are considered for the two-parameter exponential family. In case (I) we assume that the parameters
In the case of the uniform distribution
Specifically, let
where we utilize the relationship between the inverse Gamma and the
For the two-parameter inverse Gaussian distribution IG(
In above simulation, it should be noted that the problem of deciding when to stop the chain is an important issue and is the topic of current research in MCMC methods. If the resulting sequence has not converged to the target distribution, then the estimators and inferences we get from it are suspect. Let
This paper uses the normal linear model
The proposed linear Bayes estimator is simple and easy to calculate as well as a good approximation in many situations; the linear Bayes method works especially well for the case of uniform distribution.
We can always define a linear Bayes estimator if there exists sufficient statistic for the parametric model; subsequently, the conclusions of Theorems 2 and 3 always hold.
However, an advantage of the usual Bayes estimator over the linear Bayes estimator is that the former allows for noninformative (improper) priors. Of note is that the linear Bayes estimator may be an inadequate approximation in some situations even for the cases of proper priors. Hence there is still scope for the linear Bayes method to be improved. For instance, for some cases a quadratic Bayes estimator would be a better alternative. However, for the case of normal linear model, one can consider to add more other statistics into the definition of
IP(
Priors | IP( | IP ( | tr (Cov ( | |
---|---|---|---|---|
5 | 0.9712 | 0.9542 | ||
10 | 0.9516 | 0.9404 | 7/3 | |
20 | 0.9066 | 0.8974 | ||
5 | 0.9234 | 0.8779 | ||
10 | 0.8803 | 0.8524 | 40/3 | |
20 | 0.7706 | 0.7480 | ||
5 | 0.8458 | 0.7537 | ||
10 | 0.7265 | 0.6627 | 124/3 | |
20 | 0.5369 | 0.4961 |
||
The prior parameters | || | |
---|---|---|
1.3369 | ||
20 | 1.4472 | |
1.8390 | ||
1.0589 | ||
50 | 1.1407 | |
1.4855 |
Where (