In this study, we consider the likelihood ratio test for the covariance matrix of the multivariate normal data. For this, we propose a method for obtaining null distributions of the likelihood ratio statistics by the Monte-Carlo approach when it is difficult to derive the exact null distributions theoretically. Then we compare the performance and precision of distributions obtained by the asymptotic normality and the Monte-Carlo method for the likelihood ratio test through a simulation study. Finally we discuss some interesting features related to the likelihood ratio test for the covariance matrix and the Monte-Carlo method for obtaining null distributions for the likelihood ratio statistics.
The inferences about scale parameter or variance in the univariate case, have many results shown in the literature. Especially, the likelihood ratio (LR) procedure for testing the problem for variance has been completely achieved and verified for its efficiency and uniqueness under normality. For example, LR test statistics follow exactly the chi-square distributions under the null hypothesis and the LR tests themselves are optimal in the sense of power of test. However for the multivariate case with multivariate normality, only asymptotic procedures are available even though the statistics for the covariance matrix have been derived by applying the LR principle. The reason for this phenomenon may come from the fact that the distributional theories for the matrix-valued statistics have not been fully investigated. Any prospect for the theoretic development would also not be seen in any near future because of complexity or non-existence of distributions for matrix-valued statistics. For this reason, several modifications with high dimensional cases have been reported (Bai
For the test procedure of the covariance matrix, the LR functions have been mainly expressed with the corresponding eigenvalues of the sample covariance matrix. Even though the eigenvalues of a sample covariance matrix may consist of a vector instead of a matrix, discussions of the distributions and their properties for the LR functions or related statistics have not been fully investigated or justified in a theoretic manner. All the results up to this date, have been confined only to limiting distributions based on log likelihood arguments. Therefore we may question how close the limiting distributions are to the exact ones if they are obtainable or whether the conclusions based on the limiting distributions would be reliable when the
In the multivariate analysis under normality, the distributions of LR statistics have been fully studied and tabulated systematically for many cases. However when it would be difficult to derive the exact distributions theoretically, one may consider deriving the limiting distributions asymptotically which may be obtained using log likelihood arguments. However, one may obtain null distributions using one of the popular re-sampling methods such as bootstrap or permutation methods that are heavily dependent on the computer power and its facilities. Along with this, one may also obtain the null distribution of an LR statistic, LR, in the following idea and rationale. For this discussion, let
In this research, we consider obtaining null distributions of the LR functions for testing the covariance matrices under the multivariate normal distribution and compare them with the limiting distributions. For this purpose, the rest of this paper will be organized in the following order. In the next section, we review the LR tests with limiting distributions in some detail and propose the MC method to obtain quantiles as critical values for the LR functions. Then we illustrate the usage of distributions with numerical examples for the decision of structures of the covariance matrices and compare the precision between the two methods by obtaining empirical powers through a simulation study in the Section 3. In the Section 4, we discuss some interesting features related with the LR functions and the MC method.
Let
with the condition that the mean vector
is the maximum likelihood estimator of ∑, where
Then the testing rule based on
Generate pseudo random normal vectors of size
Then compute the LR statistic of
Iterate (I) and (II)
From the ordered statistics of
Repeat
Then one can carry out the LR test by obtaining the critical values for any given significance levels or
In order to investigate the behavior of quantiles obtained from the MC method and compare them with quantiles from the chi-square distributions which are the limiting distributions, we have obtained quantiles (or critical values) of
Then we may finish the test for testing
It would be interesting to compare their performance and precision between the two LR tests. This will be accomplished in the next section with a simulation study. In the tables of the next section, MC implies the LR test based on the permutation principle and AS means the LR one applying asymptotically the chi-square distribution. We begin the next section with some numerical examples.
We first illustrate the two tests, MC and AS with the head data of brothers (Frets, 1921) summarized in Mardia
and
We obtained the
As another example, 24 turtles were collected and for each turtle, the carapace dimensions were measured in three mutually perpendicular directions of space: length, maximum width, and height. More detailed definitions and explanations of these measurements and contents, you may refer to Jolicoeur and Mosimann (1960). Each specimen is therefore represented in this study by a set of three measurements. Originally Jolicoeur and Mosimann (1960) were interested in the principal component analysis among three variables. However in this study, we are interested in detecting the structures of covariance matrix which are described as the two null hypotheses,
and
We have obtained that 6.2096584 and 110.33072 as the values of
Now we compare performance and precision between the two LR tests, MC and AS by obtaining empirical powers through a simulation study under several scenarios for the bivariate normal case. We conducted a simulation study by generating bivariate normal pseudo-random vectors with a zero mean vector and varying the values of components of the covariance matrix with six cases of the sample sizes, 5, 10, 15, 20, 25, and 30 in order to inspect the behaviors of the two tests for the small sample cases. In the tables, (1, 1, 0) means that
We first note that from Table 7, the results are almost symmetric when the values of covariance are assigned with the opposite signs. For this reason, we consider only positive values for covariance in Tables 8 and 9. From Table 7, MC test achieves its nominal significance level well while AS one always achieves higher values than the nominal significance level for all cases. The reason for this may come from the fact that quantiles of the chi-square distribution with 3 df are lower than those from the MC method for all sample sizes as observed in Table 1. However as the sample sizes increase, empirical significance levels approach to the nominal one for the AS test. It is therefore recommended to use quantiles obtained from the MC method for the small sample case. In Table 8, one may observe that reversal phenomenon about empirical powers happened for the AS test. Even though the sample size increases, the empirical power decreases for the AS test for some cases. In all the tables, as the difference between two variances increases and/or covariance approaches to 1, the empirical powers increases. Finally we note that the empirical powers of MC are all lower than those of AS as we have expected since the quantiles of
In (
In Tables 1 and 2, we have already noticed that as the sample sizes increase, quantiles obtained from the MC method approach to the limiting quantiles of the chi-square distribution. This phenomenon is a standard that confirms the large sample approximation theory in general. Therefore we may recommend to apply the MC method, when sample sizes are small. Or even for any reasonable sample sizes, the computation time to obtain a distribution with the MC method would be negligible.
The MC method can be applied other than the LR statistics for the test of covariance matrix if the null hypothesis is non-ambiguous and well-defined. For example, Park (2017) has used the MC method to obtain a null distribution of LR statistics for the multivariate simultaneous test. However it would be difficult to apply the MC method for a nonparametric test since the distribution of population for the null hypothesis is too broad to choose a specific one. One may also note that Kim and Cheon (2013) applied the MC method to estimate the posterior distribution in the Bayesian analysis.
Finally we note that the quantiles from the chi-square distribution in Table 1 are higher than those obtained the MC method for all cases. This is why the empirical powers of AS test in Table 7 to 9 are higher than those of the MC one for all cases. One may also be suspicious that the empirical powers of MC test are not exactly 0.0500 under
Quantiles (or critical values) for some selected probabilities (or significance levels) and sample sizes for
Quantile | ||||||
---|---|---|---|---|---|---|
10 | 15 | 20 | 25 | 30 | ||
0.1148 | 0.1438 | 0.1328 | 0.1279 | 0.1251 | 0.1232 | |
0.3518 | 0.4402 | 0.4067 | 0.3917 | 0.3829 | 0.3775 | |
0.5844 | 0.7310 | 0.6754 | 0.6505 | 0.6361 | 0.6270 | |
6.2514 | 7.7819 | 7.2094 | 6.9489 | 6.8005 | 6.6736 | |
7.8147 | 9.7171 | 9.0078 | 8.6839 | 8.4984 | 8.3793 | |
11.3449 | 14.0750 | 13.0628 | 12.5967 | 12.3305 | 12.1624 |
Quantiles (or critical values) for some selected probabilities (or significance levels) and sample sizes for
Quantile | ||||||
---|---|---|---|---|---|---|
10 | 15 | 20 | 25 | 30 | ||
0.8721 | 1.1344 | 1.0315 | 0.9865 | 0.9611 | 0.9453 | |
1.6354 | 2.1273 | 1.9342 | 1.8503 | 1.8028 | 1.7728 | |
2.2041 | 2.8672 | 2.6068 | 2.4939 | 2.4295 | 2.3893 | |
10.6446 | 13.8392 | 12.5819 | 12.0369 | 11.7311 | 11.5359 | |
12.5916 | 16.3709 | 14.8826 | 14.2378 | 13.8763 | 13.6454 | |
16.8119 | 21.8668 | 19.8689 | 19.0038 | 18.5257 | 18.2169 |
Test | |
---|---|
MC | 0.2722 |
AS | 0.2349 |
Test | |
---|---|
MC | 0.0005 |
AS | 0.0042 |
Test | |
---|---|
MC | 0.4683 |
AS | 0.4001 |
Test | |
---|---|
MC | 0.0000 |
AS | 0.0000 |
Empirical powers by varying covariance only
Test | ( | |||||||
---|---|---|---|---|---|---|---|---|
(1, 1, 0) | (1, 1, 0.2) | (1, 1, −0.2) | (1, 1, 0.5) | (1, 1, −0.5) | (1, 1, 0.8) | (1, 1, −0.8) | ||
MC | 5 | 0.0637 | 0.0751 | 0.0736 | 0.1324 | 0.1350 | 0.3304 | 0.3341 |
10 | 0.0503 | 0.0743 | 0.0711 | 0.2282 | 0.2168 | 0.7265 | 0.7283 | |
15 | 0.0493 | 0.0813 | 0.0824 | 0.3452 | 0.3419 | 0.9419 | 0.9383 | |
20 | 0.0500 | 0.0970 | 0.0906 | 0.4583 | 0.4556 | 0.9921 | 0.9924 | |
25 | 0.0510 | 0.1152 | 0.1120 | 0.5944 | 0.5842 | 0.9999 | 0.9996 | |
30 | 0.0490 | 0.1214 | 0.1229 | 0.6886 | 0.6810 | 1.0000 | 1.0000 | |
AS | 5 | 0.1839 | 0.1969 | 0.2000 | 0.2973 | 0.2924 | 0.6144 | 0.6158 |
10 | 0.0994 | 0.1291 | 0.1276 | 0.3286 | 0.3201 | 0.8790 | 0.8821 | |
15 | 0.0806 | 0.1201 | 0.1209 | 0.4340 | 0.4287 | 0.9835 | 0.9823 | |
20 | 0.0723 | 0.1286 | 0.1255 | 0.5384 | 0.5359 | 0.9983 | 0.9975 | |
25 | 0.0673 | 0.1377 | 0.1356 | 0.6427 | 0.6282 | 0.9999 | 0.9999 | |
30 | 0.0626 | 0.1457 | 0.1457 | 0.7299 | 0.7237 | 1.0000 | 1.0000 |
Empirical powers by varying only one variance with 0 covariance
Test | ( | ||||||
---|---|---|---|---|---|---|---|
(1, 1.2, 0) | (1, 0.8, 0) | (1, 1.5, 0) | (1, 0.5, 0) | (1, 1.8, 0) | (1, 0.2, 0) | ||
MC | 5 | 0.0766 | 0.0609 | 0.1104 | 0.0841 | 0.1537 | 0.2022 |
10 | 0.0700 | 0.0550 | 0.1306 | 0.1200 | 0.2148 | 0.5617 | |
15 | 0.0716 | 0.0573 | 0.1547 | 0.1879 | 0.2830 | 0.8632 | |
20 | 0.0726 | 0.0617 | 0.1842 | 0.2598 | 0.3550 | 0.9722 | |
25 | 0.0741 | 0.0629 | 0.1935 | 0.4602 | 0.3615 | 0.9990 | |
30 | 0.0767 | 0.0711 | 0.2100 | 0.5412 | 0.4354 | 0.9998 | |
AS | 5 | 0.1749 | 0.2111 | 0.1794 | 0.2955 | 0.2038 | 0.5930 |
10 | 0.0974 | 0.1298 | 0.1338 | 0.2812 | 0.2001 | 0.8439 | |
15 | 0.0869 | 0.1172 | 0.1480 | 0.3490 | 0.2603 | 0.9683 | |
20 | 0.0805 | 0.1149 | 0.1730 | 0.4311 | 0.3300 | 0.9963 | |
25 | 0.0843 | 0.1159 | 0.2028 | 0.5108 | 0.3965 | 0.9995 | |
30 | 0.0844 | 0.1202 | 0.2406 | 0.5912 | 0.4708 | 1.0000 |
Empirical powers by varying both variances and covariance
Test | ( | ||||||
---|---|---|---|---|---|---|---|
(1.2, 0.8, 0) | (1.2, 0.8, 0.5) | (1.5, 0.5, 0) | (1.5, 0.5, 0.5) | (1.8, 0.2, 0) | (1.8, 0.2, 0.5) | ||
MC | 5 | 0.0734 | 0.1443 | 0.1343 | 0.2416 | 0.3315 | 0.7114 |
10 | 0.0739 | 0.2647 | 0.2256 | 0.5155 | 0.7313 | 0.9990 | |
15 | 0.0830 | 0.4046 | 0.3414 | 0.7595 | 0.9421 | 1.0000 | |
20 | 0.0994 | 0.5433 | 0.4616 | 0.9021 | 0.9906 | 1.0000 | |
25 | 0.1137 | 0.6919 | 0.5996 | 0.9777 | 0.9994 | 1.0000 | |
30 | 0.1249 | 0.7814 | 0.6880 | 0.9935 | 1.0000 | 1.0000 | |
AS | 5 | 0.1996 | 0.3171 | 0.2957 | 0.4701 | 0.6155 | 0.9713 |
10 | 0.1266 | 0.3780 | 0.3264 | 0.6765 | 0.8812 | 1.0000 | |
15 | 0.1208 | 0.5023 | 0.4310 | 0.8597 | 0.9809 | 1.0000 | |
20 | 0.1282 | 0.6213 | 0.5397 | 0.9501 | 0.9973 | 1.0000 | |
25 | 0.1363 | 0.7308 | 0.6427 | 0.9840 | 0.9997 | 1.0000 | |
30 | 0.1496 | 0.8153 | 0.7254 | 0.9958 | 1.0000 | 1.0000 |