TEXT SIZE

CrossRef (0)
Variance components for two-way nested design data

Jaesung Choi1,a

aDepartment of Statistics, Keimyung University, Korea
Correspondence to: Department of Statistics, Keimyung University, 1095 Dalgubeoldaero, Dalseogu, Daegu 42601, Korea. E-mail: jschoi@kmu.ac.kr
Received January 10, 2018; Revised March 19, 2018; Accepted April 7, 2018.
Abstract

This paper discusses the use of projections for the sums of squares in the analyses of variance for two-way nested design data. The model for this data is assumed to only have random effects. Two different sizes of experimental units are required for a given experimental situation, since nesting is assumed to occur both in the treatment structure and in the design structure. So, variance components are coming from the sources of random effects of treatment factors and error terms in different sizes of experimental units. The model for this type of experimental situation is a random effects model with more than one error terms and therefore estimation of variance components are concerned. A projection method is used for the calculation of sums of squares due to random components. Squared distances of projections instead of using the usual reductions in sums of squares that show how to use projections to estimate the variance components associated with the random components in the assumed model. Expectations of quadratic forms are obtained by the Hartley’s synthesis as a means of calculation.

Keywords : nested, projection, quadratic form, random effects, synthesis
1. Introduction

Nesting can occur in certain experimental situations. When it occurs, it is possible to have nested effects in either the design structure or the treatment structure (or both). If it happens in the design structure of an experiment then there will be different sizes of experimental units where a smaller experimental unit is nested within a larger one. When nesting occurs in the treatment structure the levels of one factor occur with only one level of a second factor. Hence, the levels of the first factor are nested within the level of the second factor.

Models for different nested designs are discussed in the literature such as Milliken and Johnson (1984), Montgomery (1976) and Searle (1971). Some related topics about nested designs are shown in Khan et al. (2017) and Sharma (2014). Constructing a model for nested data depends on the types of factors in the treatment structure and nesting in the design structure. Here, we assume both types of nesting (one from factors the other from different experimental sizes). A factor is random if its levels are treated as a random sample from a population of all possible levels. An effect of a level randomly chosen from a population of levels is called a random effect. A model with only random effects is called random or random effects model. Variance components are to be estimated if a model has random effects in addition to the random error. Various random effects models are discussed in Graybill (1976).

Henderson (1953) developed four methods of computing sums of squares depending on the types of models. For a random effects model Henderson’s Method 1 (or the analysis of variance method) can be used for the calculations of sums of squares. Variance components are estimated by equating sums of squares to their corresponding expected values. The calculations of the coefficients of the variance components can be obtained by Hartley’s (1967) method of synthesis.

Nesting can happen in either factors or design structures; therefore, this paper suggests a model and discuss a method using projections for analysing data in that situation because both types of nesting in nested designs are uncommon in the literature.

This paper discusses a method of using projections as a method of getting sums of squares for estimation of variance components in a two-way random model with nested effects in the design structure. The use of projections for calculations of sums of squares actually provides a different way of partioning the total sum of squares in the analysis of variance for unbalanced data. For finding appropriate projections it is necessary to decompose the vector space generated by the model matrix into the orthogonal vector subspaces at first. Establishing proper models then become important to understand the appropriate projections. The method is discussed for the estimation of variance components for nested design data.

2. Nested random effects model

Suppose that there are two factors A and B with the levels of B being nested within the levels of factor A in the treatment structure. Let a and b be the number of levels chosen randomly from each population of levels. The two factors then become random. Nesting then occurs in the design structure if smaller sizes of experimental units are assigned to the levels of the nested factor B for a given experimental situation. When this occurs, additional random error should be considered as a variation of the source due to the different size of the experimental unit.

Let yi jkl denote the observation of the lth smaller unit treated with level k of factor B nested in jth larger unit treated with level i of factor A. The model for the assumed experimental situation is

$yijkl=μ+αi+δj(i)+βk(ij)+ɛl(ijk),i=1,2,…,a; j=1,2,…,ai; k=1,2,…,b; l=1,2,…,n(ijk),$

where μ is an overall mean, αi is the effect of the ith level of random factor A, δj(i) is the random error of the jth larger experimental unit assigned to the ith level of factor A, βk(i j) is the effect of the kth level of factor B nested in ith level of A assigned to the jth larger experimental unit and εl(i jk) is the random error of the lth smaller experimental unit treated with the kth level of B and nested in jth larger unit treated with the ith level of A. In the random model, we consider that the αi’s and βk(i j)’s are all assumed to be random with zero means and variances $σα2Iα$ and $σβ2Iβ$ respectively. δj(i)’s and εl(i jk)’s are random errors one from larger experimental unit and the other from the smaller one. The two random errors are assumed to have variances $σδ2Iδ$ and $σɛ2Iɛ$ respectively. All of the random variables in the model are assumed to be independent of each other.

The model has two sizes of experimental units and thus two error terms. The matrix form of the model can be expressed as

$y=jμ+Xαα+Xδδ+Xββ+ɛ,$

where y denotes the n × 1 vector of observations ($n=∑i=1a∑j=1ai∑k=1bnijk$); j denotes n × 1 vector of ones; Xα is the n × a coefficients matrix of α; α denotes the a × 1 random vector and is assumed to be distributed as $N(0,σα2Iα)$; Xδ denotes the n × r coefficients matrix of δ; δ denotes the n × r random vector of errors ($r=∑ia∑j=1ai$) associated with the larger experimental units and is assumed to be distributed as $N(0,σδ2Ir)$; Xβ denotes the coefficients matrix of β; β denotes the b × 1 random vector and is assumed to be distributed as $N(0,σβ2Ib)$; ε is the usual n×1 random error vector and is assumed to be distributed as $N(0,σɛ2In)$.

The parameters of the model are μ, $σα2,σδ2,σβ2$, and $σɛ2$. There are many ways to estimate variance components from unbalanced data; however, Henderson’s Method 1 (largely used for random model analysis) is compared with a method of using projections. Henderson’s Method 1 (also called the analysis of variance) consists of equating mean squares to expected values. Sums of squares for the analysis of variance are given by R( ) notation denoting reduction in sum of squares due to fitting any particular model. It is therefore necessary to fit two different models for the calculations of R( ) terms; however, this laborious undertaking can be avoided by the use of projection. When this is used, R( ) terms are interpreted as squared distances of projections defined on proper vector subspaces.

3. Calculating sums of squares

To have R( ) terms in analyses of variance by Henderson’s Method 1 let X be the model matrix of the equation in (2.2). Projections can be used to calculate the sums of squares due to random effects in the model. The model for obtaining error sums of squares due to the random vector of errors from smaller size experimental units is

$y=Xν+ɛ,$

where X = (j, Xα, Xδ, δ, Xβ) and ν = (μ, α, δ, β)′. Let VX be the vector subspace generated by the model matrix X of the model (3.1). Then, XXy is the projection of y onto the vector subspace, VX where X denotes the Moore-Penrose generalized inverse. This is done first for identifying the vector subspace generated by the model matrix X before fitting sub-models of the equation in (3.1). Fitting the model (3.1) to data is equivalent to get XXy by projection. R(ν) is obtained by yXXy denoting the squared distances of XXy. Denote this squared distance by SSR. Then, SSE is y′(IXX)y denoting the sum of squares for random vector of errors. XXy is used for establishing up the submodel with which sum of squares due to μ is obtained. The model is

$XX-y=XX-(Xν+ɛ)=XX-(jμ+Xαα+Xδδ+Xββ+ɛ)=XX-jμ+XX-(Xαα+Xδδ+Xββ+ɛ)=XX-jμ+ɛM=jμ+ɛM.$

where εM = Xδδ + Xββ + ε. Denoting the reduction in sum of squares due to fitting (3.2) by SSM, we obtain it as squared distance by the projection of XXy into the vector subspace that is spanned by the model matrix, j and turns out to be SSM = yj jy. The model for fitting α is given by

$(XX--jj-)y=(XX--jj-)(jμ+Xαα+Xδδ+Xββ+ɛ)=(XX--jj-)Xαα+(XX--jj-)(Xδδ+Xββ+ɛ)=(XX--jj-)Xαα+ɛA=XAα+ɛA,$

where XA = (XXj j)Xα and εA = (XXj j)(Xδδ + Xββ + ε). The projection of equation (3.3) is $XAXA-(XX--jj-)y$. Then, SSA is given by $y′XAXA-y$. The model for random vector δ is given from the residual vector after fitting of equation (3.3), which is

$(XX--jj--XAXA-)y=(XX--jj--XAXA-)Xδδ+(XX--jj--XAXA-)(Xββ+ɛ)=XDδ+ɛD,$

where $XD=(XX--jj--XAXA-)Xδ$ and $ɛD=(XX--jj--XAXA-)(Xββ+ɛ)$. From the model fitting of (3.4), we get $XDXD-y$ as its projection. Then, $y′XDXD-y$ is the squared distance of it, which is denoted by SSD. The model for β is

$(XX--jj--XAXA--XDXD-)y=(XX--jj--XAXA--XDXD-)Xββ+(XX--jj--XAXA--XDXD-) ɛ=XBβ+ɛB,$

where $XB=(XX--jj--XAXA--XDXD-)Xβ$ and $ɛB=(XX--jj--XAXA--XDXD-)ɛ$. The sum of squares due to fitting β after μ, α, and δ is the squared distance of the projection, $XBXB-y$, which is denoted by SSB.

4. Expectations of sums of squares

The covariance matrix of y of (2.2) is

$Var(y)=Var(jμ+Xαα+Xδδ+Xββ+ɛ)=σα2XαXα′+σδ2XδXδ′+σβ2XβXβ′+σɛ2In=Σ.$

Let yQy be a quadratic form in observations of y. Then

$E(y′Qy)=tr(QΣ)+μ2jn′Qjn=tr(QΣ),$

where tr(M) is the sum of the diagonal elements of the sqare matrix M. The sums of squares in the analysis of variance are constructed such that $μ2jn′Qjn=0$. Therefore, the expectations of the sums of squares do not depend on μ. The expectation of the quadratic form yQy is

$E (y′Qy)=tr [Q (σα2XαXα′+σδ2XδXδ′+σβ2XβXβ′+σɛ2In)]=σα2tr (Xα′QXα)+σδ2tr (Xδ′QXδ)+σβ2tr (Xβ′QXβ)+σɛ2tr(Q)=cqασα2+cqδσδ2+cqβσβ2+cqɛσα2,$

where c’s are constants denoting the traces. These properties are applied to the calculations of all the expectations of squared distances of the projections. The set of expectations for four sums of squares is

$E(SSA)=tr [(XAXA-)Σ]=σα2cAα+σδ2cAδ+σβ2cAβ+σɛ2cAɛ,E(SSD)=tr [(XDXD-)Σ]=σα2cDα+σδ2cDδ+σβ2cDβ+σɛ2cDɛ,E(SSB)=tr [(XBXB-)Σ]=σα2cBα+σδ2cBδ+σβ2cBβ+σɛ2cBɛ,E(SSE)=tr [(I-XX-)Σ]=σα2cEα+σδ2cEδ+σβ2cEβ+σɛ2cEɛ.$
5. Estimation of variance components

Variance components can be estimated by equating the sums of squares to their expected values as follows.

$SSA=tr [(XAXA-)Σ]=σα2cAα+σδ2cAδ+σβ2cAβ+σɛ2cAɛ,SSD=tr [(XDXD-)Σ]=σα2cDα+σδ2cDδ+σβ2cDβ+σɛ2cDɛ,SSB=tr [(XBXB-)Σ]=σα2cBα+σδ2cBδ+σβ2cBβ+σɛ2cBɛ,SSE=tr [(I-XX-) Σ]=σα2cEα+σδ2cEδ+σβ2cEβ+σɛ2cEɛ.$

Since the equations of (5.1) are linear in variance components, $σα2,σδ2,σβ2$, and $σɛ2$, solutions are given to a system of linear equations such that

$(SSASSDSSBSSE)=(cAαcAδcAβcAɛcDαcDδcDβcDɛcBαcBδcBβcBɛcEαcEδceβcEɛ) (σα2σδ2σβ2σɛ2) or S=Cσ2,$

where S denotes (SSA, SSD, SSB, SSE)′, C matrix of the coefficients and $σ2=(σα2,σδ2,σβ2,σɛ2)′$ in (5.1). The normal equations for (5.2) are:

$C′Cσ2=C′S.$

From the normal equations, we get σ̂2 = (CC)−1(CS).

6. An example

Montgomery (1976)’s data from a two-way nested design is used as an example for the use of projections. Table 1 shows the coded purity data set.

For illustration, suppliers and batches are assumed to be random with batches nested within suppliers. The model for this two-stage nested design is a random model with just one error term and shown as

$yijk=μ+αi+βj(i)+ɛk(ij), i=1,2,3, j=1,2,3,4, k=1,2,3,$

where μ is overall mean, αi is the effect of randomly chosen ith supplier and βj(i) is the effect of the jth batch which is randomly chosen from a population of batches nested in ith supplier. This model in matrix notation is

$y=jμ+XSα+XB(S)β+ɛ,$

where j is 12 × 1 vector of ones, XS is a 12 × 3 coefficients matrix of α, α is the 3 × 1 random vector assumed to have multivariate normal distribution $N3(0,σα2I3)$, XB(S ) is 12 × 4 coefficients matrix of β, β is the 4 × 1 random vector assumed to have multivariate normal distribution $N4(0,σβ2I4)$, and ε is the 12 × 1 random error vector assumed to have multivariate normal distribution $N12(0,σɛ2I12)$. The covariance matrix of y is

$Var(y)=Var(jμ+XSα+XB(S)β+ɛ)=XS Var(α)XS′+XB(S)Var(β)XB(S)′+Var(ɛ)=σα2XSXS′+σβ2XB(S)XB(S)′+σɛ2I=Σ.$

There are three variance components; therefore, sub-models should be derived from the projection of y onto the vector space generated by the model matrix X as before. We fit the full model (6.2) first to identify the projection of y. The model matrix is X = ( j, XS, XB(S )). Hence, the projection of y is given by XXy. The model derived from XXy is used for the projection of XXy onto a vector subspace generated by the sub-model matrix. The model to be fitted first for μ is

$XX-y=jμ+ɛj.$

The projection from the model (6.4) is j jy and SSM = yj jy is obtained as 4.69. The model for α is obtained from the difference of the two projections XXy and j jy, which is

$(XX--jj-)y=(XX--jj-)XSα+ɛA=XAα+ɛA,$

where XA = (XXj j)XS, the projection from this is $XAXA-y$ and $SSA=y′XAXA-y$ is 15.06. The model for β is

$(XX--jj--XAXA-)y=(XX--jj-XAXA-)XB(S)β+ɛB=XBα+ɛB,$

where $XB=(XX--jj-XAXA-)XB(S)$, the projection from the model is $XBXB-y$ and $SSB=y′XBXB-y$ is given by 69.92. All of the sums of squares are expressed as the quadratic forms in y. The matrices associated with the quadratic forms are figured out from the fitted models. They are $QA=XAXA-,QB=XBXB-$, and QE = IXX. Hence, the expectations of the quadratic forms are given by

$E(SSA)=tr(QAΣ)=cAασα2+cAβσβ2+cAɛσɛ2,E(SSB)=tr(QBΣ)=cBασα2+cBβσβ2+cBɛσɛ2,E(SSE)=tr(QEΣ)=cEασα2+cEβσβ2+cEɛσɛ2.$

Letting S = (15.06, 69.92, 63.33)′, the expression of (5.2) is given by

$(15.0669.9263.33)=(cAαcAβcAɛcDαcDβcDɛcBαcBβcBɛcEαceβcEɛ) (σα2σβ2σɛ2).$

The solutions for the system of linear equations in σ2s are obtained from the normal equations. They are given by $σ^α2=0.63, σ^β2=2.59$, and $σ^α2=2.64$.

7. Discussion

This paper concerns the calculations of sums of squares due to random components in a two-stage nested design with which each classification has its own experimental unit. Consequently, a two-factor nested components-of-variance model with two types of random errors is proposed under the conditions.

Sums of squares in the analyses of variance are usually expressed in terms of quadratic forms and when each quadratic form is the squared distance of a projection. This idea came to the application of the projection to the calculation of sum of squares. Each projection is defined on the estimation space, spanned by the model matrix of the fitted model with the corresponding random vector. The discussion established appropriate model for the projection at each step.

This paper shows that reductions in sums of squares can be replaced by the squared distances of the projections and that the associated matrices with the quadratic forms are easily identified from the fitted models. These are quite different points of views from the traditional methods employed in ANOVA. However, the method can be used as a method for analyzing variance because it produces the same results as the analysis of variance method for the random effects model.

Acknowledgements

This research was supported by the Keimyung-Scholar Research Grant of Keimyung University in 2017.

TABLES

### Table 1

Coded purity data from balanced two-stage nested data

Suppliers123

123412341234
Batches1−2−2110−102−213
−1−304−240340−12
0−410−32−220221

Batch totals0−9−15−46−356026

Supplier totals−5414

References
1. Graybill, FA (1976). Theory and Application of the Linear Model. California: Wadsworth
2. Hartley, HO (1967). Expectations, variances and covariances of ANOVA mean squares by ‘synthesis’. Biometrics. 23, 105-114.
3. Henderson, CR (1953). Estimation of variance and covariance components. Biometrics. 9, 226-252.
4. Khan, AR, Saleem, SMA, and Mehdi, H (2017). Detection of edges using two-way nested design. International Journal of Advanced Computer Science and Applications. 8, 136-144.
5. Milliken, GA, and Johnson, DE (1984). Analysis of Messy Data Volume I: Designed Experiments. New York: Van Nostrand Reinhold
6. Montgomery, DC (1976). Design and Analysis of Experiments. New York: John Wiley & Sons
7. Searle, SR (1971). Linear Models. New York: John Wiley & Sons
8. Sharma, HL (2014). Nested balanced n-ary designs and their PB arrays. Journal of Reliability and Statistical Studies. 7, 29-36.