Multivariate regression
Yoo and Cook (2008) previously provide theoretical foundation and inference procedure for the response dimension reduction in SDR context. In the seminal work, two types of response dimension reduction subspaces are newly defined without losing information on
Cook (2007) recently showed that a semi-parametric approach in SDR can outperform nonparametric methods. Following the idea, Yoo (2018) proposed two versions of semi-parametric response dimension reduction approaches, called “principal response reduction (PRR)” and “principal fitted response reduction (PFRR)” in the context of Yoo and Cook (2008). Yoo (2018) confirms that the two semi-parametric approaches have potential advantages in the response dimension reduction over Yoo and Cook (2008). Next, Yoo (2019) developed “unstructured PFRR (UPFRR)”, which do not assume the structure of the covariance matrix of the random error vectors in Yoo (2018) in the estimation. The advantage of Yoo (2019) is a possibility of equivariant or invariant full-rank transformation. Yoo (2019) also provides good guidelines to choose either PRR or PFRR. Therefore, the semi-parametric response reduction would be complete when including the unstructured fitted response reduction.
This paper provides insightful remarks on three semi-parametric approaches in order to clearly distinguish differences among the three approaches. In addition, theoretical results for the orthogonal transformation of the response variables in the response reduction are derived for the three semi-approaches that include Yoo and Cook (2008). Normally, the full-rank transformation of the response variables are incompatible to the response reduction subspaces, not like the case for the predictors. However, in the case of the orthogonal transformation, some similar results to the predictor case can hold.
The organization of the paper is as follows. Section 2 briefly introduces the short review on Yoo and Cook (2008) and the three semi-parametric response reduction approaches. The following section is devoted to two remarks on the three approaches. Section 4 is devoted to showing the results on the orthogonal transformation. Section 5 summarizes the work.
For notational convenience, we define that stands for a subspace spanned by the columns of
For a multivariate regression of
where
If equation (2.1) holds, then predictors
Next, it is assumed that there exists a
where
Yoo and Cook (2008) show that for
A semi-parametric response reduction approach starts with the following multivariate regression with assuming
where
One important assumption required for the response reduction is that is an invariant and reducing subspace of
Under model (2.3), Yoo (2018) shows that
The primary interest is then placed onto the estimation of
The PRR utilizes the marginal information on
where
: the
: the
: an
and
Yoo (2018) uses
Under model (2.4), the MLE of
Therefore, the MLE of
In model (2.3), we assume that
The difference between models (2.3) and (2.5) is the structure of
Yoo (2019) presents the relationship between
To utilize the information of predictors in the estimation of
Let
The response reduction though model (2.6) will be called “unstructured PFRR (UPFRR)”.
Under PRR and PFRR, recall that
The structural dimension
Under UPFRR,
The next proposition summarizes results of an orthogonal transformation of
.
Proposition 2(a) indicates that
Proposition 2(b) implies that
According to Proposition 2(c), the basis matrix of the response reduction for the orthogonal transformation is estimated by the estimate before the transformation pre-multiplied by the orthogonal matrix, so the same result is derived as the Yoo-Cook response reduction in Proposition 2(a).
SDR has been successful in high-dimensional data analysis when involving multi-dimensional responses; consequently, their dimension reduction can facilitate the data analysis and induce undiscovered scientific results. Following the notion of SDR, the response dimension reduction was founded in Yoo and Cook (2008) along with a proposed non-parametrical approach. Two semi-parametric approaches were recently developed in Yoo (2018) and showed that the latter has potential advantage in the estimation of response reduction subspace over the former. Yoo (2019) also completes the semi-parametric method by proposing an unstructured approach. In the paper, the three version of the semi-parametric approach are discussed theoretically and provide insightful remarks that are beneficial to usual statistical practitioners to employ the semi-proposed approach. The paper also presents the results on an orthogonal transformation of response variables for the seminal work of Yoo and Cook (2008) and the three semi-parametric approaches. It is shown that it is possible to avoid numerical instability in practice in the estimation of a basis of the response reduction subspace.
For Jae Keun Yoo, this work was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Korean Ministry of Education (NRF-2019R1F1A1050715).
(⇒) By construction of model (2.3), we have
Recall that
Proof of part (a): By the assumption, we have the following equivalences.
By pre-multiplying
Proof of part (b): By the assumption of PRR, PFRR, and UPFRR given in (2.3), (2.4), and (2.6), respectively, we have the following equivalences.
By pre-multiplying
Proof of part (c): We have that
By this relation, for PRR, we directly have that
For PFRR, we have the following likelihood function:
Therefore,
For UPFRR in the regression of
Therefore, the largest eigenvectors of
Parameters and their dimensions in PFRR and UPFRR
Γ | ∑ | Total | ||
---|---|---|---|---|
PFRR | ( |
|||
UPFRR | ( |
( |
PFRR = principal fitted response reduction; UPFRR = unstructured PFRR.