Files
Abstract
Many multivariate statistical methods rely on matrix decompositions. An archetypal example is canonical correlation analysis (CCA). In the traditional large sample setting, CCA admits an analytical solution: the best rank-$k$ approximation of the sample version of a matrix. In high dimension, low sample size settings, it is not possible to calculate the analytical solution because it requires the inverse of a matrix for which the sample version is singular. In that situation, additional assumptions or regularization are necessary. Matrix decompositions can also be utilized in statistical methods that do not inherently rely on them. For a generalized linear model (GLM) with an image as the covariate, the corresponding parameter is a matrix or higher-order array called a tensor. In that case, we may exploit low rank matrix or tensor decompositions as a means to reduce the massive number of parameters to estimate to a feasible level. In this dissertation, we study two regularization techniques for statistical methods utilizing matrix/tensor decompositions, with emphasis on their applications in high dimensional CCA and GLMs with matrix- or tensor-valued parameters. One technique is sparse regularization that results in variable selection. In high dimensional problems, variable selection can substantially improve the interpretability of the solution. The other technique is a penalty on the nuclear norm, which amounts to soft thresholding (i.e., shrinking) the singular values. For low rank approximation problems, shrinking the singular values can be an effective alternative to finding a fixed rank-$k$ approximation. For GLMs with matrix- or tensor-valued parameters, we develop both fixed-rank and shrinkage versions of an orthogonal tensor regression model, which we intend for analyzing medical imaging data such as fMRI. For high dimensional CCA, we develop a sparse CCA method that achieves variable selection by penalizing the elements of the canonical vectors. We study the variable selection accuracy under different choices of the penalty function. We also develop a more general method of finding a sparse, low rank matrix approximation based on shrinkage, which we show aims to select the same variables as certain sparse CCA methods under some assumptions.