A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis

Abstract

We present a penalized matrix decomposition (PMD), a new framework for computing a rank-K approximation for a matrix. We approximate the matrix X as circumflexX = sigma(k=1)(K) d(k)u(k)v(k)(T), where d(k), u(k), and v(k) minimize the squared Frobenius norm of X - circumflexX, subject to penalties on u(k) and v(k). This results in a regularized version of the singular value decomposition. Of particular interest is the use of L(1)-penalties on u(k) and v(k), which yields a decomposition of X using sparse vectors. We show that when the PMD is applied using an L(1)-penalty on v(k) but not on u(k), a method for sparse principal components results. In fact, this yields an efficient algorithm for the "SCoTLASS" proposal (Jolliffe and others 2003) for obtaining sparse principal components. This method is demonstrated on a publicly available gene expression data set. We also establish connections between the SCoTLASS method for sparse principal component analysis and the method of Zou and others (2006). In addition, we show that when the PMD is applied to a cross-products matrix, it results in a method for penalized canonical correlation analysis (CCA). We apply this penalized CCA method to simulated data and to a genomic data set consisting of gene expression and DNA copy number measurements on the same set of samples.

Fig. 1.

A graphical representation of the L₁- and L₂-constraints on u in the PMD(L₁, L₁) criterion. The constraints are as follows: ‖u‖₂² ≤ 1 and ‖u‖₁ ≤ c. Here, u is two-dimensional, and the grey lines indicate the coordinate axes u₁ and u₂. Left: the L₂-constraint is the solid circle. For both the L₁- and L₂-constraints to be active, c must be between 1 and . The constraints ‖u‖₁ = 1 and ‖u‖₁= are shown using dashed lines. Right: The L₂- and L₁-constraints on u are shown for some c between 1 and . Small circles indicate the points where both the L₁- and the L₂-constraints are active. The solid arcs indicate the solutions that occur when Δ₁ = 0 in Algorithm 3. The figure shows that in 2D, the points where both the L₁- and L₂-constraints are active do not have either u₁ or u₂ equal to 0.

Fig. 2.

Simulated CGH data. Top: results of PMD(L₁, FL); middle: results of PMD(L₁, L₁); bottom: generative model. PMD(L₁, FL) successfully identifies both the region of gain and the subset of samples for which that region is present.

Fig. 3.

Breast cancer gene expression data: a greater proportion of variance is explained when SPC is used to obtain the sparse principal components, rather than SPCA. Multiple SPC components were obtained as described in Algorithm 2.

Fig. 4.

The efficacy of PMD(L₁, L₁) is demonstrated using a simulation in which Xis generated from 2 sparse latent factors, called u₁and u₂, and Yis generated from 2 sparse latent factors, called v₁and v₂. The PMD(L₁, L₁) method identifies linear combinations of these sparse factors. Details of the simulation are given in Section A. of the Appendix.

Fig. 4.

The efficacy of PMD(L₁, L₁) is demonstrated using a simulation in which Xis generated from 2 sparse latent factors, called u₁and u₂, and Yis generated from 2 sparse latent factors, called v₁and v₂. The PMD(L₁, L₁) method identifies linear combinations of these sparse factors. Details of the simulation are given in Section A. of the Appendix.

Fig. 5.

PMD(L₁, FL) was performed for the breast cancer data set. Left: for each chromosome, the weights of vobtained using PMD(L₁, FL) are shown. All the vweights shown are positive, but the results would not be affected by flipping the signs of both vand u. On chromosome 2, vhas no nonzero elements. Right: for each chromosome, uand vwere computed on a training set consisting of 3/4 of the samples, and cor(Xu,Yv)is plotted, where Xand Yare the training (dashed) and test (solid) data.