From: rusin@vesuvius.math.niu.edu (Dave Rusin) Subject: Re: matrix concept Date: 22 Feb 1999 02:32:08 GMT Newsgroups: sci.math Keywords: What is the Singular Value Decomposition Jian Wang wrote: >What is the singular value of a matrix, such as [1 2;3 4] ? I'm guessing you're asking about the _singular value decomposition_ (SVD) of a matrix. This is one of the helpful factorizations of a matrix, that is a presentation of a matrix A as a product M1 * M2 * M3 * ... where each M_i is supposed to be of a certain type. (The Jordan Canonical Form is another familiar factorization, for example). The SVD is a presentation A = U D V' where U and V are orthogonal and D is diagonal (of the same dimensions as A) with, usually, non-negative decreasing diagonal terms. For a real matrix A one can show that such U, V, and D exist; except in degenerate cases (repeated singular values) they are unique. These terms are known as the 'singular values' of A. In the special case that A is symmetric positive definite, U=V and the SVD is just the diagonalization of A, so the singular values are the same as the eigenvalues. In the general case, the terms in the SVD may be derived from the diagonalization of AA' or A'A. For the matrix A = [1,2; 3,4] the singular values are about 5.46498, .3659662 and the matrices U, V are approximately [-.4045535848 -.9145142957 ] [-.5760484368 .8174155605 ] [ ], [ ] [-.9145142957 .4045535848 ] [-.8174155605 -.5760484368 ] The singular values may be interpreted thus: if we view A as representing a linear transformation on some Euclidean space, then it carries the unit ball in that space to an ellipsoid in another Euclidean space. The singular values are the lengths of the semi-axes of that ellipsoid. (The matrices U and V carry information about the locations of those axes and the vectors in the first space which map into them.) Arguably the most important applications of the SVD are those which enable the selection of "dominant terms" in a linear map or other matrix problems; for example this can be applied to the problem of finding optimal low-dimensional representations of high-dimensional datasets. For example, see 98/multidim.scal or several other topics on pages 15-XX and 62-XX of welcome.html (Incidentally, a SVD decomposition was used to help derive good coordinates for the "bubbles" which appear on those pages!) dave ============================================================================== [Remark: uniqueness of U and V is only up to signs of the columns --djr