From: jpf@hydra.cfm.brown.edu (Jim P. Ferry)
Newsgroups: sci.math
Subject: Re: Karhunen-Loeve transformation
Date: 27 Mar 1998 20:32:14 GMT

The Karhunen-Loeve procedure has many aliases:  Proper Orthogonal
Decomposition, Principal Component Analysis, the Singular Value
Decomposition, analysis by Empirical Eigenfunctions.  These different
names connote different things, such as whether we are working in R^n
or some Hilbert space, but the ideas are essentially the same.

I think of K-L as a slimmed-down version of SVD.  The SVD of a matrix A
(of rank r) is

         H
A = U D V,

where U and V are unitary, and D is a matrix that is zero outside its
principal r x r submatrix, this submatrix being diagonal with non-
increasing, positive entries (these are the non-zero singular values
of A).  Call this submatrix L.

The K-L decompostion is

       1/2  H
A = S L    T,

where S and T are the first r columns of U and V, respectively.

K-L is traditionally defined in terms of the correlation matrices

         H        H                  H          H
K_S = A A  = S L S      or    K_T = A  A = T L T,

where the decompostions given are essentially eigensystem decompostions,
except that the zero eigenspaces are left out.

If you want to apply this to a Hilbert space other than R^n, you just
need to change your notion of what inner-product (i.e., matrix
multiplication) you're using.

In my doctoral thesis I used K-L as follows.  A is a matrix whose
columns are snapshots of a flow field at different times.  I formed the
correlation matrix K_T, computed T and L, then S = A T L^(-1/2).  The
columns of S are the so-called "eigenpictures" (in this context).  I like
to think of them as the principal directions of the ellipsoid of best fit
to the attractor of my flow.

-Jim Ferry