From: spellucci@mathematik.tu-darmstadt.de (Peter Spellucci)
Newsgroups: sci.math.num-analysis
Subject: Re: Interative least square with 2 variables?
Date: 24 Nov 1998 10:20:32 GMT

In article <73c4o3$gi5$1@bcrkh13.ca.nortel.com>,
 klu@bnr.ca (Kevin Lu) writes

snip 
|> Here is the question.
|> 
|> find a and b to fit z=a*x+b*y by iterative least square mothed. 
|> x, y, and z are the 1000 samples taken from real data.
let the number of data points be N (N=1000) in your case).
number your data points in whatever sense you want:
(x_i, y_i, z_i) i=1,....N
let the vector b be 

    b=(z_1,.....,z_N)^T   

let the matrix A be Nx2 

     A =  [ x_1,  y_1 ]
          [ x_2,  y_2 ]
          -------------
          [x_N,    y_N]

set with (.') transposition
                         [ \sum_{i=1}^N x_i^2 , \sum_{i=1}^N x_i*y_i ]
     B = A'A  (2x2)    = [ \sum_{i=1}^N x_i*y_i , \sum_{i=1}^N y_i^2 ]


     c = A'b           = [ \sum_{i=1}^N x_i*z_i, \sum_{i=1}^N y_i*z_i ]
solve the 2x2 system 

     B [alpha] = c
       [ beta]

then [alpha,beta]' give the best least squares solution  

    \sum_{i=1}^N ( z_i - alpha*x_i -beta*y_i)^2 = min
I don't see why you want to use an iterative solver in this case
(the appropriate would be LSQR from netlib, but this one woud be 
useful only if the number of your unknown parameters would be
also large). 
I propose here using the normal equations, since there is only a very remote
danger of roundoff blowup. otherwise, you could use a linear least 
squares solver for minimizing 
 ||Ax-b||_2^2, with x=[alpha,beta]'
e.g. using Householder - QR- decomposition or the svd .
I think the textbook of forsythe&malcolm&moler "computer methods for 
mathematical computations" (prentice hall)has a fairly readable account 
of linear least squares.
(If you want to add a constant term, simply add a column of 1's in A and a
further variable gamma and extend the definiton of B and c appropriately. 
hope this helps
peter