From: randyp@visionplace.com (Randy Poe) Subject: Re: Leveberg-Marquardt algorithm Date: Tue, 20 Feb 2001 17:30:09 GMT Newsgroups: sci.math Summary: How is the Leveberg-Marquardt algorithm used to find minima? On Tue, 20 Feb 2001 16:26:53 GMT, dtd@world.std.com (Don Davis) wrote: >In article <96ttfu$7tb$1@venus.telepac.pt>, "Luis" >wrote: > >> I need information about the Levenberg-Marquardt algorithm >> to solvea non-linear equation system. Does anyone know >> some site where I can I find this information? > >yes, the google search engine has 5400 hits >on this topic, and altavista has 2100. Basic idea: 1. Newton method (use 2nd derivative, or Hessian matrix, to find a descent direction) converges very quickly when it converges. But it does converge if you are far from a minimum. The direction chosen can sometimes point uphill. 2. Steepest descent (use the gradient to find a downhill direction) is guaranteed to go downhill. But convergence is very slow. For many functions, the steepest descent is almost perpendicular to the direction of the minimum you're trying to reach. (To see this, draw long elliptical contours, which represent a function only slowly dependent on one of the variables). 3. Levenberg-Marquardt gives a heuristic for choosing directions between these two vectors. As you appear to be getting closer to a minimum, it weighs more and more toward the Newton direction. But if things appear to be getting worse, it starts leaning toward steepest descent. I'd try the SIAM journal (Society of Industrial and Applied Mathematics) and maybe some old operations research journals as well. I think I saw this method (and implemented it) first around 1980. There are lots of better ideas in choosing good descent directions since then. Are you sure this is the method you want? - Randy ============================================================================== From: student@maths.edu Subject: Re: Leveberg-Marquardt algorithm Date: 20 Feb 2001 12:26:01 -0800 Newsgroups: sci.math There is an excellent discussion of the nonlinear least squares optimization problem and the application of LM at this URL: http://www-fp.mcs.anl.gov/otc/Guide/OptWeb/continuous/unconstrained/nonlinearls/index.html For a very readable discussion of nonliner data modeling and LM, see section 15.5 of Numerical Recipes: http://lib-www.lanl.gov/numerical/bookcpdf/c15-5.pdf randyp@visionplace.com wrote: [quote of previous message deleted --djr]