METHODS FOR NON-LINEAR LEAST SQUARES PROBLEMS

← Back to document page

IMMMETHODS FORNON-LINEAR LEASTSQUARES PROBLEMS2nd Edition, April 2004K. Madsen, Nielsen, O. TingleffInformatics and Mathematical ModellingTechnical University of DenmarkCONTENTS1. INTRODUCTION The Steepest Descent method ............................... Newton s Method ........................................ .. Line Search . . . ........................................ .... Trust Region and Damped METHODS ........................113. The Gauss Newton Method . . . ............................ The Levenberg Marquardt Method ......................... Powell s Dog Leg Method . . ............................... A Hybrid Method: L M and Quasi Newton ................. A Secant Version of the L M Method . ..................... A Secant Version of the Dog Leg Method ................... Final INTRODUCTION ANDDEFINITIONSIn this booklet we consider the following problem,Definition LEAST SQUARES ProblemFindx , a local minimizer for1)F(x)=12mXi=1(fi(x))2;wherefi:IRn7!

72.DESCENT METHODS Definition 2.6. Descent direction. h is a descent direction for Fat x if h>F0(x) <0: If no such hexists, then F0(x)=0, showing that in this case xis stationary. Otherwise, we have to choose fi, ie how far we should go from x in the direction given by hd, so that we get a decrease in the value of the objective function.

  Tesla, Square, Least squares