Transcription of 1 Overview 2 The Gradient Descent Algorithm
{{id}} {{{paragraph}}}
AM 221: Advanced OptimizationSpring 2016 Prof. Yaron SingerLecture 9 February 24th1 OverviewIn the previous lecture we reviewed results from multivariate calculus in preparation for our journeyinto convex optimization. In this lecture we present the Gradient Descent Algorithm for minimizinga convex function and analyze its convergence The Gradient Descent AlgorithmFrom the previous lecture, we know that in order to minimize a convex function, we need to finda stationary point. As we will see in this lecture as well as the upcoming ones, there are differentmethods and heuristics to find a stationary point. One possible approach is to start at an arbitrarypoint, and move along the Gradient at that point towards the next point, and repeat until (hopefully)converging to a stationary point.
Algorithm1GradientDescent 1: Guessx(0),setk 0 2: whilejjrf(x(k))jj do 3: x(k+1) = x(k) t krf(x(k)) 4: k k+ 1 5: endwhile 6: return x(k) f(x) x x(0) f(x 1) x(2)!f(z) x ...
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}