1 Overview 2 The Gradient Descent Algorithm
AM 221: Advanced OptimizationSpring 2016Prof. Yaron SingerLecture 9 February 24th1 OverviewIn the previous lecture we reviewed results from multivariate calculus in preparation for our journeyinto convex optimization. In this lecture we present the Gradient Descent Algorithm for minimizinga convex function and analyze its convergence The Gradient Descent AlgorithmFrom the previous lecture, we know that in order to minimize a convex function, we need to finda stationary point. As we will see in this lecture as well as the upcoming ones, there are differentmethods and heuristics to find a stationary point. One possible approach is to start at an arbitrarypoint, and move along the Gradient at that point towards the next point, and repeat until (hopefully)converging to a stationary point.
Algorithm1GradientDescent 1: Guessx(0),setk 0 2: whilejjrf(x(k))jj do 3: x(k+1) = x(k) t krf(x(k)) 4: k k+ 1 5: endwhile 6: return x(k) f(x) x x(0) f(x 1) x(2)!f(z) x ...
Download 1 Overview 2 The Gradient Descent Algorithm
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: