SAGA: A Fast Incremental Gradient Method With Support for ...

saga : A fast Incremental Gradient Method WithSupport for Non-Strongly Convex CompositeObjectivesAaron DefazioAmbiata Australian National University, CanberraFrancis BachINRIA - Sierra Project-Team Ecole Normale Sup erieure, Paris, FranceSimon Lacoste-JulienINRIA - Sierra Project-Team Ecole Normale Sup erieure, Paris, FranceAbstractIn this work we introduce a new optimisation Method called saga in the spirit ofSAG, SDCA, MISO and SVRG, a set of recently proposed Incremental gradientalgorithms with fast linear convergence rates. saga improves on the theory be-hind SAG and SVRG, with better theoretical convergence rates, and has supportfor composite objectives where a proximal operator is used on the regulariser.

SAGA is preferred over SVRG both theoretically and in practice. For neural networks, where no theory is available for either method, the storage of gradients is generally more expensive than the

Loading..

Tags:

  With, Methods, Support, Fast, Saga, Derating, Incremental, A fast incremental gradient method with support

Information

Related search queries