Stochastic Gradient Descent Tricks

← Back to document page

proposed Lasso algorithm represents each weight as the di erence of two positive variables. Applying the stochastic gradient rule to these variables and enforcing their positivity leads to sparser solutions. 2.3 The Convergence of Stochastic Gradient Descent The convergence of stochastic gradient descent has been studied extensively

  Stochastic