Neural Ordinary Differential Equations

Neural Ordinary Differential EquationsRicky T. Q. Chen*, Yulia Rubanova*, Jesse Bettencourt*, David DuvenaudUniversity of Toronto, Vector Institute AbstractWe introduce a new family of deep Neural network models. Instead of specifying adiscrete sequence of hidden layers, we parameterize the derivative of the hiddenstate using a Neural network. The output of the network is computed using a black-box Differential equation solver. These continuous-depth models have constantmemory cost, adapt their evaluation strategy to each input, and can explicitly tradenumerical precision for speed. We demonstrate these properties in continuous-depthresidual networks and continuous-time latent variable models.

an implicit method, it has better guarantees than explicit methods such as Runge-Kutta but requires solving a nonlinear optimization problem at every step. This setup makes direct backpropagation through the integrator difficult. We implement the adjoint sensitivity method in Python’s framework (Maclaurin et al., 2015).

Tags:

  Methods, Differential, Equations, Ordinary, Nonlinear, Neural, Neural ordinary differential equations

Information

Related search queries