Variational Inference with Normalizing Flows - arXiv

1 Variational Inference with Normalizing Flows Danilo Jimenez Rezende DANILOR @ GOOGLE . COM. Shakir Mohamed SHAKIR @ GOOGLE . COM. Google DeepMind, London Abstract of many physical and chemical systems. Despite these suc- [ ] 14 Jun 2016. The choice of approximate posterior distribution cesses and ongoing advances, there are a number of disad- is one of the core problems in Variational infer- vantages of Variational methods that limit their power and ence. Most applications of Variational Inference hamper their wider adoption as a default method for statis- employ simple families of posterior approxima- tical Inference . It is one of these limitations, the choice of tions in order to allow for efficient Inference , fo- posterior approximation, that we address in this paper.

2 Cusing on mean-field or other simple structured Variational Inference requires that intractable posterior dis- approximations. This restriction has a signifi- tributions be approximated by a class of known probability cant impact on the quality of inferences made distributions, over which we search for the best approxima- using Variational methods. We introduce a new tion to the true posterior. The class of approximations used approach for specifying flexible, arbitrarily com- is often limited, , mean-field approximations, implying plex and scalable approximate posterior distribu- that no solution is ever able to resemble the true posterior tions.

3 Our approximations are distributions con- distribution. This is a widely raised objection to Variational structed through a Normalizing flow, whereby a methods, in that unlike other inferential methods such as simple initial density is transformed into a more MCMC, even in the asymptotic regime we are unable re- complex one by applying a sequence of invertible cover the true posterior distribution. transformations until a desired level of complex- ity is attained. We use this view of Normalizing There is much evidence that richer, more faithful posterior Flows to develop categories of finite and infinites- approximations do result in better performance.

4 For exam- imal Flows and provide a unified view of ap- ple, when compared to sigmoid belief networks that make proaches for constructing rich posterior approxi- use of mean-field approximations, deep auto-regressive mations. We demonstrate that the theoretical ad- networks use a posterior approximation with an auto- vantages of having posteriors that better match regressive dependency structure that provides a clear im- the true posterior, combined with the scalability provement in performance (Mnih & Gregor, 2014). There of amortized Variational approaches, provides a is also a large body of evidence that describes the detri- clear improvement in performance and applica- mental effect of limited posterior approximations.

5 Turner bility of Variational Inference . & Sahani (2011) provide an exposition of two commonly experienced problems. The first is the widely-observed 1. Introduction problem of under-estimation of the variance of the poste- rior distribution, which can result in poor predictions and There has been a great deal of renewed interest in varia- unreliable decisions based on the chosen posterior approx- tional Inference as a means of scaling probabilistic mod- imation. The second is that the limited capacity of the pos- eling to increasingly complex problems on increasingly terior approximation can also result in biases in the MAP. larger data sets.

6 Variational Inference now lies at the core of estimates of any model parameters (and this is the case , large-scale topic models of text (Hoffman et al., 2013), pro- in time-series models). vides the state-of-the-art in semi-supervised classification (Kingma et al., 2014), drives the models that currently pro- A number of proposals for rich posterior approximations duce the most realistic generative models of images (Gre- have been explored, typically based on structured mean- gor et al., 2014; 2015; Rezende et al., 2014; Kingma & field approximations that incorporate some basic form of Welling, 2014), and are a default tool for the understanding dependency within the approximate posterior.

7 Another po- tentially powerful alternative would be to specify the ap- Proceedings of the 32 nd International Conference on Machine proximate posterior as a mixture model, such as those de- Learning, Lille, France, 2015. JMLR: W&CP volume 37. Copy- veloped by Jaakkola & Jordan (1998); Jordan et al. (1999);. right 2015 by the author(s). Gershman et al. (2012). But the mixture approach limits Variational Inference with Normalizing Flows the potential scalability of Variational Inference since it re- we will focus on Inference over the latent variables only. quires evaluation of the log-likelihood and its gradients for This bound is often referred to as the negative free energy each mixture component per parameter update, which is F or as the evidence lower bound (ELBO).

8 It consists of typically computationally expensive. two terms: the first is the KL divergence between the ap- proximate posterior and the prior distribution (which acts This paper presents a new approach for specifying approx- as a regularizer), and the second is a reconstruction error. imate posterior distributions for Variational Inference . We This bound (3) provides a unified objective function for op- begin by reviewing the current best practice for Inference timization of both the parameters and of the model and in general directed graphical models, based on amortized Variational approximation, respectively. Variational Inference and efficient Monte Carlo gradient es- timation, in section 2.

9 We then make the following contri- Current best practice in Variational Inference performs butions: this optimization using mini-batches and stochastic gra- We propose the specification of approximate poste- dient descent, which is what allows Variational infer- rior distributions using Normalizing Flows , a tool for ence to be scaled to problems with very large data constructing complex distributions by transforming a sets. There are two problems that must be addressed probability density through a series of invertible map- to successfully use the Variational approach: 1) effi- pings (sect. 3). Inference with Normalizing Flows pro- cient computation of the derivatives of the expected log- vides a tighter, modified Variational lower bound with likelihood Eq (z) [log p (x|z)], and 2) choosing the additional terms that only add terms with linear time richest, computationally-feasible approximate posterior complexity (sect 4).

10 Distribution q( ). The second problem is the focus of this We show that Normalizing Flows admit infinitesimal paper. To address the first problem, we make use of two Flows that allow us to specify a class of posterior ap- tools: Monte Carlo gradient estimation and Inference net- proximations that in the asymptotic regime is able to works, which when used together is what we refer to as recover the true posterior distribution, overcoming one amortized Variational Inference . oft-quoted limitation of Variational Inference . Stochastic Backpropagation We present a unified view of related approaches for improved posterior approximation as the application The bulk of research in Variational Inference over the years of special types of Normalizing Flows (sect 5).