Tuan Anh Le

variational inference with normalizing flows

10 January 2017

notes on (Rezende & Mohamed, 2015).


couldn’t find code… closest things was https://github.com/casperkaae/parmesan/issues/22

setup: probabilistic model of latents , observes , parametrized by model parameters . interested in .

the main problem addressed by this paper is choosing the family of variational approximations so that the true posterior . the usual way of doing this is to fix a parametrized family , most commonly mean-field. the authors argue that this hardly covers the true posterior.

instead, is induced by successive transformations on a sample from some initial distribution . the density of the s thus has jacobian terms (which must be cheap to evaluate). they derive the ELBO in this setting (eqn 15).


  1. Rezende, D., & Mohamed, S. (2015). Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15) (pp. 1530–1538).
      title = {Variational Inference with Normalizing Flows},
      author = {Rezende, Danilo and Mohamed, Shakir},
      booktitle = {Proceedings of the 32nd International Conference on Machine Learning (ICML-15)},
      pages = {1530--1538},
      year = {2015},
      link = {http://jmlr.org/proceedings/papers/v37/rezende15.pdf}