Tuan Anh Le

black box variational inference (wip)

04 February 2017

notes on (Ranganath, Gerrish, & Blei, 2014).

summary

understanding: 7/10
code: https://github.com/stan-dev/stan/blob/develop/src/stan/variational/advi.hpp

a key paper for modern variational inference. the elbo is maximized using a score function gradient estimator. the main problem is high variance of gradient estimators. two variance reduction techniques are presented:

i don’t understand control variates very well.


references

  1. Ranganath, R., Gerrish, S., & Blei, D. (2014). Black Box Variational Inference. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics (pp. 814–822).
    @inproceedings{ranganath2014black,
      title = {Black Box Variational Inference},
      author = {Ranganath, Rajesh and Gerrish, Sean and Blei, David},
      booktitle = {Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics},
      pages = {814--822},
      year = {2014}
    }
    

[back]