Inference Compilation and Universal Probabilistic Programming

Probabilistic inference is hard. Compile inference for probabilistic programs written in Anglican to obtain a compilation artifact, speeding up probabilistic inference during runtime.

Compilation of probabilistic programs refers to training a neural network for constructing highly efficient proposal distributions for importance sampling based inference algorithms. The compilation artifact consists of an automatically-built program-specific neural architecture and trained weights. During the inference runtime, sequential importance sampling is run with proposal distributions obtained from the compilation artifact.

The neural network part of this scheme is written in Torch. The probabilistic programming part is written in Clojure as a library, extending Anglican. The communication between the two is facilitated by ZeroMQ.

To get started, please visit:

• Tutorial: A complete walkthrough
• Examples: Examples for reproducing experiments in the paper

For the code, please visit:

• Torch code: Torch-based neural network part
• Clojure code: Clojure-based library for the probabilistic programming part
• Leiningen template code: Ready-to-go Leiningen template for the Clojure part

For more details, please check out our paper.

@article{le2016inference,
  title = {Inference Compilation and Universal Probabilistic Programming},
  author = {Le, Tuan Anh and Baydin, Atilim Gunes and Wood, Frank},
  journal = {arXiv preprint arXiv:1610.09900},
  year = {2016}
}