Tuan Anh Le

reparameterization trick on joint random variables (speculative…)

10 August 2017

goal: estimate the gradient where the joint random variable (which is parameterized by ) can be factorized (need to wait until i learn more about product measures :-( …) as

hence (?), .

let’s assume that we can find a random variables and a mappings such that \begin{align} g_\theta(A) &= X_\theta \\ h_\theta(B, x) &= (Y \given x)_\theta. \end{align}

hence and we can write

which can be approximated using a monte carlo estimator \begin{align} \hat I_N &= \frac{1}{N} \sum_{n = 1}^N \frac{\partial}{\partial \theta} \left[f_\theta(g_\theta(a_n), h_\theta(b_n, g_\theta(a_n)))\right] \end{align} where and are independent samples of random variables and respectively.