10 August 2017
goal: estimate the gradient \(\frac{\partial}{\partial \theta} \E[f_\theta((X, Y)_\theta)]\) where the joint random variable \((X, Y)_\theta\) (which is parameterized by \(\theta\)) can be factorized (need to wait until i learn more about product measures :-( …) as
hence (?), \((X, Y)_\theta = (X_\theta, (Y \given X_\theta)_\theta)\).
let’s assume that we can find a random variables \(A, B\) and a mappings \(g_\theta, h_\theta\) such that
\begin{align}
g_\theta(A) &= X_\theta \\
h_\theta(B, x) &= (Y \given x)_\theta.
\end{align}
hence \((X, Y)_\theta = (g_\theta(A), h_\theta(B, g_\theta(A)))\) and we can write
\[\frac{\partial}{\partial \theta} \E[f_\theta((X, Y)_\theta)] = \frac{\partial}{\partial \theta} \E\left[f_{\theta}(g_\theta(A), h_\theta(B, g_\theta(A)))\right] = \E\left[\frac{\partial}{\partial \theta} f_{\theta}(g_\theta(A), h_\theta(B, g_\theta(A)))\right]\]which can be approximated using a monte carlo estimator \begin{align} \hat I_N &= \frac{1}{N} \sum_{n = 1}^N \frac{\partial}{\partial \theta} \left[f_\theta(g_\theta(a_n), h_\theta(b_n, g_\theta(a_n)))\right] \end{align} where \(a_n\) and \(b_n\) are independent samples of random variables \(A\) and \(B\) respectively.
[back]