21 September 2017
Consider a linear Gaussian state space model:
\begin{align}
p(x_1) &= \mathrm{Normal}(x_1; A, B) \\
p(x_t \given x_{t - 1}) &= \mathrm{Normal}(x_t; Cx_{t - 1} + D, E) && t = 2, \dotsc, T \\
p(y_t \given x_t) &= \mathrm{Normal}(y_t; Fx_t + G, H) && t = 1, \dotsc, T.
\end{align}
We are interested in finding the optimal proposal for sequential Monte Carlo (SMC) in the sense of minimizing the variance of intermediate weights.
For \(t = 1\):
\begin{align}
q_1^{\text{opt}}(x_1 \given y_1) &= \frac{p(x_1, y_1)}{\int p(x_1, y_1) \,\mathrm dx_1} \\
&= \mathrm{Normal}(x_1; A + \Gamma_1(y_1 - FA - G), B - \Gamma_1 FB) \\
&= \mathrm{Normal}(x_1; \Gamma_1 y_1 + (A - \Gamma_1(FA + G)), B - \Gamma_1FB) \\
\Gamma_1 &= BF^T(H + FBF^T)^{-1},
\end{align}
where we have used the Gaussian identity in equation 6 in Michael Osborne’s note with the substitutions (quantities in equation 6 in blue)
\begin{align}
\color{blue}{x} &\leftarrow x_1, \\
\color{blue}{\mu} &\leftarrow A, \\
\color{blue}{A} &\leftarrow B, \\
\color{blue}{y} &\leftarrow y_1, \\
\color{blue}{M} &\leftarrow F, \\
\color{blue}{c} &\leftarrow G, \\
\color{blue}{L} &\leftarrow H.
\end{align}
For \(t = 2, \dotsc, T\):
\begin{align}
q_t^{\text{opt}}(x_t \given x_{t - 1}, y_t) &= \frac{1}{Z_t} \cdot \frac{p(x_{1:t}, y_{1:t})}{p(x_{1:t - 1}, y_{1:t - 1})} \\
&= \frac{1}{Z_t} \cdot \frac{p(x_{1:t - 1}, y_{1:t - 1}) p(x_t \given x_{t - 1}) p(y_t \given x_t)}{p(x_{1:t - 1}, y_{1:t - 1})} \\
&= \frac{1}{Z_t} \cdot p(x_t \given x_{t - 1}) p(y_t \given x_t) && (Z_t = \int p(x_t \given x_{t - 1}) p(y_t \given x_t) \,\mathrm dx_t)\\
&= \mathrm{Normal}(x_t; Cx_{t - 1} + D + \Gamma_t(y_t - F(Cx_{t - 1} + D) - G), E - \Gamma_t FE) \\
&= \mathrm{Normal}(x_t; (C - \Gamma_t F C) x_{t - 1} + \Gamma_t y_t + (D - \Gamma_t (FD + G)), E - \Gamma_t F E) \\
\Gamma_t &= EF^T(H + FEF^T)^{-1}
\end{align}
where, again, we use the Gaussian identity in equation 6 from Michael Osborne’s note with the substitutions:
\begin{align}
\color{blue}{x} &\leftarrow x_t, \\
\color{blue}{\mu} &\leftarrow (Cx_{t - 1} + D), \\
\color{blue}{A} &\leftarrow E, \\
\color{blue}{y} &\leftarrow y_t, \\
\color{blue}{M} &\leftarrow F, \\
\color{blue}{c} &\leftarrow G, \\
\color{blue}{L} &\leftarrow H.
\end{align}
[back]