Auxiliary particle filter

Comparing with the empirical filtering density which has ${\displaystyle {\widehat {f}}(\alpha _{t+1}|Y_{t+1})\propto f(y_{t+1}|\alpha _{t+1})\sum _{j=1}^{M}f(\alpha _{t+1}|\alpha _{t}^{j})\pi _{t}^{j}}$,

we now define ${\displaystyle {\widehat {f}}(\alpha _{t+1},k|Y_{t+1})\propto f(y_{t+1}|\alpha _{t+1})f(\alpha _{t+1}|\alpha _{t}^{k})\pi ^{k}}$, where ${\displaystyle k=1,...,M}$.

Being aware that ${\displaystyle {\widehat {f}}(\alpha _{t+1}|Y_{t+1})}$ is formed by the summation of ${\displaystyle M}$ particles, the auxiliary variable ${\displaystyle k}$ represents one specific particle. With the aid of ${\displaystyle k}$, we can form a set of samples which has the distribution ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$. Then, we draw from these sample set ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$ instead of directly from ${\displaystyle {\widehat {f}}(\alpha _{t+1}|Y_{t+1})}$. In other words, the samples are drawn from ${\displaystyle {\widehat {f}}(\alpha _{t+1}|Y_{t+1})}$ with different probability. The samples are ultimately utilized to approximate ${\displaystyle f(\alpha _{t+1}|Y_{t+1})}$.

Take the SIR method for example:

• The particle filters draw ${\displaystyle R}$ samples from ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$.
• Assign each samples with the weight ${\displaystyle \pi _{j}={\frac {\omega _{j}}{\sum _{i=1}^{R}\omega _{i}}},\omega _{j}={\frac {f(y_{t+1}|\alpha _{t+1}^{j})f(\alpha _{t+1}^{j}|\alpha _{t}^{k})}{g(\alpha _{t+1}^{j},k^{j}|Y_{t+1})}}}$.
• By controlling ${\displaystyle y_{t+1}}$ and ${\displaystyle \alpha _{t}^{k}}$, the weights are adjusted to be even.
• Similarly, the ${\displaystyle R}$ particles are resampled to ${\displaystyle M}$ particles with the weight ${\displaystyle \pi _{j}}$.

The original particle filters draw samples from the prior density, while the auxiliary filters draw from the joint distribution of the prior density and the likelihood. In other words, the auxiliary particle filters avoid the circumstance which the particles are generated in the regions with low likelihood. As a result, the samples can approximate ${\displaystyle f(\alpha _{t+1}|Y_{t+1})}$ more precisely.

The selection of the auxiliary variable affects ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$ and controls the distribution of the samples. A possible selection of ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$ can be:
${\displaystyle g(\alpha _{t+1},k|Y_{t+1})\propto f(y_{t+1}|\mu _{t+1}^{k})f(\alpha _{t+1}|\alpha _{t}^{k})\pi ^{k}}$, where ${\displaystyle k=1,...,M}$ and ${\displaystyle \mu _{t+1}^{k}}$ is the mean.

We sample from ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$ to approximate ${\displaystyle f(\alpha _{t+1}|Y_{t+1})}$ by the following procedure:

• First, we assign probabilities to the indexes of ${\displaystyle f(\alpha _{t+1}|\alpha _{t}^{k})}$. We named these probabilities as the first-stage weights ${\displaystyle \lambda _{k}}$, which are proportional to ${\displaystyle g(k|Y_{t+1})\propto \pi ^{k}f(y_{t+1}|\mu _{t+1}^{k})}$.
• Then, we draw ${\displaystyle R}$ samples from ${\displaystyle f(\alpha _{t+1}|\alpha _{t}^{k})}$ with the weighted indexes. By doing so, we are actually drawing the samples from ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$.
• Moreover, we reassign the second-stage weights ${\displaystyle \pi _{j}={\frac {\omega _{j}}{\sum _{i=1}^{R}\omega _{i}}}}$ as the probabilities of the ${\displaystyle R}$ samples, where ${\displaystyle \omega _{j}={\frac {f(y_{t+1}|\alpha _{t+1}^{j})}{f(y_{t+1}|\mu _{t+1}^{j})}}}$. The weights are aim to compensate the effect of ${\displaystyle \mu _{t+1}^{k}}$.

• Finally, the ${\displaystyle R}$ particles are resampled to ${\displaystyle M}$ particles with the weights ${\displaystyle \pi _{j}}$.

Following the procedure, we draw the ${\displaystyle R}$ samples from ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$. Since ${\displaystyle g(\alpha _{t+1},k|Y_{t+1})}$ is closely related to the mean ${\displaystyle \mu _{t+1}^{k}}$, it has high conditional likelihood. As a result, the sampling procedure is more efficient and the value ${\displaystyle R}$ can be reduced.

Assume that the filtered posterior is described by the following M weighted samples:

${\displaystyle p(x_{t}|z_{1:t})\approx \sum _{i=1}^{M}\omega _{t}^{(i)}\delta \left(x_{t}-x_{t}^{(i)}\right).}$

Then, each step in the algorithm consists of first drawing a sample of the particle index ${\displaystyle k}$ which will be propagated from ${\displaystyle t-1}$ into the new step ${\displaystyle t}$. These indexes are auxiliary variables only used as an intermediary step, hence the name of the algorithm. The indexes are drawn according to the likelihood of some reference point ${\displaystyle \mu _{t}^{(i)}}$ which in some way is related to the transition model ${\displaystyle x_{t}|x_{t-1}}$ (for example, the mean, a sample, etc.):

${\displaystyle k^{(i)}\sim P(i=k|z_{t})\propto \omega _{t}^{(i)}p(z_{t}|\mu _{t}^{(i)})}$

This is repeated for ${\displaystyle i=1,2,\dots ,M}$, and using these indexes we can now draw the conditional samples:

${\displaystyle x_{t}^{(i)}\sim p(x|x_{t-1}^{k^{(i)}}).}$

Finally, the weights are updated to account for the mismatch between the likelihood at the actual sample and the predicted point ${\displaystyle \mu _{t}^{k^{(i)}}}$:

${\displaystyle \omega _{t}^{(i)}\propto {\frac {p(z_{t}|x_{t}^{(i)})}{p(z_{t}|\mu _{t}^{k^{(i)}})}}.}$

• Pitt, M.K.; Shephard, N. (1999). "Filtering Via Simulation: Auxiliary Particle Filters". Journal of the American Statistical Association. 94 (446). American Statistical Association: 590–591. doi:10.2307/2670179. JSTOR 2670179. Archived from the original on 2007-10-16. Retrieved 2008-05-06.