Draft:Doob's h-transform

Homogeneous Markov processes

Let ${\displaystyle X}$ be a homogeneous Markov process with state space ${\displaystyle E}$, which is often assumed to be a locally compact Polish space to guarantee the existence of regular conditional expectations, and denote its transition semigroup by ${\displaystyle (P_{t})_{t\geq 0}}$. Let ${\displaystyle h:E\to [0,\infty )}$ be an excessive function, that is,^[1]$${\displaystyle {\begin{aligned}\forall t\geq 0:&\int _{E}h(y)P_{t}(x,\mathrm {d} y)\leq h(x),\\\lim _{t\to 0}&\int _{E}h(y)P_{t}(x,\mathrm {d} y)=h(x).\end{aligned}}}$$Then the ${\displaystyle h}$-transform ${\displaystyle X^{h}}$ is defined as the Markov process with transition semigroup ${\displaystyle (P_{t}^{h})_{t\geq 0}}$ defined by$${\displaystyle P_{t}^{h}(x,A):={\frac {1}{h(x)}}\int _{A}h(y)P_{t}(x,\mathrm {d} y),\quad \forall t\geq 0,x\in E,A\in {\mathcal {B}}(E).}$$This definition implies that ${\displaystyle P_{t}^{h}(x,E)\leq 1}$, which means that the process need not staying inside its state space for all times. For such processes, one usually defines an additional cemetery state ${\displaystyle \partial \notin E}$ that the process enters with probability ${\displaystyle 1-P_{t}(x,E)}$ for all ${\displaystyle x\in E}$ and that it never leaves, i.e., ${\displaystyle P_{t}(\partial ,\{\partial \})=1}$. In other words, the process is allowed to be killed at some finite time. On the extended state space ${\displaystyle E_{\partial }=E\cup \{\partial \}}$, the ${\displaystyle P_{t}^{h}(x,\cdot )}$ with the previous definitions are probability measures for all ${\displaystyle x\in E_{\partial }}$ by construction.

Inhomogeneous Markov processes

Let ${\displaystyle X}$ be an inhomogeneous Markov process with values in ${\displaystyle \mathbb {R} ^{d}}$, inhomogeneous transition probabilities ${\displaystyle (P_{t,s})_{t\geq s\geq 0}}$ and let ${\displaystyle h:[0,\infty )\times \mathbb {R} ^{d}\to (0,\infty )}$ be space-time regular, that is, for ${\displaystyle s>0}$, ${\displaystyle t\geq 0}$ and ${\displaystyle x\in \mathbb {R} ^{d}}$^[2]$${\displaystyle h(t,x)=\int _{E}h(t+s,y)P_{t+s,s}(x,\mathrm {d} y).}$$Then the ${\displaystyle h}$-transform ${\displaystyle X^{h}}$ is defined as the (inhomogeneous) Markov process with transition probabilities ${\displaystyle (P_{t,s}^{h})_{t\geq s\geq 0}}$ defined by$${\displaystyle P_{t+s,s}^{h}(x,A):={\frac {1}{h(t,x)}}\int _{A}h(t+s,y)P_{t+s,s}(x,\mathrm {d} y),\quad \forall x\in E,A\in {\mathcal {B}}(E).}$$

Probabilities

The transition semigroup ${\displaystyle (P_{t}^{h})_{t\geq 0}}$ generates a family of probability measures ${\displaystyle (\mathbb {P} _{h}^{x})_{x\in E}}$ on the canonical space of càdlàg paths. For any stopping time ${\displaystyle \tau }$, it holds for ${\displaystyle A\in {\mathcal {F}}_{\tau }}$ and ${\displaystyle x\in E\backslash \{h=0{\text{ or }}h=\infty \}}$^[1]$${\displaystyle \mathbb {P} _{h}^{x}(A,\zeta >\tau )={\frac {1}{h(x)}}\mathbb {E} ^{x}[1_{A}h(X_{\tau })].}$$For a family ${\displaystyle (u_{s})_{s\in S}}$ of excessive functions on ${\displaystyle E}$ over an index set ${\displaystyle S}$ (and measurable w.r.t. the product ${\displaystyle \sigma }$-algebra on ${\displaystyle E\times S}$), if ${\displaystyle h}$ is of the form$${\displaystyle h(x)=\int _{S}u_{s}(x)\mathrm {d} \beta (s)}$$with some finite measure ${\displaystyle \beta }$ on ${\displaystyle S}$, then for ${\displaystyle A\in {\mathcal {F}}}$ the probability simplifies to$${\displaystyle \mathbb {P} _{h}^{x}(A)={\frac {1}{h(x)}}\int _{S}u_{s}(x)\mathbb {P} _{u_{s}}^{x}(A)\mathrm {d} \beta (s).}$$

Killing

If ${\displaystyle \tau }$ is a co-optional time, that is, if for any ${\displaystyle t\geq 0}$ it holds ${\displaystyle \tau \circ \theta _{t}=(\tau -t)^{+}}$, where ${\displaystyle (\theta _{s})_{s\geq 0}}$ is the shift-semigroup, then one can define$${\displaystyle {\begin{aligned}&h(x):=\mathbb {P} ^{x}(\tau >0),\quad x\in E\\&X_{t}^{\tau }:={\begin{cases}X_{t}&,t<\tau \\\partial &,t\geq \tau \end{cases}}\end{aligned}}}$$for which it holds that ${\displaystyle X^{\tau }}$ is an ${\displaystyle h}$-transform of ${\displaystyle X}$.^[1]

Time-reversal

The time-reversal ${\displaystyle {\overleftarrow {X}}}$ of ${\displaystyle X}$ is defined as$${\displaystyle {\overleftarrow {X_{t}}}(\omega )={\begin{cases}X_{\zeta (\omega )-t}&,t\leq \zeta (\omega )\\\partial &,t>\zeta (\omega ){\text{ or }}\zeta (\omega )=\infty \end{cases}}.}$$${\displaystyle {\overleftarrow {X}}}$ is again a homogeneous Markov process, however now with left-continuous paths.^[1]

If ${\displaystyle X^{h}}$ has initial measure ${\displaystyle \nu \propto h\cdot \mathrm {d} \mu }$, then the reverses of ${\displaystyle X}$ and ${\displaystyle X^{h}}$ have the same transition probabilities. In particular, for a co-optional time ${\displaystyle \tau }$ (for example a last hitting time), if ${\displaystyle X}$ starts in a fixed point almost surely, then the reverses of ${\displaystyle X}$ and ${\displaystyle X^{\tau }}$ have the same transition probabilities.^[1]

Moreover, if ${\displaystyle X}$ possesses a dual process ${\displaystyle {\hat {X}}}$ (defined below), then its reverse can be expressed as an ${\displaystyle h}$-transform of its dual. To formulate this statement, we define for ${\displaystyle X}$ and ${\displaystyle {\hat {X}}}$ respectively the transition semigroups ${\displaystyle (P_{t})_{t\geq 0}}$ and ${\displaystyle ({\hat {P}}_{t})_{t\geq 0}}$, and the resolvents$${\displaystyle {\begin{aligned}U_{p}:=\int _{0}^{\infty }e^{-pt}P_{t}\mathrm {d} t,\\{\hat {U}}_{p}:=\int _{0}^{\infty }e^{-pt}{\hat {P}}_{t}\mathrm {d} t.\end{aligned}}}$$As the dual of ${\displaystyle X}$, ${\displaystyle {\hat {X}}}$ musst fulfill for some measure ${\displaystyle \nu }$

1. ${\displaystyle {\hat {X}}}$ is a strong Markov process with càdlàg paths,
2. For all ${\displaystyle p\geq 0}$ and ${\displaystyle f,g\geq 0}$ measurable: ${\displaystyle \int _{E}\int _{E}f(y)U_{p}(x,\mathrm {d} y)g(x)\mathrm {d} \nu (x)=\int _{E}f(x)\int _{E}g(y){\hat {U}}_{p}(x,\mathrm {d} y)\mathrm {d} \nu (x),}$
3. For all ${\displaystyle p\geq 0}$ and ${\displaystyle x\in E}$, both ${\displaystyle U_{p}(x,\cdot )}$ and ${\displaystyle {\hat {U}}_{p}(x,\cdot )}$ are absolutely continuous w.r.t. ${\displaystyle \nu }$.

Then by setting$${\displaystyle h(x):=\int _{E}{\frac {\mathrm {d} U_{0}(x,\cdot )}{\mathrm {d} \nu }}(y)\mathrm {d} \mu (y):=\int _{E}u_{0}(x,\mathrm {d} \mu (y),}$$${\displaystyle {\overleftarrow {X}}}$ and ${\displaystyle {\hat {X}}^{h}}$ have the same transition probabilities.

Conditioning

Doob's ${\displaystyle h}$-transform can also be used to condition the process ${\displaystyle X}$ on hitting a distinct point or, more generally, to attain a pre-defined distribution at its lifetime.

Let ${\displaystyle y\in E}$ and ${\displaystyle U_{0}}$ be the resolvent of ${\displaystyle X}$ for ${\displaystyle p=0}$ as defined above. Then with$${\displaystyle h(x)=u_{0}(x,y),}$$it holds that ${\displaystyle X_{\zeta -}^{h}=y}$ almost surely on ${\displaystyle \{\zeta <\infty \}}$.

If ${\displaystyle \mu =\delta _{x_{0}}}$, that is, ${\displaystyle X}$ starts in ${\displaystyle x_{0}}$ almost surely, then one can achieve ${\displaystyle X_{\zeta -}^{h}\sim \alpha }$ for any distribution ${\displaystyle \alpha }$ on ${\displaystyle E}$ by choosing^[1]$${\displaystyle h(x):=\int _{E}{\frac {u_{0}(x,y)}{u_{0}(x_{0},y)}}\mathrm {d} \alpha (y).}$$

Generative modelling

Doob's h-transform has begun to find applications in generative modeling, for example for point clouds, climate data, graphs and image segmentation^[4]. The method has also been proposed for faster conditional training and sampling of diffusion models.^[5]