However, this direct connection between DDPMs and numerical methods (e.g., forward Euler method, linear multi-step method and RungeKutta method (Timothy, 2017)) has weaknesses in both speed and effect (see Section 3.1).
Furthermore, we also notice that numerical methods can introduce noticeable noise at a high speedup rate, which makes high-order numerical methods (e.g., Runge-Kutta method) even less effective than DDIMs.
To tackle these problems, we design new numerical methods called pseudo numerical methods for diffusion models (PNDMs) to generate samples along a specific manifold in R n, which is the highdensity region of the data. We first compute the corresponding differential equations of diffusion models directly and self-consistently, which builds a theoretical connection between DDPMs and numerical methods. Considering that classical numerical methods cannot guarantee to generate samples on certain manifolds, we provide brand-new numerical methods called pseudo numerical methods based on our theoretical analyses. We also find that DDIMs are simple cases of pseudo numerical methods, which means that we also provide a new way to understand DDIMs better. Furthermore, we find that the pseudo linear multi-step method is the fastest method for diffusion models under similar generated quality.
there is another understanding of DDPMs. The diffusion process can be treated as solving a certain stochastic differential equation dx = (p p 1 − β(t) − 1)x(t)dt + β(t)dw.
This is Variance Preserving stochastic differential equations (VP-SDEs). Here, we change the domain of t from [1, N] to [0, 1]. When N tends to infinity, {βi} N i=1, {xi} N i=1 become continuous functions β(t) and x(t) on [0, 1]. Song et al. (2020b) also show that this equation has an ordinary differential equation (ODE) version with the same marginal probability density as Equation (4):
NUMERICAL METHOD
Many classical numerical methods can be used to solve ODEs, including the forward Euler method, Runge-Kutta method and linear multi-step method (Timothy, 2017).
the reverse process of DDPMs and DDIMs satisfies:
σt controls the ratio of random noise. If σt equals one, Equation (8) represents the reverse process of DDPMs; if σt equals zero, this equation represents the reverse process of DDIMs. And only when σt equals zero, this equation removes the random item and becomes a discrete form of a certain ODE.
Therefore, our work concentrate on the case σt equals zero.
To find the corresponding ODE of Equation (8), we replace discrete t − 1 with a continuous version t − δ according to (Song et al., 2020a) and change this equation into a differential form, namely, subtract xt from both sides of this equation: