we introduce an efficient approach named Denoising Diffusion Restoration Models (DDRM), that can achieve competitive results in as low as 20 NFEs. DDRM is a denoising diffusion generative model [44, 19, 45] that gradually and stochastically denoises a sample to the desired output, conditioned on the measurements and the inverse problem. This way we introduce a variational inference objective for learning the posterior distribution of the inverse problem at hand. We then show its equivalence to the objective of an unconditional denoising diffusion generative model [19], which enables us to deploy such models in DDRM for various linear inverse problems (see Figure 2).
Inverse problem solvers based on posterior sampling often face a dilemma: unsupervised approaches apply to general problems but are inefficient, whereas supervised ones are efficient but can only address specific problems.
To solve this dilemma, we introduce Denoising Diffusion Restoration Models (DDRM), an unsupervised solver for general linear inverse problems, capable of handling such tasks with or without noise in the measurements. DDRM is efficient and exhibits competitive performance compared to popular unsupervised solvers
For any linear inverse problem, we define DDRM as a Markov chain xT → xT −1 → . . . → x1 → x0 conditioned on y, where
and x0 is the final diffusion output. In order to perform inference, we consider the following factorized variational distribution conditioned on y:
leading to an ELBO objective for diffusion models conditioned on y
we aim to recover the signal $x ∈ R^n$ from measurements $y ∈ R^m$, where H ∈ R m×n is a known linear degradation matrix, and z ∼ N (0, σ2 yI) i
we construct suitable variational problems given H and σy and connect them to unconditional diffusion generative models.
we consider the singular value decomposition (SVD) of H, and perform the diffusion in its spectral space. . The idea behind this is to tie the noise present in the measurements y with the diffusion noise in x1:T , ensuring that the diffusion result x0 is faithful to the measurements. By using the SVD, we identify the data from x that is missing in y, and synthesize it using a diffusion process. In conjunction, the noisy data in y undergoes a denoising process. For example, in inpainting with noise (e.g., H = diag([1, . . . , 1, 0, . . . , 0]), σy ≥ 0), the spectral space is simply the pixel space, so the model should generate the missing pixels and denoise the observed ones in y. For a general linear H, its SVD is given as
