The paper proposes a method for improving score-based diffusion models by enforcing the underlying score Fokker-Planck equation (FPE), which characterizes the noise-conditional scores of the perturbed data densities, and shows its effectiveness across various datasets.

Key insights and lessons learned from the paper:

• Score-based generative models learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise.
• The perturbed data densities are tied together by the Fokker-Planck equation (FPE), a partial differential equation (PDE) governing the spatial-temporal evolution of a density undergoing a diffusion process.
• The paper derives a corresponding equation, called the score FPE that characterizes the noise-conditional scores of the perturbed data densities (i.e., their gradients).
• The paper shows that satisfying the score FPE is desirable as it improves the likelihood and the degree of conservativity.
• The paper proposes to regularize the denoising score matching (DSM) objective to enforce satisfaction of the score FPE, and shows the effectiveness of this approach across various datasets.

Questions for the authors:

1. Can you provide more details on how the proposed approach improves the likelihood and the degree of conservativity of score-based diffusion models?
2. How does the proposed method compare to other regularization techniques for score-based generative models?
3. Have you considered applying the proposed approach to other types of diffusion models, such as Langevin dynamics?
4. How does the proposed method scale to larger datasets and higher-dimensional spaces?
5. What are the limitations and potential drawbacks of the proposed approach?

Suggestions for future research:

1. Investigating the effectiveness of the proposed approach on more complex and diverse datasets, including natural images and videos.
2. Extending the proposed method to other types of generative models, such as flow-based models.
3. Exploring the theoretical properties of the score Fokker-Planck equation and its implications for generative modeling.
4. Studying the connections between score-based diffusion models and other types of generative models, such as autoregressive models and variational autoencoders.
5. Developing methods for optimizing the proposed regularized DSM objective more efficiently and effectively.

Relevant references: