Diffusion Probabilistic Modeling for Video Generation
- Resource Type
- article
- Authors
- Ruihan Yang; Prakhar Srivastava; Stephan Mandt
- Source
- Entropy, Vol 25, Iss 10, p 1469 (2023)
- Subject
- diffusion models
deep generative models
video generation
autoregressive models
Science
Astrophysics
QB460-466
Physics
QC1-999
- Language
- English
- ISSN
- 1099-4300
Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.