Space-time Neural Irradiance Fields for Free-Viewpoint Video
- Resource Type
- Conference
- Authors
- Xian, Wenqi; Huang, Jia-Bin; Kopf, Johannes; Kim, Changil
- Source
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2021 IEEE/CVF Conference on. :9416-9426 Jun, 2021
- Subject
- Computing and Processing
Geometry
Computer vision
Three-dimensional displays
Heuristic algorithms
Dynamics
Estimation
Rendering (computer graphics)
- Language
- ISSN
- 2575-7075
We present a method that learns a spatiotemporal neural irradiance field for dynamic scenes from a single video. Our learned representation enables free-viewpoint rendering of the input video. Our method builds upon recent advances in implicit representations. Learning a spatiotemporal irradiance field from a single video poses significant challenges because the video contains only one observation of the scene at any point in time. The 3D geometry of a scene can be legitimately represented in numerous ways since varying geometry (motion) can be explained with varying appearance and vice versa. We address this ambiguity by constraining the time-varying geometry of our dynamic scene representation using the scene depth estimated from video depth estimation methods, aggregating contents from individual frames into a single global representation. We provide an extensive quantitative evaluation and demonstrate compelling free-viewpoint rendering results.