Sain: Similarity-Aware Video Frame Interpolation
- Resource Type
- Conference
- Authors
- Lv, Yue; Yang, Wenming; Zuo, Wangmeng; Liao, Qingmin; Zhu, Rui
- Source
- ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2022 - 2022 IEEE International Conference on. :1920-1924 May, 2022
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Representation learning
Image quality
Image texture
Interpolation
Superresolution
Feature extraction
Search problems
VFI
Implicit Neural Representation
Similar Patches Aggregation
Restore Image Texture
- Language
- ISSN
- 2379-190X
Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive original frames. Most existing methods simply linearly combine the warped frames, leading to a loss of image texture. Since moving objects usually have similarities in consecutive frames, we propose a similarity-aware video frame interpolation method (SAIN) that searches patches with similar texture in the embedding space from input frames to extract features and capture image details. To gather the frame details and restore image texture, SAIN incorporates an implicit neural representation learning from similar patches to enrich image details and refine outputs in frame synthesis networks. Experiments demonstrate that SAIN preserves image texture and enhances interpolated image quality significantly.