Perception-Oriented Omnidirectional Image Super-Resolution Based on Transformer Network
- Resource Type
- Conference
- Authors
- An, Hongyu; Zhang, Xinfeng
- Source
- 2023 IEEE International Conference on Image Processing (ICIP) Image Processing (ICIP), 2023 IEEE International Conference on. :3583-3587 Oct, 2023
- Subject
- Computing and Processing
Signal Processing and Analysis
Training
Visualization
Convolution
Superresolution
Bandwidth
Transformers
Data augmentation
Omnidirectional image
super-resolution
perceptual quality
- Language
Omnidirectional image (ODI) super-resolution (SR) is an important technique in augmented reality and virtual reality applications to address the low-resolution problem caused by limitations in capturing devices or bandwidth. The ODI projection distortion makes it challenging to apply existing SR methods. In this paper, we propose an ODI SR method by leveraging the characteristics of ODIs and human visual characteristics. Specifically, we firstly design a perception-orientated adaptive loss function by jointly utilizing saliency map and latitude map. In our proposed ODI-SR network, we introduce an attention module to aggregate multi-scale information and leverage spherical convolution to adapt to the spheric format of ODIs. Furthermore, we design a data augmentation strategy for ODIs according to viewpoint distribution to further improve the visual quality of SR images. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance according to both qualitative and quantitative evaluations.