PRRD: Pixel-Region Relation Distillation For Efficient Semantic Segmentation
- Resource Type
- Conference
- Authors
- Wang, Chen; Zhong, Jiang; Dai, Qizhu; Qi, Yafei; Li, Rongzhen; Lei, Qin; Fang, Bin; Li, Xue
- Source
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2023 - 2023 IEEE International Conference on. :1-5 Jun, 2023
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Semantic segmentation
Signal processing
Network architecture
Acoustics
Speech processing
Knowledge Distillation
Semantic Segmentation
Multi-Scale Context
Pixel-Region Relation
- Language
- ISSN
- 2379-190X
Current state-of-the-art semantic segmentation methods usually require high computational resources for accurate segmentation. Knowledge distillation has been one promising way to achieve a good trade-off between accuracy and efficiency. However, current distillation methods focus on transferring the spatial relations and ignore the multi-scale context interaction. This paper proposes one novel pixel- region relation distillation (PPRD) to transfer the multi-scale pixel-region relation (PRR) from the teacher to the student. We get the multi-scale regions with pyramid pooling and characterize the multi-scale PRR between the feature and the multi-scale regions. Transferring such PRR from the teacher to the student is beneficial for the student to mimic the teacher better in terms of multi-scale context interaction. Experimental results on two challenging datasets, Cityscapes and Pascal VOC 2012, show that the proposed approach outperforms state-of-the-art distillation methods.