Guide and Select: A Transformer-Based Multimodal Fusion Method for Points of Interest Description Generation
- Resource Type
- Conference
- Authors
- Liu, Hanqing; Wang, Wei; Hu, Niu; Zheng, Hai-Tao; Xie, Rui; Wu, Wei; Bai, Yang
- Source
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2023 - 2023 IEEE International Conference on. :1-5 Jun, 2023
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Fuses
Signal processing
Transformers
Encoding
Data models
Acoustics
Task analysis
Points of Interest
Text Generation
Multimodal Fusion
- Language
- ISSN
- 2379-190X
The task of Points of Interest (POI) description generation aims to generate an objective and informative description for a given POI based on POI-related information. High-quality descriptions can better guide users and improve the performance of POI-related recommendation systems. A practical POI description generation model should have effective multimodal fusion and information encoding methods suitable for various data forms. However, due to model structure and data utilization limitations, the previous method is challenging to meet the above requirements. We propose a novel Guide-Select multimodal fusion method that combines the guiding and selecting process to fuse various POI-related information efficiently. In addition, we propose a reasonable review encoding method and a category encoding method that has strong generalization ability. We integrate these methods into our Guide-Select Generation Model (GSGM). Experimental results demonstrate that our model significantly outperforms the state-of-the-art model while having a strong generalization ability on category information.