An Extensive Empirical Study of Automated Evaluation of Multi-Document Summarization
- Resource Type
- Conference
- Authors
- Wu, Ying-Qiang; Zhou, Gang; Qiu, Li-Qing
- Source
- 2008 First International Conference on Intelligent Networks and Intelligent Systems Intelligent Networks and Intelligent Systems, 2008. ICINIS '08. First International Conference on. :720-724 Nov, 2008
- Subject
- Computing and Processing
Communication, Networking and Broadcast Technologies
Humans
Intelligent networks
Space technology
NIST
Frequency
Intelligent systems
Programming
Switching systems
Systems engineering and theory
Computer networks
multi-document summarization
automated evaluation
n-gram
- Language
This paper discusses an approach to automated evaluation of multi-document summarization by computing the similarities of automated summaries and human summaries and scoring the automated summaries by their similarities to the human ones. Several schemes are used in our experiment, as well as the effects of stop words and stemming. Our method experimental result is compared to Rouge which is based on n-gram. The test materials for experiments are from DUC 2005 corpus. The results show that our novel scheme produces acceptable results, and may avoid some defects of n-gram.