Examining Manual and Automatic MT Evaluation: A Grey Relational Analysis for Chinese-Portuguese Translation Quality
- Resource Type
- Conference
- Authors
- Sun, Yuqi; Hoi, Lap Man; Im, Sio Kei
- Source
- 2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence (CCAI) Computer Communication and Artificial Intelligence (CCAI), 2023 IEEE 3rd International Conference on. :425-430 May, 2023
- Subject
- Communication, Networking and Broadcast Technologies
Computing and Processing
Correlation
Semantics
Manuals
Linguistics
Quality assessment
Machine translation
Indexes
Chinese-Portuguese Machine Translation
Manual Evaluation Index
BLEU
Grey Relational Analysis
- Language
This study investigates the relationship between manual and automatic machine translation evaluation methodologies by employing Grey Correlation Analysis (GRA) to assess the correlation between Chinese-Portuguese machine translation outputs’ BLEU scores and human evaluation scores based on a proposed evaluation index system. The research aims to provide insights into the factors impacting machine translation quality and the most relevant linguistic dimensions in translation evaluation. The findings reveal that “usability” and “adequacy” exhibit the highest correlation with BLEU scores, while “Semantics” ranks highest among the ten manual evaluation indicators, when correlated with the aggregated human evaluation results, followed by “Correction” (correct information) and “Omission and/or Addition”. The findings contribute to the field of machine translation evaluation by illuminating the complex relationship between manual and automatic evaluation techniques and guiding future improvements in machine translation systems and evaluation methodologies.