Show simple item record

dc.contributor.authorFarrus, Mireia
dc.contributor.authorRuiz Costa-Jussà, Marta
dc.contributor.authorPopovic, Maya
dc.contributor.authorHenriquez, Carlos A
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions
dc.identifier.citationFarrus, M., Ruiz, M., Popovic, M., Henriquez, C. Study and correlation analysis of linguistic, perceptual and automatic machine translation evaluations. "Journal of the Association for Information Science and Technology", 1 Gener 2012, vol. 63, núm. 1, p. 174-184.
dc.description.abstractEvaluation of machine translation output is an important task. Various human evaluation techniques as well as automatic metrics have been proposed and investigated in the last decade. However, very few evaluation methods take the linguistic aspect into account. In this article, we use an objective evaluation method for machine translation output that classifies all translation errors into one of the five following linguistic levels: orthographic, morphological, lexical, semantic, and syntactic. Linguistic guidelines for the target language are required, and human evaluators use them in to classify the output errors. The experiments are performed on English-to-Catalan and Spanish-to-Catalan translation outputs generated by four different systems: 2 rule-based and 2 statistical. All translations are evaluated using the 3 following methods: a standard human perceptual evaluation method, several widely used automatic metrics, and the human linguistic evaluation. Pearson and Spearman correlation coefficients between the linguistic, perceptual, and automatic results are then calculated, showing that the semantic level correlates significantly with both perceptual evaluation and automatic metrics.
dc.format.extent11 p.
dc.subjectÀrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural
dc.subject.lcshMachine translating -- Evaluation
dc.subject.otherLinguistic machine translation evaluation
dc.titleStudy and correlation analysis of linguistic, perceptual and automatic machine translation evaluations
dc.subject.lemacTraducció automàtica -- Avaluació
dc.rights.accessOpen Access
dc.description.versionPostprint (author's final draft)
local.citation.authorFarrus, M.; Ruiz, M.; Popovic, M.; Henriquez, C.
local.citation.publicationNameJournal of the Association for Information Science and Technology

Files in this item


This item appears in the following Collection(s)

Show simple item record

Attribution-NonCommercial-NoDerivs 3.0 Spain
Except where otherwise noted, content on this work is licensed under a Creative Commons license : Attribution-NonCommercial-NoDerivs 3.0 Spain