Show simple item record

dc.contributor.authorLabaka, Gorka
dc.contributor.authorDíaz de Ilarraza Sánchez, Arantza
dc.contributor.authorSarasola Gabiola, Kepa
dc.contributor.authorEspaña Bonet, Cristina
dc.contributor.authorMàrquez Villodre, Lluís
dc.contributor.otherUniversitat Politècnica de Catalunya. Departament de Llenguatges i Sistemes Informàtics
dc.date.accessioned2012-12-03T16:21:19Z
dc.date.available2012-12-03T16:21:19Z
dc.date.created2011
dc.date.issued2011
dc.identifier.citationLabaka, G. [et al.]. Deep evaluation of hybrid architectures: simple metrics correlated with human judgments. A: International Workshop on Using Linguistic Information for Hybrid Machine Translation. "LIHMT 2011 Sponsors International Workshop on Using Linguistic Information for Hybrid Machine Translation". Barcelona: 2011, p. 50-57.
dc.identifier.urihttp://hdl.handle.net/2117/17063
dc.description.abstractThe process of developing hybrid MT systems is guided by the evaluation method used to compare different combinations of basic subsystems. This work presents a deep evaluation experiment of a hybrid architecture that tries to get the best of both worlds, rule-based and statistical. In a first evaluation human assessments were used to compare just the single statistical system and the hybrid one, the rule-based system was not compared by hand because the results of automatic evaluation showed a clear disadvantage. But a second and wider evaluation experiment surprisingly showed that according to human evaluation the best system was the rule-based, the one that achieved the worst results using automatic evaluation. An examination of sentences with controversial results suggested that linguistic well-formedness in the output should be considered in evaluation. After experimenting with 6 possible metrics we conclude that a simple arithmetic mean of BLEU and BLEU calculated on parts of speech of words is clearly a more human conformant metric than lexical metrics alone.
dc.format.extent8 p.
dc.language.isoeng
dc.subjectÀrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural
dc.subject.lcshMachine translation
dc.subject.lcshRule-based machine translation
dc.titleDeep evaluation of hybrid architectures: simple metrics correlated with human judgments
dc.typeConference lecture
dc.subject.lemacTraducció automàtica
dc.contributor.groupUniversitat Politècnica de Catalunya. GPLN - Grup de Processament del Llenguatge Natural
dc.description.peerreviewedPeer Reviewed
dc.relation.publisherversionhttp://ixa2.si.ehu.es/lihmt2011/proceedings.pdf
dc.rights.accessOpen Access
drac.iddocument10177456
dc.description.versionPostprint (author’s final draft)
upcommons.citation.authorLabaka, G.; Díaz de Ilarraza, A.; Sarasola , K.; España-Bonet, C.; Marquez, L.
upcommons.citation.contributorInternational Workshop on Using Linguistic Information for Hybrid Machine Translation
upcommons.citation.pubplaceBarcelona
upcommons.citation.publishedtrue
upcommons.citation.publicationNameLIHMT 2011 Sponsors International Workshop on Using Linguistic Information for Hybrid Machine Translation
upcommons.citation.startingPage50
upcommons.citation.endingPage57


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder