Neural network language models to select the best translation

View/Open
Document typeArticle
Defense date2013-12-20
Rights accessOpen Access
Abstract
The quality of translations produced by statistical machine translation (SMT) systems crucially
depends on the generalization ability provided by the statistical models involved in the process.
While most modern SMT systems use n-gram models to predict the next element in a sequence
of tokens, our system uses a continuous space language model (LM) based on neural networks
(NN). In contrast to works in which the NN LM is only used to estimate the probabilities of
shortlist words (Schwenk 2010), we calculate the posterior probabilities of out-of-shortlist words
using an additional neuron and unigram probabilities. Experimental results on a small Italian-
to-English and a large Arabic-to-English translation task, which take into account di erent word
history lengths (n-gram order), show that the NN LMs are scalable to small and large data and
can improve an n-gram-based SMT system. For the most part, this approach aims to improve
translation quality for tasks that lack translation data, but we also demonstrate its scalability to
large-vocabulary tasks.
CitationKhalilov, M. [et al.]. Neural network language models to select the best translation. "Computational Linguistics in the Netherlands Journal", 20 Desembre 2013, vol. 3, p. 217-233.
ISSN2211-4009
Files | Description | Size | Format | View |
---|---|---|---|---|
13-Khalilov-etal-CLIN2013.pdf | 454,4Kb | View/Open |
Except where otherwise noted, content on this work
is licensed under a Creative Commons license
:
Attribution-NonCommercial-NoDerivs 3.0 Spain