Multi-way, multilingual neural machine translation

Firat O., Cho K., Sankaran B., YARMAN VURAL F. T. , Bengio Y.

COMPUTER SPEECH AND LANGUAGE, vol.45, pp.236-252, 2017 (Peer-Reviewed Journal) identifier identifier

  • Publication Type: Article / Article
  • Volume: 45
  • Publication Date: 2017
  • Doi Number: 10.1016/j.csl.2016.10.006
  • Journal Indexes: Science Citation Index Expanded, Scopus
  • Page Numbers: pp.236-252
  • Keywords: Neural machine translation, Multi-lingual, Low resource translation


We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT'15 simultaneously and observe clear performance improvements over models trained on only one language pair. We empirically evaluate the proposed model on low-resource language translation tasks. In particular, we observe that the proposed multilingual model outperforms strong conventional statistical machine translation systems on Turkish-English and Uzbek-English by incorporating the resources of other language pairs. (C) 2016 Elsevier Ltd. All rights reserved