Do online machine translation systems care for context?
What about a GPT model?
Castilho, SheilaORCID: 0000-0002-8416-6555, Mallon, Clodagh, Meister, Rahel and Yue, Shengya
(2023)
Do online machine translation systems care for context?
What about a GPT model?
In: 24th Annual Conference of the European Association for Machine Translation (EAMT 2023), 12-15 June 2023, Tampere, Finland.
(In Press)
This paper addresses the challenges of
evaluating document-level machine translation
(MT) in the context of recent advances
in context-aware neural machine
translation (NMT). It investigates how well
online MT systems deal with six contextrelated
issues, namely lexical ambiguity,
grammatical gender, grammatical number,
reference, ellipsis, and terminology, when
a larger context span containing the solution
for those issues is given as input. Results
are compared to the translation outputs
from the online ChatGPT. Our results
show that, while the change of punctuation
in the input yields great variability in
the output translations, the context position
does not seem to have a great impact.
Moreover, the GPT model seems to outperform
the NMT systems but performs
poorly for Irish.