Lost in translation: the problems of using mainstream MT evaluation metrics for sign language translation
Morrissey, Sara and Way, Andy (2006) Lost in translation: the problems of using mainstream MT evaluation metrics for sign language translation. In: SALTMIL 2006 - 5th SALTMIL Workshop on Minority Languages, 23 May 2006, Genoa, Italy.
Full text available as:
In this paper we consider the problems of applying corpus-based techniques to minority languages that are neither politically recognised nor have a formally accepted writing system, namely sign languages. We discuss the adoption of an annotated form of sign language data as a suitable corpus for the development of a data-driven machine translation (MT) system, and deal with issues that arise from its use. Useful software tools that facilitate easy annotation of video data are also discussed. Furthermore, we address the problems of using traditional MT evaluation metrics for sign language translation. Based on the candidate translations produced from our example-based machine translation system, we discuss why standard metrics fall short of providing an accurate evaluation and suggest more suitable evaluation methods.
Archive Staff Only: edit this record