Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Can machine translation output be evaluated through eye tracking?

Doherty, Stephen orcid logoORCID: 0000-0003-0887-1049 and O'Brien, Sharon orcid logoORCID: 0000-0003-4864-5986 (2009) Can machine translation output be evaluated through eye tracking? In: Machine Translation Summit XII, 26-30 Aug 2009, Ottawa, Canada.

This paper reports on a preliminary study testing the use of eye tracking as a method for evaluating machine translation output. 50 French machine translated sentences, 25 rated as excellent and 25 rated as poor in an earlier human evaluation, were selected. 10 native speakers of French were instructed to read the MT sentences for comprehensibility. Their eye gaze data were recorded noninvasively using a Tobii 1750 eye tracker. They were also asked to record retrospective protocols while watching a replay of their eye gaze reading data. The average gaze time and fixation count were found to be significantly higher for the “bad” sentences, while average fixation duration was not significantly different. Evaluative comments uttered during the retrospective protocols were also found to agree to a satisfactory degree with previous human evaluation. Overall, we found that the eye tracking method correlates reasonably well with human evaluation of MT output.
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Uncontrolled Keywords:MT Evaluation; user-based evaluation; Eye tracking; Gaze time; Fixation count; Fixation duration; Retrospective protocols
Subjects:Computer Science > Machine translating
Medical Sciences > Psychology
DCU Faculties and Centres:Research Institutes and Centres > Centre for Next Generation Localisation (CNGL)
DCU Faculties and Schools > Faculty of Humanities and Social Science > School of Applied Language and Intercultural Studies
Official URL:http://summitxii.amtaweb.org/
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland
ID Code:19469
Deposited On:02 Oct 2013 09:06 by Stephen Doherty . Last Modified 19 Jan 2022 12:28

Full text available as:

[thumbnail of MTS-2009-Doherty.pdf]
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Downloads per month over past year

Archive Staff Only: edit this record