Castilho, Sheila ORCID: 0000-0002-8416-6555 (2020) Document-level machine translation evaluation project: methodology, effort and inter-annotator agreement. In: 22nd Annual Conference of the European Association for Machine Translation, 3-5 Nov 2020, Lisbon, Portugal (Online). ISBN 978-989-33-0589-8
Abstract
Recently, document-level (doc-level) human evaluation of machine translation
(MT) has raised interest in the community after a few attempts have disproved
claims of “human parity” (Toral et al.,
2018; Laubli et al., 2018). However, lit- ¨
tle is still known about best practices regarding doc-level human evaluation. This
project aims to identify methodologies to
better cope with i) the current state-of-theart (SOTA) human metrics, ii) a possible
complexity when assigning a single score
to a text consisted of ‘good’ and ‘bad’
sentences, iii) a possible tiredness bias in
doc-level set-ups, and iv) the difference in
inter-annotator agreement (IAA) between
sentence and doc-level set-ups.
Metadata
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Event Type: | Conference |
Refereed: | Yes |
Uncontrolled Keywords: | machine translation evaluation; human evaluation of machine translation |
Subjects: | Computer Science > Machine translating Humanities > Translating and interpreting |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Institutes and Centres > ADAPT |
Published in: | Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. . EAMT. ISBN 978-989-33-0589-8 |
Publisher: | EAMT |
Official URL: | https://eamt2020.inesc-id.pt/proceedings-eamt2020.... |
Copyright Information: | © 2020 The Authors. CC-BY-ND |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | EAMT under its programme “2019 Sponsorship of Activities, Science Foundation Ireland Research Centres Programme (Grant 13/RC/2106), European Regional Development Fund |
ID Code: | 24647 |
Deposited On: | 18 Jun 2020 11:58 by Sheila Castilho . Last Modified 18 Jun 2020 12:09 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
178kB |
Metrics
Altmetric Badge
Dimensions Badge
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record