Alekseeva, Alexandra ORCID: 0000-0002-7990-4592, Gladkoff, Serge, Sorokina, Irina and Han, Lifeng ORCID: 0000-0002-3221-2185 (2021) Monte Carlo modelling of confidence intervals in translation quality evaluation (TQE) and post-editing dstance (PED) measurement. In: Metrics 2021: Workshop on Informetric and Scientometric Research (SIG-MET), 23-24 Oct 2021, Online.
Abstract
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task. This is especially the case, when language service providers (LSPs) face huge amount of request frequently from their clients and users to acquire high-quality translations.
While automatic translation quality assessment (TQA) metrics and quality estimation (QE) tools are widely available and easy to access, human assessment from professional translators (HAP) are often chosen as the golden standard \cite{han-etal-2021-TQA}.
One challenge that comes to this point is this: \textit{to avoid the overall text quality checking from both cost and efficiency perspectives, how to choose the confidence sample size of the translated text, so as to properly estimate the overall text or document translation quality}?
This work carries out such an motivated research to correctly estimate the confidence intervals \cite{Brown_etal2001Interval} regarding the sample size of translated text, e.g. the amount of words or sentences, that needs to be taken into account for confident evaluation of overall translation quality. The methodology we applied for this work is from Bernoulli Statistical Distribution Modelling (BSDM) and Monte Carlo Sampling Analysis (MCSA).
Metadata
Item Type: | Conference or Workshop Item (Poster) |
---|---|
Event Type: | Workshop |
Refereed: | Yes |
Uncontrolled Keywords: | Translation Quality Evaluation; Quality Estimation; Post-editing Distance; Confidence Intervals; Monte Carlo Modeling; Bernoulli Statistics |
Subjects: | Computer Science > Algorithms Computer Science > Artificial intelligence Computer Science > Computational linguistics Computer Science > Computer software Computer Science > Information technology Mathematics > Probabilities Mathematics > Stochastic analysis Mathematics > Statistics |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Institutes and Centres > ADAPT |
Copyright Information: | © 2021 The Authors |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | ADAPT |
ID Code: | 26281 |
Deposited On: | 21 Sep 2021 09:48 by Lifeng Han . Last Modified 21 Sep 2021 09:48 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
237kB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record