Multiple segmentations of Thai sentences for neural machine translation
Poncelas, AlbertoORCID: 0000-0002-5089-1687, Pidchamook, Wichaya, Liu, Chao-HongORCID: 0000-0002-1235-6026, Hadley, James and Way, AndyORCID: 0000-0001-5736-5930
(2020)
Multiple segmentations of Thai sentences for neural machine translation.
In: Spoken Language Technologies for Under-resourced languages and Collaboration and Computing for Under-Resourced Languages Workshop, (SLTU-CCURL 2020), 11 - 12 May 2020, Marseille, France. (Virtual).
Thai is a low-resource language, so it is often the case that data is not available in sufficient quantities to train an Neural Machine Translation (NMT) model which perform to a high level of quality. In addition, the Thai script does not use white spaces to delimit the boundaries between words, which adds more complexity when building sequence to sequence models. In this work, we explore how to augment a set of English–Thai parallel data by replicating sentence-pairs with different word segmentation methods on Thai, as training data for NMT model training. Using different merge operations of Byte Pair Encoding, different segmentations of Thai sentences can be obtained. The experiments show that combining these datasets, performance is improved for NMT models trained with a dataset that has been split using a supervised splitting tool.
Item Type:
Conference or Workshop Item (Paper)
Event Type:
Workshop
Refereed:
Yes
Additional Information:
Part of LREC 2020 Workshop
Language Resources and Evaluation Conference
11-16 May 2020
Beermann, Dorothee and Besacier, Laurent and Sakti, Sakriani and Soria, Claudia, (eds.)
Proceedings of 1st Joint SLTU and CCURL Workshop (SLTU-CCURL 2020).
.
LREC.