Data Selection is a popular step in Machine Translation pipelines. Feature Decay Algorithms (FDA) is a technique for data selection that has shown a good performance in several tasks. FDA aims to maximize the coverage of n-grams in the test set. However, intuitively, more ambiguous n-grams require more training examples in order to adequately estimate their translation probabilities. This ambiguity can be measured by alignment entropy. In this paper we propose two methods for calculating the alignment entropies for n-grams of any size, which can be used for improving the performance of FDA. We evaluate the substitution of the n-gram-specific entropy values computed by these methods to the parameters of both the exponential and linear decay factor of FDA. The experiments conducted on German-to-English and Czech-to-English translation demonstrate that the use of alignment entropies can lead to an increase in the quality of the results of FDA.
This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:
ADAPT Centre under the SFI Research Centres Programme (Grant 13/RC/2106)., European Union’s Horizon 2020 under the European Union’s Horizon 2020 research and innovthe Marie Skłodowska-Curie grant agreement No 713567.
ID Code:
22304
Deposited On:
29 Mar 2018 10:34 by
Gideon Maillette De buy
. Last Modified 22 Jan 2021 14:17