Improving the objective function in minimum error rate training
He, Yifan and Way, AndyORCID: 0000-0001-5736-5930
(2009)
Improving the objective function in minimum error rate training.
In: MT Summit XII - The twelfth Machine Translation Summit, 26-30 August 2009, Ottawa, Canada.
In Minimum Error Rate Training (MERT), the parameters of an SMT system are tuned on a certain evaluation metric to improve translation quality. In this paper, we present empirical results in which parameters tuned on one metric (e.g. BLEU) may not lead to optimal scores on the same metric. The score can be improved significantly by tuning on an entirely different metric (e.g. METEOR, by 0.82
BLEU points or 3.38% relative improvement on WMT08 English–French dataset). We analyse the impact of choice of objective function in MERT and further propose three
combination strategies of different metrics to reduce the bias of a single metric, and obtain parameters that receive better scores (0.99 BLEU points or 4.08% relative improvement) on evaluation metrics than those tuned on the
standalone metric itself.