The DCU dependency-based metric in WMT-MetricsMATR 2010
He, Yifan and Du, Jinhua and Way, Andy and van Genabith, Josef (2010) The DCU dependency-based metric in WMT-MetricsMATR 2010. In: WMT 2010 - Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR, ACL 2010., 15-16 July, Uppsala, Sweden.
Full text available as:
We describe DCU’s LFG dependencybased
metric submitted to the shared evaluation
task of WMT-MetricsMATR 2010.
The metric is built on the LFG F-structurebased
approach presented in (Owczarzak
et al., 2007). We explore the following
improvements on the original metric: 1)
we replace the in-house LFG parser with
an open source dependency parser that
directly parses strings into LFG dependencies;
2) we add a stemming module
and unigram paraphrases to strengthen the
aligner; 3) we introduce a chunk penalty
following the practice of METEOR to reward
continuous matches; and 4) we introduce
and tune parameters to maximize the
correlation with human judgement. Experiments
show that these enhancements improve
the dependency-based metric's correlation
with human judgement.
Archive Staff Only: edit this record