Fast gated neural domain adaptation: language model as a case study
Zhang, Jian, Wu, Xiaofeng, Way, AndyORCID: 0000-0001-5736-5930 and Liu, QunORCID: 0000-0002-7000-1792
(2017)
Fast gated neural domain adaptation: language model as a case study.
In: Proceedings of FETLT 2016: Future and Emerging Trends in Language Technologies, Machine Learning and Big Data, 30 Nov- 2 Dec 2016, Seville, Spain.
Neural network training has been shown to be advantageous in many natural language processing
applications, such as language modelling or machine translation. In this paper, we describe in
detail a novel domain adaptation mechanism in neural network training. Instead of learning
and adapting the neural network on millions of training sentences – which can be very timeconsuming or even infeasible in some cases – we design a domain adaptation gating mechanism
which can be used in recurrent neural networks and quickly learn the out-of-domain knowledge
directly from the word vector representations with little speed overhead. In our experiments,
we use the recurrent neural network language model (LM) as a case study. We show that the
neural LM perplexity can be reduced by 7.395 and 12.011 using the proposed domain adaptation
mechanism on the Penn Treebank and News data, respectively. Furthermore, we show that using
the domain-adapted neural LM to re-rank the statistical machine translation n-best list on the
French-to-English language pair can significantly improve translation quality
Proceedings of FETLT 2016: Future and Emerging Trends in Language Technologies, Machine Learning and Big Data.
.
Association for Computational Linguistics.
This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:
ADAPT Centre for Digital Content Technology (www.adaptcentre.ie) at Dublin City University is funded under the Science Foundation Ireland Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
ID Code:
23233
Deposited On:
02 May 2019 12:28 by
Thomas Murtagh
. Last Modified 27 Apr 2023 11:28