Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Parser evaluation and the BNC: evaluating 4 constituency parsers with 3 metrics

Foster, Jennifer orcid logoORCID: 0000-0002-7789-4853 and van Genabith, Josef (2008) Parser evaluation and the BNC: evaluating 4 constituency parsers with 3 metrics. In: LREC 2008 - Sixth International Conference on Language Resources and Evaluation, 28-30 May 2008, Marrakech, Morocco.

Abstract
We evaluate discriminative parse reranking and parser self-training on a new English test set using four versions of the Charniak parser and a variety of parser evaluation metrics. The new test set consists of 1,000 hand-corrected British National Corpus parse trees. We directly evaluate parser output using both the Parseval and the Leaf Ancestor metrics. We also convert the hand-corrected and parser output phrase structure trees to dependency trees using a state-of-the-art functional tag labeller and constituent-to-dependency conversion tool, and then calculate label accuracy, unlabelled attachment and labelled attachment scores over the dependency structures. We find that reranking leads to a performance improvement on the new test set (albeit a modest one). We find that self-training using BNC data leads to significantly better results. However, it is not clear how effective self-training is when the training material comes from the North American News Corpus.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:parser;
Subjects:Computer Science > Machine translating
DCU Faculties and Centres:Research Institutes and Centres > National Centre for Language Technology (NCLT)
Official URL:http://www.lrec-conf.org/proceedings/lrec2008/
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Irish Research Council for Science Engineering and Technology, IRCSET P/04/232, Science Foundation Ireland, SFI 04/IN/1527
ID Code:15191
Deposited On:16 Feb 2010 14:17 by DORAS Administrator . Last Modified 10 Oct 2018 15:16
Documents

Full text available as:

[thumbnail of foster_lrec2008.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
75kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record