Task 2: ShARe/CLEF eHealth evaluation lab 2014
Mowery, Danielle L. and Velupillai, Sumithra and South, Brett R. and Christensen, Lee and Martinez, David and Kelly, Liadh and Goeuriot, Lorraine and Elhadad, Noemie and Pradhan, Sameer and Savova, Guergana and Chapman, Wendy (2014) Task 2: ShARe/CLEF eHealth evaluation lab 2014. CLEF 2014 Online Working Notes, 1180 . pp. 31-42. ISSN 1613-0073
Full text available as:
This paper reports on Task 2 of the 2014 ShARe/CLEF
eHealth evaluation lab which extended Task 1 of the 2013 ShARe/CLEF eHealth evaluation lab by focusing on template lling of disorder attributes. The task was comprised of two subtasks: attribute normalization (task 2a) and cue identication (task 2b).We instructed participants
to develop a system which either kept or updated a default attribute value for each task. Participant systems were evaluated against a blind reference standard of 133 discharge summaries using Accuracy (task 2a)
and F-score (task 2b). In total, ten teams participated in task 2a, and three teams in task 2b. For task 2a and 2b, the HITACHI team systems (run 2) had the highest performances, with an overall average average accuracy of 0.868 and F1-score (strict) of 0.676, respectively.
Archive Staff Only: edit this record