Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Dataset creation framework for personalized type-based facet ranking tasks evaluation

Ali, Esraa orcid logoORCID: 0000-0003-1600-3161, Caputo, Annalina orcid logoORCID: 0000-0002-7144-8545, Lawless, Séamus orcid logoORCID: 0000-0001-6302-258X and Conlan, Owen orcid logoORCID: 0000-0002-9054-9747 (2021) Dataset creation framework for personalized type-based facet ranking tasks evaluation. In: 12th International Conference of the CLEF Association, CLEF 2021, 21-24 Sept 2021, Online Event.

Abstract
Faceted Search Systems (FSS) have gained prominence in many existing vertical search systems. They provide facets to assist users in allocating their desired search target quickly. In this paper, we present a framework to generate datasets appropriate for simulation-based evaluation of these systems. We focus on the task of personalized type-based facet ranking. Type-based facets (t-facets) represent the categories of the resources being searched in the FSS. They are usually organized in a large multilevel taxonomy. Personalized t-facet ranking methods aim at identifying and ranking the parts of the taxonomy which reflects query relevance as well as user interests. While evaluation protocols have been developed for facet ranking, the problem of personalising the facet rank based on user profiles has lagged behind due to the lack of appropriate datasets. To fill this gap, this paper introduces a framework to reuse and customise existing real-life data collections. The framework outlines the eligibility criteria and the data structure requirements needed for this task. It also details the process to transform the data into a ground-truth dataset. We apply this framework to two existing data collections in the domain of Point-of-Interest (POI) suggestion. The generated datasets are analysed with respect to the taxonomy richness (variety of types) and user profile diversity and length. In order to experiment with the generated datasets, we combine this framework with a widely adopted simulated user-facet interaction model to evaluate a number of existing personalized t-facet ranking baselines.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:Type-based Facets; Faceted Search; Personalisation; Dataset Collection; Evaluation Framework; Simulated Users
Subjects:Computer Science > Information retrieval
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > ADAPT
Published in: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings. Lecture Notes in Computer Science 12880. Springer.
Publisher:Springer
Official URL:https://dx.doi.org/10.1007/978-3-030-85251-1_3
Copyright Information:© 2021 Springer
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland, SFI 13/RC/2106, Science Foundation Ireland, SFI 13/RC/2106_P2, European Regional Development Fund
ID Code:26371
Deposited On:20 Oct 2021 12:41 by Annalina Caputo . Last Modified 20 Oct 2021 12:41
Documents

Full text available as:

[thumbnail of Eval_TFacet_Ranking_CLEF_CameraReady.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
427kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record