Ali, Esraa ORCID: 0000-0003-1600-3161, Caputo, Annalina ORCID: 0000-0002-7144-8545, Lawless, Séamus ORCID: 0000-0001-6302-258X and Conlan, Owen ORCID: 0000-0002-9054-9747 (2021) Dataset creation framework for personalized type-based facet ranking tasks evaluation. In: 12th International Conference of the CLEF Association, CLEF 2021, 21-24 Sept 2021, Online Event.
Abstract
Faceted Search Systems (FSS) have gained prominence in
many existing vertical search systems. They provide facets to assist users in allocating their desired search target quickly. In this paper, we present a framework to generate datasets appropriate for simulation-based evaluation of these systems. We focus on the task of personalized type-based facet ranking. Type-based facets (t-facets) represent the categories of the resources being searched in the FSS. They are usually organized in a large multilevel taxonomy. Personalized t-facet ranking methods aim at identifying and ranking the parts of the taxonomy which reflects query relevance as well as user interests. While evaluation protocols have been developed for facet ranking, the problem of personalising the facet rank based on user profiles has lagged behind due to the lack of appropriate
datasets. To fill this gap, this paper introduces a framework to reuse and customise existing real-life data collections. The framework outlines the eligibility criteria and the data structure requirements needed for this task. It also details the process to transform the data into a ground-truth dataset. We apply this framework to two existing data collections in the domain of Point-of-Interest (POI) suggestion. The generated datasets
are analysed with respect to the taxonomy richness (variety of types) and user profile diversity and length. In order to experiment with the generated datasets, we combine this framework with a widely adopted simulated user-facet interaction model to evaluate a number of existing personalized t-facet ranking baselines.
Metadata
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Event Type: | Conference |
Refereed: | Yes |
Uncontrolled Keywords: | Type-based Facets; Faceted Search; Personalisation; Dataset Collection; Evaluation Framework; Simulated Users |
Subjects: | Computer Science > Information retrieval |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Institutes and Centres > ADAPT |
Published in: | Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings. Lecture Notes in Computer Science 12880. Springer. |
Publisher: | Springer |
Official URL: | https://dx.doi.org/10.1007/978-3-030-85251-1_3 |
Copyright Information: | © 2021 Springer |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | Science Foundation Ireland, SFI 13/RC/2106, Science Foundation Ireland, SFI 13/RC/2106_P2, European Regional Development Fund |
ID Code: | 26371 |
Deposited On: | 20 Oct 2021 12:41 by Annalina Caputo . Last Modified 20 Oct 2021 12:41 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
427kB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record