Procrastination is the thief of time: evaluating the effectiveness of proactive search systems
Sen, Procheta, Ganguly, DebasisORCID: 0000-0003-0050-7138 and Jones, Gareth J.F.ORCID: 0000-0003-2923-8365
(2018)
Procrastination is the thief of time: evaluating the effectiveness of proactive search systems.
In: 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, 8-12 July 2018, Ann Arbor, MI, USA.
ISBN 978-1-4503-5657-2
Traditional search system users actively interact with the system
to complete their search task. We believe that the next generation
of search systems will see a shift towards proactive understanding
of user intent based on analysis of user activities. Such a proactive
system could start recommending documents that are likely to help
users accomplish their tasks without requiring them to explicitly
submit queries to the system. We propose a framework to evaluate
such a search system. The key idea behind our proposed metric is to
aggregate a correlation measure over a search session between the
expected outcome, which in this case refers to the list of documents
retrieved with a true user query, and the predicted outcome, which
refers to the list of documents recommended by a proactive search
system. Experiments on the AOL query log data show that the
ranking of two sample proactive IR systems induced by our metric
conforms to the expected ranking between these systems.
Proceedings of The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.
.
Association for Computing Machinery (ACM). ISBN 978-1-4503-5657-2