Interactive experiments in object-based retrieval

Abstract

Object-based retrieval is a modality for video retrieval based on segmenting objects from video and allowing end-users to use these objects as part of querying. This uses similarity between query objects and objects appearing in the video, and in theory allows retrieval based on what is actually appearing on-screen. We conducted an empirical TRECVid-like evaluation of object-based search in an interactive search experiment with 24 search topics and 16 users each performing 12 search tasks on 50 hours of video. This was done in an attempt to measure the impact of object-based search on annotation-free video where text from automatic speech recognition (ASR), from video OCR, or from closed captions is not available.