Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Conversational search for image and video with augmented labelling

Potyagalova, Anastasia (2025) Conversational search for image and video with augmented labelling. PhD thesis, Dublin City University.

Abstract
The rapid growth of media archives—including text, speech, video, and audio—has driven strong interest in developing advanced search methods for multimedia content. In particular, conversational search has emerged as a promising approach, where users engage in a dialogue with an AI agent to support and enhance their search activities. While most existing systems focus on text-based archives, this research extends conversational search methods to image and video retrieval. Our approach involves developing an experimental framework to explore how conversational engagement can improve multimedia search. We introduce a prototype system that combines dialogue-based interaction with state-of-the-art visual indexing techniques. While multimedia information retrieval (MIR) has long been studied through conventional user-driven interfaces, the integration of conversational agents introduces a new layer of interactivity. The agent aims to assist users by suggesting relevant content and helping to filter out irrelevant results. Effective dialogue in this context requires the agent to demonstrate an understanding of the content and its relevance to the user’s needs. Although MIR techniques have advanced significantly, little attention has been paid to how retrieved content is represented and communicated during the search process. Our system addresses this gap by incorporating object detection to highlight key visual features, enhancing both the accuracy and contextual relevance of search results. To evaluate the framework, we conducted three user studies focused on the effectiveness of conversational engagement in multimedia search. These studies examined how AI-driven dialogue affects users’ ability to retrieve relevant image and video content and improves the overall search experience. Results indicate that conversational interaction not only refines retrieval accuracy but also increases user satisfaction by creating a more intuitive and responsive search environment.
Metadata
Item Type:Thesis (PhD)
Date of Award:24 June 2025
Refereed:No
Supervisor(s):Jones, Gareth
Uncontrolled Keywords:Conversational search; conversational image search; conversational video search; augmented image and video labelling
Subjects:Computer Science > Artificial intelligence
Computer Science > Image processing
Computer Science > Information retrieval
Computer Science > Interactive computer systems
Computer Science > Multimedia systems
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing
DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 License. View License
Funders:Science Foundation Ireland
ID Code:31207
Deposited On:21 Nov 2025 14:07 by Gareth Jones . Last Modified 21 Nov 2025 14:07
Documents

Full text available as:

[thumbnail of Thesis_final_corrected_sign.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0
12MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record