Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Information Lifelogging: Leveraging Eye Movements and Reading Comprehension for Efficient Retrieval of Previously Encountered On-Screen Information

Le, Tu-Khiem orcid logoORCID: 0000-0003-3013-9380 (2024) Information Lifelogging: Leveraging Eye Movements and Reading Comprehension for Efficient Retrieval of Previously Encountered On-Screen Information. PhD thesis, Dublin City University.

Abstract
The progress of lifelog research has enabled individuals to comprehensively capture their daily experiences. As a result, previous studies have primarily focused on developing tools to organise and retrieve lifelog moments effectively. However, existing lifelog data often lacks the ability to capture the lifelogger’s focal points (their attention), despite providing information-rich first-person-view lifelog images of their surroundings and activities. Consequently, this limitation hinders the lifelog retrieval systems’ utility when lifeloggers seek to retrieve specific information they have previously encountered. To address this research gap, a subjective point of view, represented through eye movements, should be incorporated as a new modality into lifelog data, thereby enhancing the retrieval performance. In pursuit of this objective, this dissertation investigates the feasibility of retriving on-screen information by analysing lifelogger’s reading activities and comprehension level. The primary contributions of this dissertation are as follows. Firstly, the development of LifeSeeker, an advanced interactive lifelog retrieval system, is developed and benchmarked in numerous lifelog retrieval challenges and competitions. By efficiently integrating various modality processing components (e.g., visual, text, location, biometrics) and user interaction components (e.g., search, filtering, browsing, relevance feedback) into a single interactive retrieval framework, LifeSeeker ranked among the top systems in these benchmarking activities, serving as the foundation for the rest of the thesis. Secondly, a novel reading comprehension dataset was created to explore the feasibility of recognising reading activities and estimating reading comprehension levels in daily life. Statistical tests and machine learning analyses on the dataset have revealed the strong connection between eye movement patterns, reading conditions, and reading comprehension. This led to a novel method for estimating reading comprehension with potential real-world applications. Furthermore, the longitudinal aspect of reading comprehension was investigated to examine the stability and generalisation of reading comprehension estimation models over time. Lastly, the reading comprehension estimation model was integrated into LifeSeeker as a new modality processor, resulting in a significant improvement in the system’s overall retrieval performance. In summary, this dissertation contributes to the understanding of reading activities and reading comprehension in real-world settings and showcases the potential of integrating reading comprehension estimation to enhance the retrieval of previously encountered information in lifelog data.
Metadata
Item Type:Thesis (PhD)
Date of Award:August 2024
Refereed:No
Supervisor(s):Gurrin, Cathal
Uncontrolled Keywords:Eye Movements, Eye Tracking, Reading Analytics, Reading Comprehension
Subjects:Computer Science > Artificial intelligence
Computer Science > Information retrieval
Computer Science > Interactive computer systems
Computer Science > Machine learning
Computer Science > Multimedia systems
Computer Science > Information storage and retrieval systems
Computer Science > Lifelog
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 License. View License
Funders:Science Foundation Ireland under grant numbers SFI/12/RC/2289_P2
ID Code:30172
Deposited On:18 Nov 2024 11:37 by Tu-Khiem Le . Last Modified 18 Nov 2024 11:37
Documents

Full text available as:

[thumbnail of Khiem_Thesis_Final.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0
11MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record