Le, Tu-Khiem ORCID: 0000-0003-3013-9380 (2024) Information Lifelogging: Leveraging Eye Movements and Reading Comprehension for Efficient Retrieval of Previously Encountered On-Screen Information. PhD thesis, Dublin City University.
Abstract
The progress of lifelog research has enabled individuals to comprehensively capture their daily experiences. As a result, previous studies have primarily focused on developing tools to organise and retrieve lifelog moments effectively. However, existing lifelog data often lacks the ability to capture the lifelogger’s focal points (their attention), despite providing information-rich first-person-view lifelog images of their surroundings and activities. Consequently, this limitation hinders the lifelog retrieval systems’ utility when lifeloggers seek to retrieve specific information they have previously encountered. To address this research gap, a subjective point of view, represented through eye movements, should be incorporated as a new modality into lifelog data, thereby enhancing the retrieval performance. In pursuit of this objective, this dissertation investigates the feasibility of retriving on-screen information by analysing lifelogger’s reading activities and comprehension level.
The primary contributions of this dissertation are as follows. Firstly, the development of LifeSeeker, an advanced interactive lifelog retrieval system, is developed and benchmarked in numerous lifelog retrieval challenges and competitions. By efficiently integrating various modality processing components (e.g., visual, text, location, biometrics) and user interaction components (e.g., search, filtering, browsing, relevance feedback) into a single interactive retrieval framework, LifeSeeker ranked among the top systems in these benchmarking activities, serving as the foundation for the rest of the thesis. Secondly, a novel reading comprehension dataset was created to explore the feasibility of recognising reading activities and estimating reading comprehension levels in daily life. Statistical tests and machine learning analyses on the dataset have revealed the strong connection between eye movement patterns, reading conditions, and reading comprehension. This led to a novel method for estimating reading comprehension with potential real-world applications. Furthermore, the longitudinal aspect of reading comprehension was investigated to examine the stability and generalisation of reading comprehension estimation models over time. Lastly, the reading comprehension estimation model was integrated into LifeSeeker as a new modality processor, resulting in a significant improvement in the system’s overall retrieval performance. In summary, this dissertation contributes to the understanding of reading activities and reading comprehension in real-world settings and showcases the potential of integrating reading comprehension estimation to enhance the retrieval of previously encountered information in lifelog data.
Metadata
Item Type: | Thesis (PhD) |
---|---|
Date of Award: | August 2024 |
Refereed: | No |
Supervisor(s): | Gurrin, Cathal |
Uncontrolled Keywords: | Eye Movements, Eye Tracking, Reading Analytics, Reading Comprehension |
Subjects: | Computer Science > Artificial intelligence Computer Science > Information retrieval Computer Science > Interactive computer systems Computer Science > Machine learning Computer Science > Multimedia systems Computer Science > Information storage and retrieval systems Computer Science > Lifelog |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Institutes and Centres > INSIGHT Centre for Data Analytics |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 License. View License |
Funders: | Science Foundation Ireland under grant numbers SFI/12/RC/2289_P2 |
ID Code: | 30172 |
Deposited On: | 18 Nov 2024 11:37 by Tu-Khiem Le . Last Modified 18 Nov 2024 11:37 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-No Derivative Works 4.0 11MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record