Nowadays, almost everyone holds some form or other of a personal life archive. Automatically maintaining such an archive is an activity that is becoming increasingly common, however without automatic support the users will quickly be overwhelmed by the volume of data and will miss out on the potential benefits that lifelogs provide. This research is to build an effective and efficient lifelog information retrieval system, which can address the challenges of organising and searching personal life archives, using advanced IR models and ranking approaches. The main contributions of this thesis are as follows. Firstly, lifelog data is defined and a first generation of lifelog datasets are constructed based on our proposed process. Secondly, a baseline search engine is proposed and developed for lifelog data collection, which aims to make the lifelog data searchable, organizable and serves as a baseline comparison for other lifelog retrieval systems that took part in the various benchmarking activities we organised. Thirdly, research on investigating and enriching image concept features using human-object interaction(HOI) annotations are presented and a causality score between different HOIs are calculated and automatic / manual HOI features are compared to validate which feature is most valuable for lifelog moment retrieval. Finally, the approaches presented use pre-trained textual and visual-semantic embeddings for the lifelog retrieval problem and enhance retrieval performance with a high-level semantic mapping of user information. In summary, this thesis primarily contributes to the field of building an effective and efficient lifelog information retrieval system and delivers a number of pioneering contributions.