Lifelog, a personal digital record of daily activities, is becoming popular and can operate as a form of a digital diary. Although having a wide range of applications, the basic underlying challenge of lifelogs is how to build a lifelog retrieval system that can retrieve a specific activity efficiently from large multimodal lifelog data. Recently, there have been many workshops organized to encourage researchers to solve that problem leading to the introduction of many lifelog retrieval systems with different approaches. In addition to the user interface, the backend search engine plays a critical role in the lifelog retrieval system. The fundamental approach for the lifelog search engine is the concept-based model which matches the objects in lifelog images with keywords in a semantic query. However, many state-of-the-art embedding multimodal retrieval models, such as CLIP, have been introduced recently where they encode both images and texts into the same vector space to perform the retrieval. This has opened the opportunity to apply this new generation search engine to the lifelog retrieval task. In this thesis, I propose applying embedding- based models to the lifelog retrieval system. Furthermore, I enhance these models by applying a graph-based structure to them. I also build an interactive lifelog retrieval system that follows the concept-based approach to serve as the baseline in my experiment. Through my interactive user studies, I observe that the graph- based enhanced retrieval model surpasses the conventional concept-based by a large margin in various evaluation metrics. In summary, this thesis primarily contributes to the field of creating an accurate lifelog search engine using graph neural network structures that can be employed to build an effective and efficient lifelog retrieval system to address lifelog retrieval tasks.