The performance of automatic detection of concepts in im- age and video data has been improved to a satisfactory level for some generic concepts like indoor, outdoor, faces, etc. on high quality data from broadcast TV or movies. How- ever it remains a challenge to apply this to interpreting the high-level semantics of events as they occur in visual lifelogs from wearable cameras. This is because poorer quality im- age data and the activities of the wearer make it difficult to automatically categorise them. In this paper, we propose an interestingness-based semantic aggregation and representa- tion algorithm, to tackle the problem of event management and representation in visual lifelogging. Semantic concept interestingness is calculated by fusing image-level concepts which are then exploited to select a representation for the semantic event correlated to various event topics. Experi- mental results show the efficacy of our algorithm in fusing semantics at the event level, and in selecting representations for event management in visual lifelogging.