Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Automated annotation of landmark images using community contributed datasets and web resources

Jones, Gareth J.F. orcid logoORCID: 0000-0003-2923-8365, Byrne, Daragh orcid logoORCID: 0000-0002-2040-9765, Hughes, Mark, O'Connor, Noel E. orcid logoORCID: 0000-0002-4033-9135 and Salway, Andrew (2010) Automated annotation of landmark images using community contributed datasets and web resources. In: The 5th International Conference on Semantic and Digital Media Technologies (SAMT 2010), 1-3 Dec. 2010, Saarbrücken, Germany.

Abstract
A novel solution to the challenge of automatic image annotation is described. Given an image with GPS data of its location of capture, our system returns a semantically-rich annotation comprising tags which both identify the landmark in the image, and provide an interesting fact about it, e.g. "A view of the Eiffel Tower, which was built in 1889 for an international exhibition in Paris". This exploits visual and textual web mining in combination with content-based image analysis and natural language processing. In the first stage, an input image is matched to a set of community contributed images (with keyword tags) on the basis of its GPS information and image classification techniques. The depicted landmark is inferred from the keyword tags for the matched set. The system then takes advantage of the information written about landmarks available on the web at large to extract a fact about the landmark in the image. We report component evaluation results from an implementation of our solution on a mobile device. Image localisation and matching oers 93.6% classication accuracy; the selection of appropriate tags for use in annotation performs well (F1M of 0.59), and it subsequently automatically identies a correct toponym for use in captioning and fact extraction in 69.0% of the tested cases; finally the fact extraction returns an interesting caption in 78% of cases.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:web mining; geo-tagged images; landmark identi�cation; automated image captioning
Subjects:Computer Science > Information retrieval
DCU Faculties and Centres:Research Institutes and Centres > Centre for Digital Video Processing (CDVP)
DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > CLARITY: The Centre for Sensor Web Technologies
ID Code:16437
Deposited On:15 Jul 2011 13:31 by Shane Harper . Last Modified 04 Oct 2018 12:26
Documents

Full text available as:

[thumbnail of Automated_Annotation_of_Landmark_Images_using_Community_Contributed_Datasets_and_Web_Resources.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
1MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record