We present a novel machine learning based approach to de- termining the semantic relevance of community contributed image annotations for the purposes of image retrieval. Cur- rent large scale community image retrieval systems typically rely on human annotated tags which are subjectively assigned and may not provide useful or semantically meaningful la- bels to the images. Homogeneous tags which fail to distin- guish between are a common occurrence, which can lead to poor search effectiveness on this data. We described a method to improve text based image retrieval systems by eliminating generic or non relevant image tags. To classify tag relevance, we propose a novel feature set based on statistical information available for each tag within a collection of geotagged images harvested from Flickr. Using this feature set machine learning models are trained to classify the relevance of each tag to its associated image. The goal of this process is to allow for rich and accurate captioning of these images, with the objective of improving the accuracy of text based image retrieval systems. A thorough evaluation is carried out using a human annotated benchmark collection of Flickr tags.