Segmenting broadcast news streams using lexical chains
Stokes, Nicola and Carthy, Joe and Smeaton, Alan F. (2002) Segmenting broadcast news streams using lexical chains. In: STAIRS 2002 - STarting Artificial Intelligence Researchers Symposium, 22-23 July 2002, Lyon, France.
Full text available as:
In this paper we propose a course-grained NLP approach to text segmentation based on the
analysis of lexical cohesion within text. Most work in this area has focused on the discovery of textual
units that discuss subtopic structure within documents. In contrast our segmentation task requires the discovery of topical units of text i.e. distinct news stories from broadcast news programmes. Our system SeLeCT first builds a set of lexical chains, in order to model the discourse structure of the text. A boundary detector is then used to search for breaking points in this structure indicated by patterns of cohesive strength and weakness within the text. We evaluate this technique on a test set of concatenated CNN news story transcripts and compare it with an established statistical approach to segmentation called TextTiling.
Archive Staff Only: edit this record