Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos

Kucuk D., YAZICI A.

KNOWLEDGE-BASED SYSTEMS, vol.24, no.6, pp.844-857, 2011 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 24 Issue: 6
  • Publication Date: 2011
  • Doi Number: 10.1016/j.knosys.2011.03.006
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Page Numbers: pp.844-857
  • Keywords: Semantic video annotation, News video retrieval, Information extraction, Text mining, Video indexing, TEXT, RETRIEVAL
  • Middle East Technical University Affiliated: Yes


This paper targets at the problem of automatic semantic indexing of news videos by presenting a video annotation and retrieval system which is able to perform automatic semantic annotation of news video archives and provide access to the archives via these annotations. The presented system relies on the video texts as the information source and exploits several information extraction techniques on these texts to arrive at representative semantic information regarding the underlying videos. These techniques include named entity recognition, person entity extraction, coreference resolution, and semantic event extraction. Apart from the information extraction components, the proposed system also encompasses modules for news story segmentation, text extraction, and video retrieval along with a news video database to make it a full-fledged system to be employed in practical settings. The proposed system is a generic one employing a wide range of techniques to automate the semantic video indexing process and to bridge the semantic gap between what can be automatically extracted from videos and what people perceive as the video semantics. Based on the proposed system, a novel automatic semantic annotation and retrieval system is built for Turkish and evaluated on a broadcast news video collection, providing evidence for its feasibility and convenience for news videos with a satisfactory overall performance. (C) 2011 Elsevier B.V. All rights reserved.