Current Research

Contents Understanding of Large-scale Broadcast Video Data Collection

1.Overview
 Various media for obtaining and to distributing video streams have been providing our society with a huge amount of video data. It is necessary to analyze them and exploit these data to support a wide-range of human activities. However, it is difficult to apply conventional contents analysis or pattern recognition methods in handling large-scale collections. We are thus working on supporting efficient and effective understanding of video contents by providing retrieval and browsing abilities to users supported by knowledge extraction from a large-scale video data collection
2.Research theme
 Among various genres, we are focusing on broadcast news video, which could be considered a common media heritage of human society that records its detailed activities. These video data, however, have long been in the danger of disintegration, until fairly recent efforts have been made to store them in archives. Even now, research on methods and tools to retrieve specific video data or to extract valuable information from massive amounts of video data is under way.
 In order to cope with these problems, We have been working on several hundreds of hours of broadcast news video related to the following tasks :
・Extracting news topic thread structures
 A “Topic thread” structure that represents both chronological and semantic relations between news stories is automatically extracted in order to track the development of a news topic in focus. By constructing a browsing interface based on this structure, users are provided with the ability to efficiently and effectively understand the topic in focus.
・Assembling monologue collections
 The merit of news video over text (newspapers and magazines) or audio (radio) is that it contains nonverbal visual information, e.g. facial expressions and the mood of the person in a monologue scene, such as speech or interview scenes. We are thus working on automatically detecting monologue scenes by referring to multimodal cues in the video data.
・Extracting human relationships in news
 We are working on extraction of human relationships from news stories according to the co-occurrences of news subjects, together with better understanding of news contents based on such knowledge.
 In addition to these developments, we are also interested in pattern recognition methods in image, text, and audio data to support them.
3.Future works
 Systems that respond promptly to a user’s query, such as automatic documentary generation on a specified topic, are also under consideration.
4.Other works
 We have been working on other video data besides news video :
・Video data from car-mounted cameras : Driver support, Creation of a cityscape database
・Cooking shows : Indexing cooking steps, Application for cooking support
・Soccer shows : Tracking of players on the field
・Soap-operas : Indexing of characters’ activities
 Apart from work in this field, we have also worked on the following theme :
・Designing an infrastructure for academic information distribution : Designing a researcher's personal information management system