Multiplex Video

MEDIA SYNCHRONIZATION METHOD FOR VIDEO HYPERMEDIA APPLICATION BASED ON EXTENDED EVENT MODEL Hironobu Abe1, Hiroshi Shigeno2, Ken-ichi Okada2
1

Mitsubishi Electric Corporation, 2Keio University
the broadcasting type without time information in the video targeted with this paper. This system periodically inserts event information which defines time relationship between video and metadata inthe video multiplex, and establishes media synchronized delivery between video and metadata. We implemented a system prototype considering the performance of synchronization accuracy, because handling event processing has a large impact on the CPU load. And we evaluated real time synchronization performance of the system prototype. 2. EXTENDED EVENT MODEL 2.1. Design concept In this paper, wepropose extending the method of event inserting in video as a design concept of media synchronization between video and metadata. Events are independent of the video format and delivery method, because they can be inserted at any point of the video stream. Media synchronization is accomplished by inserting information required for synchronized delivery of video and metadata. Media synchronization isalso possible for the case of broadcasting video over multicast technology, because synchronization information is inserted in the video stream. Also, depending on the intended use, event information can be inserted periodically to achieve start point and real time synchronization of video and metadata. 2.2. Metadata model Spatio-Temporal Locator[2] defined in MPEG-7 Visual is a metadata model usedfor video hypermedia applications. But since the description cost of this model is high, we choose as data model for our research the model described in [3]. Figure 1 shows an overview of the metadata. In our model, metadata in the video stream does not necessarily contain size and position data in every frame, it only contains data in the frames we call start point, end point, and intermediatepoints. Frames which

ABSTRACT This paper describes a proposal of an extended event model using a media synchronization method for video hypermedia applications. In this extended event model, video and metadata are synchronized by periodically inserting event information in the video multiplex. We considered the following design policies: 1) a model that is independent of the video format anddelivery method, 2) the synchronization accuracy can be tuned depending on the purpose and use of the metadata. We designed the extended event model based on the above design policies, and implemented this model as an encode/decode library for Windows Media. Based on this model we developed a video hypermedia system prototype and performed evaluation experiments. The evaluation results of real timesynchronization performance of the system prototype showed that in the case of sports video content a synchronization accuracy of 100 msec between video and metadata, makes our method effective for use in video hypermedia applications. 1. INTRODUCTION Recently, the rapid proliferation of broadband Internet access has enabled high-speed Internet connections possible. As a consequence of thisbroadbandization, delivery of multimedia information such as voice and video is possible. In this paper, we define metadata that combine video and related information for the realization of video hypermedia applications using streaming technologies, and design a media synchronization method of video and metadata, and describe the implementation and evaluation of our system prototype. System requirements ofvideo hypermedia application in this paper are, 1) a model that is independent of the video format and delivery method, 2) the synchronization accuracy can be tuned depending on the purpose and use of the metadata. Because it is a method in existing media synchronization method such as SMIL[1] to describe time information in the script as synchronization information with the metadata, it is not...