ISSN : 1796-2048
Volume : 4    Issue : 5    Date : October 2009

Using the MPEG Query Format for Cross-Modal Identification
Matthias Gruhne, Peter Dunker, and Ruben Tous
Page(s): 266-276
Full Text:
PDF (1,136 KB)

During the last years a vast number of multimedia databases have been established for search and
retrieval of multimedia data, due to large audiovisual storage capacities and efficient compression
methods. The metadata format of such databases can be based on the MPEG-7 standard. For the
successful and efficient search, a query language for the interaction between client and database is
crucial. One of the latest developments of the MPEG committee is the MPEG Query Format (MPQF)
which is destined for the interaction between clients and MPEG-7 databases. The client may be
interested to express more complex queries, which requires an interaction with several databases.
Therefore an additional service provider can be used in the according environment, which accepts
and understands a query formulation from a client and forwards parts of the query to one or more
databases. The service provider furthermore retrieves the responses from these databases,
postprocesses these results and replies a combined result list to the client.
During the last years, the cross-modal search of visual and audio information has become more
and more important. Using both domains, video and audio, turned out to be much more robust for
the identification of video streams, than the visual part of the video stream alone. This paper
describes a method for the audiovisual identification on remote databases using the MPQF.
Additionally a service provider is deployed, which splits and aggregates the query and send them to
two remote MPEG-7 databases (visual and audio) for identification. Among others a novel technique
for the feature extraction on the service provider side is described, which is based on the MPQF. The
interface between user and database is described in detail, examples are given and extensive
results for the cross-modal search are presented.

Index Terms
cross-modal identification, MPQF, audiovisual identification, search and retrieval