Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Audio-visual query and retrieval: A system that uses dynamic programming and relevance feedback

Audio-visual query and retrieval: A system that uses dynamic programming and relevance feedback A necessary capability for content-based retrieval is to support the paradigm of query by example. In the past, there have been several attempts to use low-level features for video retrieval. However, most of the approaches support queries using image sequences only. We present an algorithm for matching multimodal (audio-visual) patterns for the purpose of content-based video retrieval. The novel ability of our approach to use the information content in multiple media coupled with a strong emphasis on temporal similarity differentiates it from the state of the art in content-based retrieval. At the core of the pattern matching scheme is a dynamic programming algorithm, which leads to a significant improvement in performance. Coupling the use of audio with video this algorithm can be applied to grouping of shots based on audio-visual similarity. We also support relevance feedback. The user can provide feedback to the system, by choosing clips, which are closer to the user’s desired target. The system then automatically adjusts the relative weights or relevance of the media and fetches different sets of target clips accordingly. It is our observation that a few iterations of such feedback are generally sufficient, for retrieving the desired video clips. © 2001 SPIE and IS&T. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Electronic Imaging SPIE

Audio-visual query and retrieval: A system that uses dynamic programming and relevance feedback

Loading next page...
 
/lp/spie/audio-visual-query-and-retrieval-a-system-that-uses-dynamic-xUbAHameaR

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
SPIE
Copyright
Copyright © 2001 SPIE and IS&T
ISSN
1017-9909
eISSN
1560-229X
DOI
10.1117/1.1407822
Publisher site
See Article on Publisher Site

Abstract

A necessary capability for content-based retrieval is to support the paradigm of query by example. In the past, there have been several attempts to use low-level features for video retrieval. However, most of the approaches support queries using image sequences only. We present an algorithm for matching multimodal (audio-visual) patterns for the purpose of content-based video retrieval. The novel ability of our approach to use the information content in multiple media coupled with a strong emphasis on temporal similarity differentiates it from the state of the art in content-based retrieval. At the core of the pattern matching scheme is a dynamic programming algorithm, which leads to a significant improvement in performance. Coupling the use of audio with video this algorithm can be applied to grouping of shots based on audio-visual similarity. We also support relevance feedback. The user can provide feedback to the system, by choosing clips, which are closer to the user’s desired target. The system then automatically adjusts the relative weights or relevance of the media and fetches different sets of target clips accordingly. It is our observation that a few iterations of such feedback are generally sufficient, for retrieving the desired video clips. © 2001 SPIE and IS&T.

Journal

Journal of Electronic ImagingSPIE

Published: Oct 1, 2001

There are no references for this article.