14 February 2013
Hybrid Models for Visual and Textual Systems' Combinations in the Context of User Feedback
Meeting Room 10, 2nd Floor, JLB
12:30pm - 13:45pm
The biggest challenge in Multimedia Retrieval is how to reduce the "semantic gap", the difference between human perception and machine representation of an information object (image/music track/video). The aforementioned gap can be reduced by improving multimedia content representation (higher level features) and combining multimedia content features with textual features. Such combination can improve the retrieval effectiveness because multimedia content and textual features represent complementary yet correlated aspects of the same information object. Moreover, semantic gap can be further reduced by user profiling, which can narrow down and personalize the search. Multimedia content and textual features can be also combined in the context of user feedback.
In this talk, I am going to present my work on the prototype Content-Based Image Retrieval (CBIR) System. We will start by introducing some basic Information Retrieval (IR) concepts (i.e. "vector space model") and current state-of-the-art in visual IR (higher level features). Novel visual features with hybrid sampling, designed for generic image retrieval will be presented next. We will also look at some applications of our visual features to Food Recognition and Music Information Retrieval.
Analogies can be found between Quantum Mechanics (QM) and IR, and mathematical tools utilized in QM are well suited for IR modelling. Moreover, Hilbert spaces and their subspaces utilized in QM can be thought of as generalization and extension of vector spaces. Thus, I am going to present the Quantum Theory Inspired Image Retrieval Framework consisting of Image Auto-Annotation, combination of visual and textual features and combination of aforementioned features in the context of user feedback. Finally, we will focus on a recently discovered duality of fusion strategies and see how query modification models can be represented as combinations of scores computed on individual feature spaces. One of the implications is, for example, that clustering of tensored vectors can be performed without actual tensoring operation.
Some of the introduced models were implemented in our prototype CBIR system with interactive user interface and will be also presented as a working demo.
Leszek Kaliciak is a visiting PhD student from the Robert Gordon University, Aberdeen. He works under the supervision of Prof. Dawei Song. Leszek did his undergraduate and Master degree in pure as well as applied mathematics. His research interests include multimedia retrieval, image processing, chaos theory and long and boring mathematical formulas. He likes going out for drinks after seminars.
Save to your Calendar