Scholars Central

Conference abstractPublished 2018

Modeling human visual attention in multimedia indexing and retrieval

Authors: Jenny Benois Pineau

LABRI (UMR 5800) - University of Bordeaux, France

Ask about this article

Abstract

Select any words in the title or abstract, or press this and choose a sentence.

Since the beginning of the era of multimedia indexing and retrieval, the research communities in computer vision and in multimedia worked in parallel to some extent on the same problem: How can we incorporate our knowledge on human understanding of visual scenes into pattern recognition, visual information analysis, indexing and retrieval. It is quite natural to seek for driving visual content mining methods towards regions and areas which attract human attention. The first classical model to be combined with prediction of saliency was the bag-of-visual-words (BoVW), used as a signature for images and video retrieval, object and concept recognition in visual content. From simple weighting of BoVW to a non-uniform sampling and psychovisual filtering of the content, the incorporation of models of human visual system in content description, retrieval and classification has allowed obtaining very competitive results without heavy window-based scanning in training of models and in generalization. The most popular visual saliency models proposed for this purpose still remained those by L Itti and J Harel, but for specific kind of content such as wearable multimedia, specific models of prediction of visual attention had to be designed. With the adventure of powerful classification models such as deep neural networks (Deep NN) prediction of visual saliency is being passing from ad-hoc a priory mastering of local features to fit Treisman’s and Gelade’s feature integration theory for visual attention, to a massive supervised learning approach, requiring understanding of inherent bottlenecks of these tools in terms of availability- and noise in training data, of initialization, of optimization algorithms and knowledge transfer. In the talk we will give a vast panorama of the topic including examples in various application domains of multimedia indexing and retrieval. We will present applications of human attention use in human assistance for wearers of neuro-prostheses.

Ask about this article

Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.

Try:

More from this journal

The newest in Journal of Computer Engineering & Information Technology.

All its articles

Conferences in this field

Upcoming conferences in this field.

Follow

New articles from what you follow go into your email digest, once you agree to alert emails.

  • JournalJournal of Computer Engineering & Information Technology
  • SubjectComputer Science & IT
  • SubjectEngineering
  • BylineArticles whose byline prints “Jenny Benois Pineau”

For authors and organisers

Publish, present and keep up with your field

Find the right journal for your paper and submit your manuscript.

Photo: Pexels