RELIEF-MM: effective modality weighting for multimedia information retrieval

Yilmaz T., YAZICI A., Kitsuregawa M.

MULTIMEDIA SYSTEMS, cilt.20, sa.4, ss.389-413, 2014 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 20 Sayı: 4
  • Basım Tarihi: 2014
  • Doi Numarası: 10.1007/s00530-014-0360-6
  • Dergi Adı: MULTIMEDIA SYSTEMS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Sayfa Sayıları: ss.389-413
  • Anahtar Kelimeler: RELIEF, Feature weighting, Multimodal fusion, Multimedia information retrieval, FEATURE-SELECTION, FUSION, CLASSIFICATION
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.