Efficiency and effectiveness of query processing in cluster-based retrieval

Creative Commons License

Can F., Altingovde İ. S., Demir E.

INFORMATION SYSTEMS, cilt.29, sa.8, ss.697-717, 2004 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 29 Sayı: 8
  • Basım Tarihi: 2004
  • Doi Numarası: 10.1016/s0306-4379(03)00062-0
  • Dergi Adı: INFORMATION SYSTEMS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Sayfa Sayıları: ss.697-717
  • Anahtar Kelimeler: clustering, cluster-based retrieval, information retrieval, performance, query processing, DOCUMENT-RETRIEVAL, TEXT RETRIEVAL, INFORMATION, DATABASES, SYSTEMS, ALGORITHMS
  • Orta Doğu Teknik Üniversitesi Adresli: Hayır

Özet

Our research shows that for large databases, without considerable additional storage overhead, cluster-based retrieval (CBR) can compete with the time efficiency and effectiveness of the inverted index-based full search (FS). The proposed CBR method employs a storage structure that blends the cluster membership information into the inverted file posting lists. This approach significantly reduces the cost of similarity calculations for document ranking during query processing and improves efficiency. For example, in terms of in-memory computations, our new approach can reduce query processing time to 39% of FS. The experiments confirm that the approach is scalable and system performance improves with increasing database size. In the experiments, we use the cover coefficient-based clustering methodology ((CM)-M-3), and the Financial Times database of TREC containing 210 158 documents of size 564 MB defined by 229 748 terms with total of 29 545 234 inverted index elements. This study provides CBR efficiency and effectiveness experiments using the largest corpus in an environment that employs no user interaction or user behavior assumption for clustering. (C) 2003 Elsevier Ltd. All rights reserved.