Effective & efficient methods for web search result diversification


Tezin Türü: Doktora

Tezin Yürütüldüğü Kurum: Orta Doğu Teknik Üniversitesi, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü, Türkiye

Tezin Onay Tarihi: 2015

Öğrenci: AHMET MURAT ÖZDEMİRAY

Danışman: İSMAİL SENGÖR ALTINGÖVDE

Özet:

Search result diversification is one of the key techniques to cope with the ambiguous and/or underspecified information needs of the web users. In this study we first extensively evaluate the performance of a state-of-the-art explicit diversification strategy and pin-point its weaknesses. We propose basic yet novel optimizations to remedy these weaknesses and boost the performance of this algorithm. Secondly, we cast the diversification problem to the problem of ranking aggregation and propose to materialize the re-rankings of the candidate documents for each query aspect and then merge these rankings by adapting the score(-based) and rank(-based) aggregation methods. As a third contribution, for the first time in the literature, we propose using post-retrieval query performance predictors (QPPs) to estimate, for each aspect, the retrieval effectiveness on the candidate document set, and leverage these estimations to set the aspect weights. In addition to utilizing well-known QPPs from the literature, we also introduce three new QPPs that are based on score distributions and hence, can be employed for online query processing in real-life search engines. For the last contribution, we use retrieval performance predictions of query aspects to selectively expand those aspects that perform below some threshold, using the top retrieved documents of the aspect's own results. Our extensive experimental evaluations show that, despite having lower computational complexity than the state-of-the-art diversification strategies, certain ranking aggregation methods are superior to the existing explicit diversification strategies in terms of the diversification effectiveness. Furthermore, using QPPs for aspect weighting improves almost all state-of-the-art diversification algorithms in comparison to using a uniform weight estimator and also the proposed QPPs are comparable or superior to the existing predictors in the context of aspect weighting. Lastly, using QPP methods to selectively expand the query aspects provide better diversification performance compared to unexpanded or fully expanded aspects, for most of the diversification strategies.