Learning to Rank for Educational Search Engines

Usta A., ALTINGÖVDE İ. S. , Ozcan R., Ulusoy O.

IEEE Transactions on Learning Technologies, vol.14, no.2, pp.211-225, 2021 (Peer-Reviewed Journal) identifier identifier

  • Publication Type: Article / Article
  • Volume: 14 Issue: 2
  • Publication Date: 2021
  • Doi Number: 10.1109/tlt.2021.3075196
  • Journal Name: IEEE Transactions on Learning Technologies
  • Journal Indexes: Science Citation Index Expanded, Social Sciences Citation Index, Scopus, Compendex, ERIC (Education Resources Information Center), INSPEC, Psycinfo
  • Page Numbers: pp.211-225
  • Keywords: Search engines, Engines, Web search, Education, Context modeling, Task analysis, Feature extraction, Educational search, learning to rank (LTR), query-dependent ranking, search engines, INFORMATION-RETRIEVAL, METRICS


IEEEIn this digital age, there is an abundance of online educational materials in public and proprietary platforms. To allow effective retrieval of educational resources, it is a necessity to build keyword-based search engines over these collections. In modern web search engines, high quality rankings are obtained by applying machine learning techniques, known as Learning to Rank (LTR). In this paper, our focus is on constructing machine-learned ranking models to be employed in a search engine in the education domain. Our contributions are threefold. First, we identify and analyze a rich set of features (including click-based and domain-specific ones) to be employed in educational search. LTR models trained on these features outperform various baselines based on ad-hoc retrieval functions and two neural models. As our second contribution, we utilize domain knowledge to build query-dependent ranking models specialized for certain courses or education levels. Our experiments reveal that query-dependent models outperform both the general ranking model and other baselines. Finally, given well-known importance of user clicks in LTR, our third contribution is for handling singleton queries without any click information. To this end, we propose a new strategy to "propagate" click information from the other, similar, queries to the singleton queries. The proposed click propagation approach yields a better ranking performance than the general ranking model and another baseline from the literature. Overall, these findings reveal that both the general and query-dependent ranking models, trained using LTR approaches, yield high effectiveness in educational search, which may ultimately lead to a better learning experience.