Effects of Various Preprocessing Techniques to Turkish Text Categorization Using N-Gram Features

Deniz A., Kiziloz L. E.

2017 International Conference on Computer Science and Engineering (UBMK), Antalya, Türkiye, 5 - 08 Ekim 2017, ss.655-660 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Antalya
  • Basıldığı Ülke: Türkiye
  • Sayfa Sayıları: ss.655-660
  • Anahtar Kelimeler: Turkish text classification, n-gram features, supervised machine learning
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

Natural Language Processing (NLP) is a prominent subject which includes various subcategories such as text classification, error correction, machine translation, etc. Unlike other languages, there are limited number of Turkish NLP studies in literature. In this study, we apply text classification on Turkish documents by using n-gram features. Our algorithm applies different preprocessing techniques, namely, n-gram choice (character level or word level, bigram or trigram models), stemming, and use of punctuation, and then determines the Turkish document's author and genre, and the gender of the author. For this purpose, Naive Bayes, Support Vector Machines and Random Forest are used as classification techniques. Finally, we discuss the effects of above mentioned preprocessing techniques to the performance of Turkish text classification.