Effects of Various Preprocessing Techniques to Turkish Text Categorization Using N-Gram Features

Deniz A., Kiziloz L. E.

2017 International Conference on Computer Science and Engineering (UBMK), Antalya, Turkey, 5 - 08 October 2017, pp.655-660 identifier

  • Publication Type: Conference Paper / Full Text
  • City: Antalya
  • Country: Turkey
  • Page Numbers: pp.655-660
  • Keywords: Turkish text classification, n-gram features, supervised machine learning
  • Middle East Technical University Affiliated: Yes

Abstract

Natural Language Processing (NLP) is a prominent subject which includes various subcategories such as text classification, error correction, machine translation, etc. Unlike other languages, there are limited number of Turkish NLP studies in literature. In this study, we apply text classification on Turkish documents by using n-gram features. Our algorithm applies different preprocessing techniques, namely, n-gram choice (character level or word level, bigram or trigram models), stemming, and use of punctuation, and then determines the Turkish document's author and genre, and the gender of the author. For this purpose, Naive Bayes, Support Vector Machines and Random Forest are used as classification techniques. Finally, we discuss the effects of above mentioned preprocessing techniques to the performance of Turkish text classification.