Local image features, which are robust to scale, view, and orientation changes in images, play a key factor in developing effective visual classification systems. However, there are two main limitations to exploit these features in image classification problems: 1) a large number of key-points are located during the feature detection process, and 2) most of the key-points arise in background regions, which do not contribute to the classification process. In order to decrease the inverse effects of these limitations, we propose a new codebook generation approach through employing a new clustering method that generates class-specific codebooks along with a novel feature selection method in the bag-of-words model. We evaluate the performance of different classification techniques including Naive Bayesian, k-NN, and SVM on distinctive features. Experiments conducted on PASCAL Visual Object Classification collections have shown that the class-specific codebooks along with distinctive image features can significantly improve the classification performances.