In feature selection problems, the aim is to select a subset of features to characterize an output of interest. In characterizing an output, we may want to consider multiple objectives such as maximizing classification performance, minimizing number of selected features or cost, etc. We develop a preference-based approach for multiobjective feature selection problems. Finding all Pareto-optimal subsets may turn out to be a computationally demanding problem and we still would need to select a solution. Therefore, we develop interactive evolutionary approaches that aim to converge to a subset that is highly preferred by the decision maker (DM). We test our approaches on several instances simulating DM preferences by underlying preference functions and demonstrate that they work well.