CMARS: a new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimization


Weber G., Batmaz I., Köksal G., Taylan P., Yerlikaya-Ozkurt F.

INVERSE PROBLEMS IN SCIENCE AND ENGINEERING, vol.20, pp.371-400, 2012 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 20
  • Publication Date: 2012
  • Doi Number: 10.1080/17415977.2011.624770
  • Journal Name: INVERSE PROBLEMS IN SCIENCE AND ENGINEERING
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Page Numbers: pp.371-400
  • Keywords: Tikhonov regularization, conic quadratic programming, interior point methods, nonparametric regression, multivariate adaptive regression splines, MODELING TECHNIQUES, TREE
  • Middle East Technical University Affiliated: Yes

Abstract

Regression analysis is a widely used statistical method for modelling relationships between variables. Multivariate adaptive regression splines (MARS) especially is very useful for high-dimensional problems and fitting nonlinear multivariate functions. A special advantage of MARS lies in its ability to estimate contributions of some basis functions so that both additive and interactive effects of the predictors are allowed to determine the response variable. The MARS method consists of two parts: forward and backward algorithms. Through these algorithms, it seeks to achieve two objectives: a good fit to the data, but a simple model. In this article, we use a penalized residual sum of squares for MARS as a Tikhonov regularization problem, and treat this with continuous optimization technique, in particular, the framework of conic quadratic programming. We call this new approach to MARS as CMARS, and consider it as becoming an important complementary and model-based alternative to the backward stepwise algorithm. The performance of CMARS is also evaluated using different data sets with different features, and the results are discussed.