High prediction accuracies at time series modeling and forecasting is of the utmost importance for a variety of application domains. Many methods have been proposed in the literature to improve time series forecasting accuracy. Those which focus on univariate time series forecasting methods use only the values in the prior time steps to predict the next value. In this study in addition to the historical values, it is aimed to increase the forecasting performance by using extra statistical and structural features which summarize characteristics of the time series. Feature importance scores are determined by gradient boosting trees (GBT). Features with the highest importance score are given as explanatory additional variable to the hybrid ARIMA-ANN model. The evaluation of the developed method is performed on four different publicly available datasets. Our experimental results show higher accuracy performance for the proposed method as compared to the currently well-accepted methods.