Due to the increasing sophistication of cyber-attacks, intrusion detection systems need to be improved constantly. Each machine learning classifier has different advantages against intrusion detection and combining the advantages of different classifiers increases detection rates. In this study, we combine a machine learning classifier with a deep learning model to propose a new approach called GRU-GBM. The LightGBM gradient boosting machine framework is used for feature selection, and each feature in the dataset is evaluated by a second LightGBM classifier to determine the optimal feature set using a novel threshold-based approach. After the selection of the feature set, a gated recurrent unit is used for attack detection by a recurrent neural network model. Besides, different training/testing ratios (60/40-70/30) are chosen for comparison of GRU-GBM accuracy. The proposed combined model achieved 76.61% and 93.65% overall accuracy in multi-class experiments conducted with the UNSW-NB15 and LITNET-2020 datasets, respectively. Lastly, the GRU-GBM model is compared to other machine learning models. The overall accuracy result is tested with a non-parametric Friedman test to determine the significance of the results. The test result shows that there is enough evidence that the accuracy of the GRU-GBM classifier is statistically significant.