Group Method of Data Handling (GMDH) - type neural network algorithm is the heuristic self-organizing algorithm to model the sophisticated systems. In this study, we propose a new algorithm assembling different classifiers based on GMDH algorithm for binary classification. A Monte Carlo simulation study is conducted to compare diverse classifier ensemble based on GMDH (dce-GMDH) algorithm to the other well-known classifiers and to give recommendations for applied researchers on the selection of appropriate classifier under the different conditions. The simulation study illustrates the proposed approach is more successful than the other classifiers in classification in most scenarios generated under the different conditions. Our proposed method is compared to the other classifiers on Cleveland heart disease data. An implementation of the proposed approach is demonstrated on urine data. Moreover, the proposed algorithm is released under R package GMDH2 under the name of "dceGMDH" for implementation.