Research Information System
15th International Conference on Information Security and Cryptography, ISCTURKEY 2022, Ankara, Turkey, 19 - 20 October 2022, pp.92-97
© 2022 IEEE.Although several binary classification performance metrics have been defined, a few of them are used for performance evaluation of classifiers and performance comparison/reporting in the literature. Specifically, F1 and Accuracy (ACC) are the most known and conventionally used metrics. Despite their popularity and easy-to-understand characteristics, those metrics exhibit critical robustness issues. This paper suggests a new instrument category named 'performance indicators' and proposes a novel indicator named accuracy barrier (ACCBAR for short) that works to uncover confounding problems in performance reporting of ACC metric. The given case study in mobile malware classification, which is a domain of cyber security, has shown that the indicator gives an accurate interpretation of the results presented in terms of ACC. This study also recommends that researchers should use ACCBAR to eliminate potential publication or confirmation bias in classification performance evaluation.