A Scoring Method for Interpretability of Concepts in Convolutional Neural Networks Evrişimsel Sinir Aǧlarinda Kavram Yorumlama için bir Puanlama Yöntemi

Gurkan M. K., Arica N., Vural F. Y.

30th Signal Processing and Communications Applications Conference, SIU 2022, Safranbolu, Turkey, 15 - 18 May 2022 identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/siu55565.2022.9864930
  • City: Safranbolu
  • Country: Turkey
  • Keywords: concept-based analysis, convolutional neural networks, Explainable AI, runtime interpretation
  • Middle East Technical University Affiliated: Yes


© 2022 IEEE.In this paper, we propose a scoring algorithm for measuring the interpretability of CNN models by focusing on the feature extraction operation at the convolutional layers. The proposed approach is based on the principal of concept analysis, for a predefined list of concepts. A map of the network is created based on its responsiveness against each concept. Once this map is ready, various images can be applied as inputs and they are matched with the concepts whose hidden nodes are highly activated. Finally, the evaluation algorithm kicks in to use these descriptions during the final prediction and provides human-understandable explanations.