Convolutional neural networks analysed via inverse problem theory and sparse representations

Creative Commons License

Tarhan C., Akar G.

IET SIGNAL PROCESSING, vol.13, no.2, pp.215-223, 2019 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 13 Issue: 2
  • Publication Date: 2019
  • Doi Number: 10.1049/iet-spr.2018.5220
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Page Numbers: pp.215-223
  • Middle East Technical University Affiliated: Yes


Inverse problems in imaging such as denoising, deblurring, superresolution have been addressed for many decades. In recent years, convolutional neural networks (CNNs) have been widely used for many inverse problem areas. Although their indisputable success, CNNs are not mathematically validated as to how and what they learn. In this study, the authors prove that during training, CNN elements solve for inverse problems which are optimum solutions stored as CNN neuron filters. They discuss the necessity of mutual coherence between CNN layer elements in order for a network to converge to the optimum solution. They prove that required mutual coherence can be provided by the usage of residual learning and skip connections. They have set rules over training sets and depth of networks for better convergence, i.e. performance. They have experimentally validated theoretical assertions.