NON-EUCLIDEAN VECTOR PRODUCT FOR NEURAL NETWORKS


Afrasiyabi A., Badawi D., Nasir B., Yildiz O., YARMAN VURAL F. T. , ÇETİN A. E.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, Canada, 15 - 20 April 2018, pp.6862-6866 identifier

  • Publication Type: Conference Paper / Full Text
  • City: Calgary
  • Country: Canada
  • Page Numbers: pp.6862-6866
  • Keywords: non-Euclidean operator, additive neural networks, multiplication-free operator
  • Middle East Technical University Affiliated: Yes

Abstract

We present a non-Euclidean vector product for artificial neural networks. The vector product operator does not require any multiplications while providing correlation information between two vectors. Ordinary neurons require inner product of two vectors. We propose a class of neural networks with the universal approximation property over the space of Lebesgue integrable functions based on the proposed non-Euclidean vector product. In this new network, the "product" of two real numbers is defined as the sum of their absolute values, with the sign determined by the sign of the product of the numbers. This "product' is used to construct a vector product in R-N. The vector product induces the l(1) norm. The additive neural network successfully solves the XOR problem. Experiments on MNIST and CIFAR datasets show that the classification performance of the proposed additive neural network is comparable to the corresponding multi-layer perceptron and convolutional neural networks.