The computational power of neural networks depends on properties of the real numbers used as weights. We focus on networks restricted to compute in polynomial time, operating on boolean inputs. Previous work has demonstrated that their computational power happens to coincide with the complexity classes P and P/poly, respectively, for networks with rational and arbitrary real weights. Here we prove that the crucial concept that characterizes this computational power is the Kolmogorov complexity of the weights, in the sense that, for each bound on this complexity, the networks can solve exactly the problems in a related nonuniform complexity class located between P and P/poly. By proving that the family of such nonuniform classes is infinite, we show that neural networks can be classified into an infinite hierarchy of different computing capabilities.
CitationBalcázar Navarro, José Luis; Gavaldà Mestre, Ricard; Siegelmann, H. "Computational power of neural networks: a Kolmogorov complexity characterization". 1993.
All rights reserved. This work is protected by the corresponding intellectual and industrial property rights. Without prejudice to any existing legal exemptions, reproduction, distribution, public communication or transformation of this work are prohibited without permission of the copyright holder. If you wish to make any use of the work not provided for in the law, please contact: firstname.lastname@example.org