Large deviations for linear-region counts in randomly initialized deep piecewise-linear networks
27th International Conference on COMPUTATIONAL STATISTICS (COMPSTAT 2026), Athens, Yunanistan, 25 - 28 Ağustos 2026, (Özet Bildiri)
- Yayın Türü: Bildiri / Özet Bildiri
- Basıldığı Şehir: Athens
- Basıldığı Ülke: Yunanistan
- Orta Doğu Teknik Üniversitesi Adresli: Evet
Özet
The number of linear regions is a widely used statistic for quantifying the expressive capacity of deep piecewise-linear neural networks, and is routinely estimated empirically for randomly initialized architectures. A tractable random compositional model is studied in which each layer is an independent random perturbation of the symmetric tent map, so that the depth-$n$ composition is a one-dimensional ReLU network and the observable $N_n$ is its exact linear region count. A limiting pressure function $\Lambda(q)=\lim_{n \to \infty} n^{-1}\log E[N_n^q]$ is shown to exist via submultiplicativity, and both tails of the empirical growth rate $n^{-1}\log N_n$ are controlled by explicit convex rate functions. The method covers general submultiplicative complexity statistics, including region counts of multivariate networks along polytopal covers and worst-case input lines, with architecture-level constants that are explicitly computable. As no matching supermultiplicative structure is available, complementary lower bounds are built constructively from a finite-state certified process tracking how close branch images come to losing splitting ability; its deterministic core has spectral radius increasing to two, so that in a small-noise regime growth rates bounded away from $\log 2$ become eventually impossible. The results quantify the probability that near-maximal expressivity is achieved or lost under random perturbations of classical depth-separation constructions.