Depth Separation in Norm-Bounded Infinite-Width Neural Networks
Published in 37th Annual Conference on Learning Theory (COLT), 2024
Establishes a separation in the representation cost and sample complexity needed to approximate functions with two vs. three layer neural networks.