Depth Separation in Norm-Bounded Infinite-Width Neural Networks

Published in 37th Annual Conference on Learning Theory (COLT), 2024

Establishes a separation in the representation cost and sample complexity needed to approximate functions with two vs. three layer neural networks.

Joint work with Greg Ongie, Rebecca Willett, Ohad Shamir, and Nati Srebro.