5 - Large Neural Networks
Published online by Cambridge University Press: 30 June 2022
Summary
This chapter covers large neural networks with random weights, in both feed-forward and recurrent settings. While being rather different from modern deep neural networks, these preliminary results shed new light on the interplay between data, network structure, and nonlinear neurons, leading to the somewhat surprising double descent phenomenon. The impact of gradient-based optimization method on the resulting network and more advanced consideration of deep networks are also discussed.
- Type
- Chapter
- Information
- Random Matrix Methods for Machine Learning , pp. 277 - 312Publisher: Cambridge University PressPrint publication year: 2022