The Statistical Physics of Deep Learning
 01:08:53
Description 

Neuronal networks have enjoyed a resurgence both in the worlds of neuroscience, where they yield mathematical frameworks for thinking about complex neural datasets, and in machine learning, where they achieve state of the art results on a variety of tasks, including machine vision, speech recognition, and language translation. Despite their empirical success, a mathematical theory of how deep neural circuits, with many layers of cascaded nonlinearities, learn and compute remains elusive. We will discuss three recent vignettes in which ideas from statistical physics can shed light on this issue. In particular, we show how dynamical criticality can help in neural learning, how the nonintuitive geometry of high dimensional error landscapes can be exploited to speed up learning, and how modern ideas from nonequilibrium statistical physics, like the Jarzynski equality, can be extended to yield powerful algorithms for modeling complex probability distributions. Time permitting, we will also discuss the relationship between neural network learning dynamics and the developmental time course of semantic concepts in infants. 
Details 


Title 
The Statistical Physics of Deep Learning 
Creator 
University of California, Berkeley. Dept. of Physics 
Published 
Berkeley, CA, University of California, Berkeley, Dept. of Physics, October 5, 2015 
Full Collection Name 
Physics Colloquia 
Type 
Video 
Format 
Lecture. 
Extent 
1 streaming video file 
Other Physical Details 
digital, sd., col. 
Archive 
Physics Library 
Note 
Recorded at a colloquium held on October 5, 2015, sponsored by the Dept. of Physics, University of California, Berkeley. originally produced as an .mts file in 2016 Speakers: Ganguli, Surya. 
Collection 
Physics Colloquia 
Tracks 
colloquia/10515Ganguli.mp4 01:08:53 
Linked Resources 