Please note special date/time for this Applied Math seminar.
Speaker: Thomas Chen, UT Austin
Title: Explicit construction of global minimizers and the interpretability problem in Deep Learning
Abstract: In this talk, we present some recent results aimed at the rigorous mathematical understanding of how and why supervised learning works. We point out genericness conditions related to reachability of zero loss minimization, in underparametrized versus overparametrized Deep Learning (DL) networks. For underparametrized DL networks, we explicitly construct global, zero loss cost minimizers for sufficiently clustered data. In addition, we derive effective equations governing the cumulative biases and weights, and show that gradient descent corresponds to a dynamical process in the input layer, whereby clusters of data are progressively reduced in complexity ("truncated") at an exponential rate that increases with the number of data points that have already been truncated. For overparametrized DL networks, we prove that the gradient descent flow is homotopy equivalent to a geometrically adapted flow that induces a (constrained) Euclidean gradient flow in output space. If a certain rank condition holds, the latter is, upon reparametrization of the time variable, equivalent to simple linear interpolation. This in turn implies zero loss minimization and the phenomenon known as “Neural Collapse”. Moreover, we derive zero loss guarantees, and construct explicit global minimizers for overparametrized deep networks, given generic training data. The work presented includes collaborations with Patricia Munoz Ewald and Andrew G. Moore (UT Austin).
In person attendance at this seminar is only open to Columbia Univesity affiliates. External guests are welcome to attend remotely. Please contact [email protected] if you need the Zoom link for this seminar.
