Mathematical Sciences Department Colloquium
Elizabeth Newman, Emory University
Thursday, January 30, 2025
11am-12pm
Stratton 201
Title: Exploiting Coupling for Efficient Deep Learning and Optimal Multiway Representations
Abstract: Harnessing the power of modern computational tools and big data demands the creation of powerful data-driven algorithms and quality representations of large, real-world data. In this talk, we will address these needs in two distinct ways: by designing efficient deep learning algorithms and by optimizing multidimensional approximations.
Deep neural networks (DNNs) have been successful high-dimensional function approximators in countless applications. However, training DNNs is notoriously challenging, requiring significant time and computational resources. In the first half of this talk, we will describe improved training algorithms that exploit the commonly used separable DNN structure in which the weights of the final layer are applied linearly. We will leverage this linearity in two ways: using
partial optimization in the deterministic setting and iterative sampling in the stochastic setting. We will demonstrate empirically that both approaches yield faster convergence to more accurate DNN models and less sensitivity to training hyperparameters.