Mathematical Sciences-PhD Proposal- Wenjing Li “Optimal Ensembles for Deep Learning Classification: Theory and Practice"

Wednesday, September 30, 2020
2:00 pm to 4:00 pm
Floor/Room #: 
via Zoom

PhD Dissertation Proposal
Candidate:
Wenjing Li
Title: “Optimal Ensembles for Deep Learning Classification: Theory and Practice”
Location: Via Zoom, please contact Rhonda Podell (rpodell@wpi.edu) for Zoom details.

Abstract: Ensemble learning is a process by which multiple base learners are strategically generated and combined into one composite learner. There are two features that are essential to an ensemble's performance, namely the individual accuracies of the component base learners and the overall diversity level of the ensemble. It is evident that diversity must have a role to play in the effectiveness of an ensemble, since multiple identical copies of the same learner clearly cannot improve on the performance of a single copy of that learner. However, the right balance of learner accuracy and ensemble diversity can improve the performance of machine learning tasks on benchmark data sets and real-world data sets, and recent theoretical and practical work has demonstrated the subtle trade-off between accuracy and diversity in an ensemble. Herein, we extend the extant literature by providing a deeper theoretical understanding for assessing and improving the optimality of any given ensemble, including random forests and deep neural network ensembles. We also propose a training algorithm for neural network ensembles and demonstrate that our approach provides improved performance when compared to both state-of-the-art individual learners as well as ensembles of state-of-the-art learners trained using standard loss functions. Our key insight is that it is better to explicitly encourage diversity in an ensemble, rather than merely allowing diversity to occur by happenstance, and that rigorous theoretical bounds on the trade-off between diversity and learner accuracy allow one to know when an optimal arrangement has been achieved. While our focus is on image classification problems, we wish to emphasize that the theory and approaches provided herein are also applicable to many other machine learning tasks where neural networks already provide state-of-the-art performance.

Committee Members (arranged by Alphabetic order):

  1. Prof. Randy C. Paffenroth (PHD advisor), Associate Professor of Department of Mathematical Sciences, Computer Science, and Data Science, WPI, rcpaffenroth@wpi.edu.
  2. Prof. Xiangnan Kong, Associate Professor of Department of Computer Science, WPI, xkong@wpi.edu.
  3. Prof. Oren Mangoubi, Assistant Professor of Department of Mathematical Sciences and Data Science, WPI, omangoubi@wpi.edu.
  4. Prof. Simon Tavener, Professor of Department of Mathematics, Colorado State University, tavener@math.colostate.edu.
  5. Prof. Frank Zou, Associate Professor of Department of Mathematical Sciences, WPI, jzou@wpi.edu.