WPI - Computer Science Department, PhD Proposal Defense,Yiyang Zhao " Efficient and Sustainable Neural Architecture Search""

Zoom: https://wpi.zoom.us/j/9958714387

Committee Members:

Dissertation Advisor: Prof. Tian Guo, WPI – Computer Science

 Prof. Xiangnan Kong, WPI – Computer Science

Prof. Xiaozhong Liu. WPI – Computer Science

External Committee Member: Dr. Tieying Zhang - Bytedance

Abstract:

Artificial intelligence (AI) now plays an indispensable role not only in the computer science domain but also in people's everyday lives. AI solutions have greatly outperformed conventional methods in many real-world tasks and problems, such as image classification, object detection, and image segmentation. However, the design of AI models and systems is still reserved for domain experts, which largely restricts the development and spread of AI. This proposal seeks to design an AI pipeline to automate the production line of AI to remove this restriction, in an efficient and sustainable way, from three aspects.

(1)Multi-Objective Neural Architecture Search (NAS) Algorithm. Previous NAS works have mainly focused on designing models with good performance metrics while neglecting other important factors such as the number of parameters. In this thesis, we propose to design an effective multi-objective search algorithm for NAS, allowing it to consider multiple factors during the design process. (2) Efficient Network Evaluations. The vanilla NAS approach requires training all the searched/sampled neural architectures from scratch to evaluate their performance, incurring a significant amount of time and computational costs, often hundreds to thousands of GPU days. In order to reduce the search cost, we propose a few-shot NAS approach that leverages multiple super-nets as proxies to accurately estimate the performance of models in a shorter amount of time. (3) Sustainable Neural Architecture Search. Energy cost and carbon emissions are major environmental issues in Neural Architecture Search (NAS). Prior work reports that a single architecture search by NAS can produce as much carbon emissions as five cars' lifetimes. We propose an adaptive carbon-aware NAS strategy that reduces carbon consumption during the search while maintaining good search performance. Additionally, we aim to design an adaptive energy-aware architecture that can operate within different energy budgets.