Computer Science Department , PhD Dissertation Defense Yiyang Zhao " Efficient and Sustainable Neural Architecture Search"

Yiyang Zhao 

PhD Candidate 

WPI – Computer Science Department 

Wednesday, June 19, 2024

Time: 1:00 p.m. – 2:00 p.m. 

Zoom:  https://wpi.zoom.us/j/9958714387

• Committee Members:

Dissertation Advisor: Prof. Tian Guo, WPI – Computer Science 
 Prof. Xiangnan Kong, WPI – Computer Science 
Prof. Xiaozhong Liu. WPI – Computer Science 
External Committee Member: Dr. Tieying Zhang – Bytedance

Abstract:

Artificial intelligence (AI) now plays an indispensable role not only in the computer science domain but also in people's everyday lives. AI solutions have greatly outperformed conventional methods in many real-world tasks and problems, such as image classification, object detection, and image segmentation. However, the design of AI models and systems is still reserved for domain experts, which largely restricts the development and spread of AI. This proposal seeks to design an AI pipeline to automate the production line of AI to remove this restriction, in an efficient and sustainable way, from three aspects. (1)Multi-Objective Neural Architecture Search (NAS) Algorithm. Previous NAS works have mainly focused on designing models with good performance metrics while neglecting other important factors such as the number of parameters. In this thesis, we propose to design an effective multi-objective search algorithm for NAS, allowing it to consider multiple factors during the design process. (2) Efficient Network Evaluations. 

The vanilla NAS approach requires training all the searched/sampled neural architectures from scratch to evaluate their performance, incurring a significant amount of time and computational costs, often hundreds to thousands of GPU days. In order to reduce the search cost, we propose a few-shot NAS approach that leverages multiple super-nets as proxies to accurately estimate the performance of models in a shorter amount of time. (3) Sustainable Neural Architecture Search. Energy cost and carbon emissions are major environmental issues in NAS. Prior work reports that a single architecture search by NAS can produce as much carbon emissions as five cars' lifetimes. Despite many efforts to reduce energy costs in NAS, the carbon emissions can still vary with energy generation methods, e.g., solar vs. fuel, and locations. Therefore, we propose an adaptive carbon-aware NAS framework that reduces carbon consumption during the search while maintaining good search performance.