TB Computer Vision System-On-Chip Designs for Intelligent Vehicles
Advanced driver assistance systems (ADAS) have been studied in order to aid drivers with vehicle operation. Many sensors like LiDAR, radar, cameras have been deployed on intelligent vehicles. Among these sensors, optical cameras are most widely used due to their low costs and easy installation. However, most computer vision algorithms are complicated and computationally slow, making them difficult to be deployed on power constraint systems. This dissertation investigates several mainstream ADAS applications, and proposes corresponding efficient digital circuits implementations for these applications.
In this dissertation, real time ADAS systems are demonstrated on Xilinx Zynq platform, which has a dual core ARM processor and FPGA fabric. The Xilinx Zynq platform integrates the software programmability of an ARM processor with the hardware programmability of an FPGA. The dissertation presents our design approach of a real time ADAS system on system-on-chip platform. Three ADAS applications are demonstrated in this dissertation: lane detection, traffic sign classification, and traffic light detection. Using FPGA to offload critical parts of the algorithm, the entire computer vision system is able to run in real time while maintaining a low power consumption and a high detection rate.
We also present two deep learning based hardware implementations on application specific integrated circuits (ASIC) to achieve even lower power consumption and higher accuracy. Nowadays, convolutional neural network (CNN) is revolutionizing computer vision due to learnable layer by layer feature extraction. However, when coming into inference, CNNs are usually slow to train and slow to execute. In this dissertation, we investigated potential CNN alternative - principal component analysis based network (PCANet), which strikes a balance between algorithm robustness and computational complexity. Implementing in Synopsys 32nm process technology, the proposed chip only consumes 0.5 watt power consumption. Another CNN alternative is the binary neural network (BNN). An optimized hardware implementation of BNN is presented in this dissertation. Implementing in Synopsys 32nm process, the BNN accelerator consumes 0.6 watt power, and is hence suitable for embedded platforms.
Prof. Xinming Huang
ECE Department, WPI
Prof. Lifeng Lai
ECE Department, University of California, Davis
Prof. Jie Fu
ECE/RBE Department, WPI