Customized Systems of Deep Neural Networks for Energy Efficiency

Abstract

The size and complexity growth of deep neural networks (DNNs), which is driven by the push for higher accuracies and wider ranges of functionality, is outpacing the growth of hardware for mobile systems. Thus, edge devices must collect and transmit sensory data for analysis on a remote server. However, offloading the data to be processed on the backend server can be problematic due to the high cost/latency of wireless communication of the raw data as well as potential privacy/security concerns.

Hence, there has been significant research interest to increase performance and energy efficiency of deep learning computation on both mobile devices and servers. These techniques can be divided into two major categories: input-invariant and input-variant. Input-invariant methods leverage hardware and software techniques, such as pruning, to accelerate a single DNN for the entire dataset. On the other hand, input-variant approaches focus on the fact that most of the inputs do not require the entire computational power of the model to produce an accurate final output. Consequently, they employ a combination of simple and complex models to dynamically adjust the complexity of the DNN to input difficulty.

Both DNN pruning and intelligent combination of DNNs require machine learning expertise and manual design, which makes them hard to implement and deploy. This thesis proposes techniques to improve performance and applicability of both input-invariant and input-variant methods. First, it introduces a new category of input-variant solutions to maximize energy efficiency and minimize latency of DNNs by customizing systems of DNNs based on input variations. Instead of conventional DNN ensembles, it proposes a data heterogeneous multi-NN system to divide the data space into subsets with one specialized learner for each subset. In addition, an intelligent hybrid server-edge deep learning system is introduced to dynamically distribute DNN computation between the cloud and edge device based on the input data and environmental conditions. Furthermore, it suggests an input-driven synergistic deep learning system, which dynamically distributes DNN computation between a more accurate big and a less accurate little DNN. At the end, it introduces a noniterative double-shot pruning method, which takes advantage of both architectural features and weight values to improve the simplicity and applicability of pruning.

Compared to the conventional approaches, on average, an energy consumption reduction up to 91% and a performance improvement up to 12.3x are achieved, while maintaining the original accuracy.