Low-Power Localization Systems with Hardware-Efficient Deep Neural Networks

Abstract

Localization systems solve the problem of identifying the location of the agent or surrounding objects with the information gathered from various sensors. It enables a wide range of practical applications, such as autonomous navigation, self-driving cars, virtual reality, augmented reality, and enhanced surveillance. In recent years, deep neural networks have achieved great success in various computer vision and machine learning tasks, including more accurate localization systems with extensive computation complexity and power consumption. However, deploying such systems on energy-constrained mobile Internet-of-Things (IoT) platforms remains a big challenge due to the contradiction between system performance and power consumption. This thesis presents several practical approaches to develop energy-efficient localization systems for real-world applications. First, a real-time visual based simultaneous localization and mapping system is investigated and optimized for hardware implementation, which is ported on a low-power, application specific integrated circuit accelerator. The second work focuses on reducing the complexity of deep learning based visual-inertial odometry systems by finding the most efficient network architecture through neural architecture search and adaptively disabling visual sensor modality on the fly. The third work proposes an accurate learning based end-to-end audio source separation and localization framework with only low-power microphone sensor array, taking advantage of self-supervised learning and adversarial learning techniques. Finally, a new hardware-efficient heterogeneous transform-domain neural network is introduced to reduce computation complexity by replacing convolution operations with element-wise multiplications, learning sparse-orthogonal weights in heterogeneous transform domains, and non-uniform quantization with canonical-signed-digit representation. These works explore four different yet effective ways to balance the system performance and power consumption for mobile IoT platforms, namely reducing deep neural network complexity, adaptively selecting and fusing sensor modalities, employing lower power sensors, and developing hardware-efficient systems for specialized accelerators.