Neural Network Implementations on Analog In-Memory-Computing Fabric

Abstract

Deep neural networks (DNNs) have achieved unprecedented capabilities in tasks such as analysis and recognition of images and voices, leading to their widespread adaptation. However, computation requirements and the associated energy consumption of neural network implementations have been growing rapidly. In addition, traditional computing architectures are ineffective for DNN workloads due to the high memory access demands, making it even more challenging to meet these computational requirements. The most important limiting factor for DNN computing is the transfer of data between processors and off-chip memories due to the limited density of existing on-chip memory technologies. In-memory computing (IMC) systems, utilizing the density advantage of emerging memory technologies like RRAM, can potentially store entire DNN models on-chip, thus eliminating off-chip memory access. Particularly, analog IMC systems that utilize the memory device properties to directly perform vector-matrix multiplication (VMM) operations allow device-level parallelism that promises drastic improvement in energy efficiency.

However, the analog computation means computation accuracy is an additional concern, even though neural networks are known for their fault tolerance. In general, for analog computing systems, computation accuracy needs to be ensured before any benefit in energy efficiency can become material. This dissertation presents studies on the implementation of DNNs in realistic analog IMC systems from an accuracy perspective under realistic memory devices and system non-idealities. In this work, memory device performance requirement was established, and methods to mitigate the impact of the non-idealities was also developed. Deterministic error sources including memory device on/off ratio, programming precision, array size limitation, and ADC characteristics were considered. Stochastic error sources including device programming variation, device defect, and ADC noise were considered. Particularly, a tiled architecture was developed to mitigate the effects of limited practical memory array sizes, and the consequence of this architecture was carefully studied.

First, inference operation on analog IMC systems is investigated. An architecture-aware training method was developed to mitigate the deterministic error sources, and noise injection was used to mitigate device programming variation. Inference accuracies similar to that of the floating-point baselines was achieved on simulated realistic analog IMC system for large-scale neural networks by using these mitigation methods. Minimum requirements for device defect rate and programming variation were established.

Second, DNN training on analog IMC systems was also explored. A mixed-precision training method was used, where weight updates are accumulated in software and only programmed onto memory devices when a certain threshold is reached. This drastically reduces device programming cycles during the training process while leveraging analog IMC systems for efficient computation in the forward and backward propagation progress. DNN training was shown to be effective in the simulated analog IMC system, even achieving better than floating-point baseline validation accuracies in some situations.