In-SRAM Computing for Neural Network Acceleration

Abstract

For decades, the computing paradigm has been composed of separate memory and compute units. Processing-in-Memory(PIM) has often been proposed as a solution to break past the memory wall. With PIM, compute logic is moved near the memory, which can reduce the data movement. In-memory computing expands on PIM by morphing the memory into hybrid memory compute units, where data can be stored and computed on in-place. Recent work has modified SRAM arrays to allow logical operations to be performed directly inside the arrays. Our work extends basic logical operations and additionally adds support for arithmetic operations.

Coinciding with the rise of increasing memory on-chip and more focus on near and in-memory computing is the ascendance of neural networks. Neural networks are highly data-parallel applications that are challenging to accelerate due to being data-bound, compute-bound, or both. In-memory computation reduces on-chip data movement and will increase the amount of compute available as well as the amount of storage in a custom chip. These factors can greatly alleviate compute and data bottlenecks.

First, this thesis observes that SRAM memory has increasingly dominated the on-chip area for general-purpose processors. This area comes at the cost of compute potential and can be repurposed to function as a dual storage and compute unit. The benefits of such repurposing are greatly expanding the parallel compute capability of the chip while also reducing the on-chip data movement, all with minimal area increase. When SRAM is repurposed, the storage area can be reclaimed with minimal overhead. Modifications to the SRAM arrays are presented that allow the SRAM to function as hybrid compute/storage units capable of arithmetic operations. Additionally, this work presents a mapping strategy for supporting CNNs in the hybrid SRAM storage compute arrays.

Second, this thesis proposes a custom ASIC called Eidetic that utilizes hybrid compute/storage SRAM arrays as both its primary storage and compute units. Repurposing a processor's SRAM is hamstrung by maintaining the cache's original functions and area footprint. Too many modifications to the cache would render the solution undesirable to chip designers. By further customizing the SRAM we can create more efficient PE units. Additionally, the increased SRAM storage allows more weights to be stored on-chip. Finally, the custom ASIC allows for a control logic that supports a graph-based programming model that further reduces off-chip data movement. These customizations allow Eidetic to target data-bound applications such as RNNs and MLPs.

Third, we propose a detailed comparison between the in-cache and ASIC approaches to ML acceleration. Between repurposing the cache and creating an SRAM-based custom ASIC, in-SRAM computing offers multiple viable approaches. In-Cache computing is cheaper but comes with limitations, while an ASIC design is more expensive due to the total cost of ownership (TCO).

We compare the performance and energy efficiency of our repurposed cache with a server-class GPU and the baseline CPU. We similarly evaluate our custom ASIC to other state-of-the-art ASIC DNN accelerators. For both the repurposed cache and the ASIC, we develop cycle-accurate simulators to determine the performance.