Hardware architectures to support low power natural I/O applications.

Abstract

As computing systems become an ever more integral aspect of our everyday lives, the demand for more domain appropriate forms of interface technology has increased substantially. Nowhere is this better seen than small hand-held devices, which currently utilize awkward techniques such as specialized scripts or on-screen keyboards. Achieving the goal of ubiquitous computing will require a departure from standard methods, leading to renewed interest in natural I/O technologies such as speech recognition. Unfortunately, the computational demands of robust, large vocabulary speech recognition are well beyond the capabilities likely to be seen on handheld systems in the near future. This work explores the potential for utilizing domain specific characteristics of natural I/O workloads to achieve the necessary functionality within the energy constraints of low power systems. We focus on speech recognition, and exploit the massive amounts of easily exposed thread level concurrency (1000s to 10,000s of potential threads) inherent in this application. We present a hybrid multi-threaded, chip-multiprocessor architecture utilizing small, unsophisticated processing elements and numerous latency tolerating mechanisms to maximize processor utilization and system throughput. Under idealized memory constraints, such a system achieves near linear speedup with added parallel resources, often correlating to a reduction in workload energy consumption despite the added power dissipation. We next turn our attention to the memory system, determining that memory bandwidth is the key constraint in this domain. Streaming access to knowledge base data can severely constrain achievable performance. We find that a multi-level caching strategy, targeting L2 sizes to match higher locality program metadata (<256K) can achieve significant improvements in performance by reducing bandwidth demand. Overall, we find that a fairly small CMP/MT architecture (4 processors, 2--4 hardware thread contexts each) with such a multi-level memory system can achieve or exceed realtime performance on moderate complexity tasks, and that the inherent properties of our architecture make techniques such as dynamic concurrency management, data compression in the lower levels of the memory system, and voltage and frequency scaling quite effective at providing realtime speech recognition within the energy budget of low-power portable systems.