The Fast, Efficient, and Representative Benchmarking of Future Microarchitectures.

Abstract

A methodology is introduced to reduce the overall simulation time of large benchmarking suites. Previous work shows that it is possible to simulate only small sections of a benchmark's dynamic instruction stream in detail, without sacrificing accuracy in simulation results with respect to overall behavior. As benchmarking suites increase in size, many such techniques still require a great deal of simulation time to complete. The methods presented in this dissertation build on this previous work by converting representative sections of a benchmark's execution into either augmented binaries or intrinsically checkpointed assembly code. This new code can then serve as a replacement for the original benchmark. In addition, a methodology is proposed that creates new benchmark binaries that no longer need input files or system calls in order to execute properly. Since the new benchmarks only contain portions of the original benchmarks and input data is effectively hidden within them, corporations can safely release benchmarks to the public created using their own internal, proprietary test programs without the fear of losing sensitive information.

Simulations of the new benchmarks are much faster, require less overhead, and still properly represent the original benchmark's execution profile. Results show that benchmarks created using these techniques are very portable and accurately predict the performance of the original benchmark. An average error rate of less than 5% is achieved when compared to the original representative sections. In addition, a speedup of roughly 60x per benchmark is achieved when the new benchmarks are executed serially and 1000x when they are executed in parallel. This translates into a reduction in simulation time from months to minutes and greatly decreases the amount of time necessary to test a new design.