The Linux Scalability Project

Abstract

The Linux Scalability Project is adapting Linux for use in enterprise-scale networking environments. We focus on kernel algorithms and data structures that scale poorly when presented with thousands or tens of thousands of simultaneous service requests. For example, we uncovered a "thundering herd" problem in the accept system call. A few dozen lines of code corrects this behavior to awaken only one, instead of all, waiting threads. This small change improves macro-benchmark performance by over 50% on high-performance hardware. Other examples include improving poll performance, adding read-ahead support for sendfile and mmaped files, and identifying areas of significant kernel SMP lock contention. The project is also implementing an open source, reference implementation of NFSv4, a highly scalable evolution of the popular distributed file system.& Building relationships between academia, industry, and open source communities is a primary goal of our effort. While our research-oriented, "cathedral" approach is sometimes at odds with the "bazaar" style of software development prevalent in the Linux community, we find ways to build reputation and influence among Linux kernel developers and the growing Linux commercial sector.