Manually Classifying User Search Queries on an Academic Library Web Site

Abstract

The University of Michigan Library wanted to learn more about the kinds of searches its users were conducting through the “one search” search box on the Library Web site. Library staff conducted two investigations. A preliminary investigation in 2011 involved the manual review of the 100 most frequently occurring queries con- ducted through the site search box over the course of a month. Those 100 search terms accounted for 16 percent of total queries and were largely one-word searches for databases. In the follow-up investigation, the Library embarked on a more ambitious exploration of the 454,443 searches conducted during the winter 2011 semester, devising a method for selecting, categorizing, and summarizing user search queries. A sample of 1,201 searches from the search query logs was examined; after eliminating duplicate searches, there were 992 unique terms available for categorization. Using a non-overlapping sample of queries, a rubric was developed for categorizing user searches. Each of seven library staff members reviewed all unique terms in the sample to categorize them into the best fitting category from the rubric. After establishing a threshold for reliability among the individuals categorizing the queries, 862 unique search terms were analyzed. Based on this analysis, the most frequent kinds of searches conducted in the winter semester in 2011 on the University of Michigan Library’s Web site were specific databases (28 percent), topical/exploratory types of queries (28 percent), and books (including searches by title, ISBN, call number, or a combination thereof) (16 percent).Within the sample, known-item searches comprised nearly half (44 percent) of searches in the sample. An- other fifth (20 percent) of total searches were categorized as “exploratory,” supporting the need to provide broader, subject-based paths to information through the site. Somewhat surprisingly, there were a small number of article searches (article titles, or mixed searches of journal names and authors and/or title words) in the search box—an indication that users understand the University of Michigan Library primary search box is not for articles.