Augmenting Interactive Information Seeking with System-Level Assistance

Abstract

Within the past two decades there has been an increased amount of interest in human-centered information retrieval research beyond the traditional system-focused view embodied by Cranfield- and TREC-style evaluation. From blending search and recommender systems to work on search as learning, the limits of conventional search engines' focus on query relevance to deliver ten blue links at millisecond speeds has become evident as use cases become more varied. My work focuses on the types of scenarios that typical systems neglect in searching and browsing, particularly tasks involving multi-attribute queries and information seeking as learning, and explores ways to guide users towards optimal behavior through system design. Enabling this across the three studies comprising this dissertation is a sidebar affordance, serving as a means for enabling complementary information seeking interactions. The contributions of this work will have implications on the effective design and implementation of new types of user-centered interactive IR systems.

We begin with an investigation of time-quality tradeoffs with slow search. Taking inspiration from movements such as slow food, slow travel, and slow technology, slow search serves as an acknowledgement of the fact that there are tasks for which users have indicated a willingness to wait for the perfect set of results. By implementing a user study where searchers were exposed to a system that embodied characteristics of slow search, where speed could be traded for an better results, we analyzed user behavior as they performed tasks which typically required multiple queries with a baseline Web search engine and saw how their effectiveness in using the system improved as they used this novel interface.

Next, we performed a simulation study to explore the implications of changing attributes of our slow search system on the behavioral outcomes of synthetic users modelled based on human interaction log data. By incorporating the users' cost model, we were able to identify fruitful directions for further interactive search experiments. In this way, we showed the potential for guiding low-performing users towards higher performance.

We finally focus on search as learning, using a large language model (LLM) as an enabler of slow search. Here, our study tests a contextual chatbot assistant that aids in users' reading and searching in a specialized domain -- data science. The chatbot can provide responses to questions about documents and domain-specific vocabulary. Using mixed methods, we identify patterns of use and investigate learning and interaction behaviors. Results show learning gains reveals that trust is a prominent factor in users' perceptions of usefulness. We furthermore propose an extension to develop a retrieval framework that can be used to directly optimize the set of interactions that a user may take in order to extract the maximum utility of a document. Using vocabulary learning and searching-to-learn as a foundation, we propose both an algorithm and user study to evaluate effectiveness in jointly considering relevance and familiarity with technical terms to learn to ensure users get the most out of the documents they search for.

The theme linking these studies together is a focus on improving user behavior to reduce effort or time-on-task, and increase value over time during interactive search. This dissertation serves as a basis for future system design and experiments that preserve interactivity, encourage effective mental models, and reduce user effort while increasing the value users receive during the search process.