Models and Algorithms for Understanding and Supporting Learning Goals in Information Retrieval

Abstract

While search technology is widely used for learning-oriented information needs, the results provided by popular services such as Web search engines are optimized primarily for generic relevance, not effective learning outcomes. As a result, the typical information trail that a user must follow while searching to achieve a learning goal may be an inefficient one, possibly involving unnecessarily difficult content, or material that is irrelevant to actual learning progress relative to a user's existing knowledge. My work addresses these problems through multiple studies where various models and frameworks are developed and tested to support particular dimensions of search as learning. Empirical analysis of these studies through user studies demonstrate promising results and provide a solid foundation for further work.

The earliest work we focused on centered on developing a framework and algorithms to support vocabulary learning objectives in a Web document context. The proposed framework incorporates user information, topic information and effort constraints to provide a desirable combination of personalized and efficient (by word length) learning experience. Our user studies demonstrate the effectiveness of our framework against a strong commercial baseline's (Google search) results in both short- and long-term assessment.

While topic-specific content features (such as frequency of subtopic occurrences) naturally play a role in influencing learning outcomes, stylistic and structural features of the documents themselves may also play a role. Using such features we construct robust regression models that show strong predictive strength for multiple measures of learning outcomes. We also show early evidence that regression models trained on one dataset of search as learning can show strong test-set predictions on an independent dataset of search as learning, suggesting a certain degree of generalizability of stylistic content features.

The models developed in my work are designed to be as generalizable, scalable and efficient as possible to make it easier for practitioners in the field to improve how people use search engines for learning. Finally, we investigate how gaze-tracking and automatic question generation could be used to scale a form of active learning to arbitrary text material. Our results show promising potential for incorporating interactive learning experiences in arbitrary text documents on the Web. A major theme in these studies centers on understanding and improving how people learn when using Web search engines. We also put specific emphasis on long-term learning outcomes and demonstrate that our models and frameworks actually yield sustainable knowledge gains, both for passive and interactive learning. Taken together, these research studies provide a solid foundation for multiple promising directions in exploring search as learning.