Short-term Event Tracking in Dynamic Online News

Abstract

When an important event happens, such as a terrorist attack or natural disaster, many people turn to the World Wide Web to keep track of the most current information. Because large numbers of online agencies report on such events, and continually update their stories, the Web provides timely access to a variety of perspectives. However, following facts in a breaking story is challenging for a number of reasons. For example, news agencies have their own reputation and agenda, such that sources often contradict one another. In addition, it takes time for accounts of stories to stabilize and to be accepted as the ground truth, such that previously reported information is often corrected. Information retrieval applications, such as text summarizers and question answering systems, are designed to help users find relevant information effectively when faced with large amounts of text. However, they typically do not account for the fact that information may be time or source sensitive. The current thesis works towards designing tools that can support users in following dynamic information, by focusing on the problem of finding facts from sets of related news articles, published while a news story is developing.

Based on the findings of a corpus analysis, as well as an annotation experiment, a prototype system was built. An important finding was that when presented with a factual question and a set of articles about a story, users agreed on which sentences reported answers to the question. However, the agreement as to which answers were new, or had changed with time, was no better than expected by chance. Therefore, rather than detecting changing information, the system finds sentences that are relevant to an input question, and presents them to the user with their respective publication times and sources. The system was evaluated intrinsically and extrinsically with significant results. In particular, in a task-oriented user evaluation, in which the use of the system was compared that of another state-of-the-art system, it was shown that users exerted less effort in searching for the answers to questions with the new system, while obtaining the same level of task accuracy.