Supporting Monitoring and Interruption Management in Complex Domains through Graded Multimodal Notifications.

Abstract

Operators working in complex data-rich environments, such as air traffic control, need to cope with considerable, often competing, attentional demands. They experience data overload in vision and audition, are required to timeshare tasks, and need to manage unexpected changes and events. Current technologies fail to support them in handling these challenges. This has led to breakdowns in performance and, in some cases, accidents. The goal of the present research is to develop novel informative types of notifications that minimize unnecessary attention switching and better assist operators in attention management. They inform operators about the presence of an interruption as well as its urgency and location, thereby helping operators avoid performance costs associated with attention switching. Based on the assertion that information can be better processed in parallel if distributed across modalities, these notifications employ and combine two under-utilized modalities: touch and peripheral vision. They are graded, i.e., their salience varies over time, to reflect changes in the importance of attending to an interruption. Gradation was implemented either intra- or crossmodally (i.e., within or across peripheral vision and touch). The proposed designs were expected to improve the detection of unexpected events and the decision making about attention switching without significantly affecting performance on ongoing primary tasks. A series of studies were conducted to a) identify effective tactile notification designs, b) compare the effectiveness of peripheral visual and tactile notifications, and c) evaluate 5 notification schemes that employ peripheral vision and touch as well as gradation. The findings from this research show significantly improved interruption management and overall task performance for all cued over uncued conditions, especially in the case of crossmodally graded notifications. They contribute to the knowledge base in multimodal information processing and display design as well as attention/ interruption management. This work goes beyond earlier studies by comparing the robustness of peripheral visual and tactile notifications under high workload and by exploring not only intra- but also crossmodal gradation in interruption cueing. At an applied level, it suggests ways in which future ATC operations can be supported more effectively to ensure the continued safety of the air transportation system.