Detecting and Overcoming Trust Miscalibration in Real Time Using an Eye-tracking Based Technique

Abstract

The introduction of automation technologies to various application domains has resulted in improved efficiency and precision of operations. However, it has also created challenges. One problem that has recently received considerable attention is trust miscalibration, i.e., a mismatch between a person’s trust in automation and the actual capabilities and reliability of the system. Trust miscalibration can refer to too much or too little trust in a system which, in turn, leads to misuse (e.g., overreliance) or disuse (e.g., slow adoption) of automation. Avoiding these undesirable outcomes requires a better understanding of, and support for trust calibration – the focus of the proposed research. To date, most studies on trust suffer from a number of limitations, including highly intrusive techniques for measuring trust, a limited understanding of important factors affecting the process of trust development and a lack of effective countermeasures to deal with trust miscalibration. To address these shortcomings, the goals of this dissertation were to (1) develop an eye-tracking based technique to infer trust levels and variations in real time, (2) identify how the magnitude and duration of changes in system reliability affect the process of trust evolution and calibration, and (3) develop and evaluate the effectiveness of a real-time intervention (an audio alert) for supporting trust calibration. To this end, a series of empirical studies were conducted in the context of an Unmanned Arial Vehicle (UAV) control simulation. Participants were required to perform two tasks in parallel: a tracking task and a target detection task, the latter with the assistance of an imperfectly reliable automated system. Subjective trust measures, eye movements, behavioral data, and performance outcome data were recorded. As expected, participants in the first study monitored low-reliability UAVs more closely, as indicated by a set of eye-tracking metrics. Variations in their monitoring behavior aligned with their subjective trust ratings, suggesting that eye tracking is indeed a promising less intrusive technique for inferring trust in real time. The second experiment showed that participants were sensitive to four types of UAV reliability changes that differed with respect to magnitude and duration. Both duration and, even more so, magnitude affected participants’ trust calibration and recovery. The large and long reliability drop in system performance had the most severe negative impact on trust and target detection performance. To prevent performance breakdowns due to trust miscalibration, an eye-tracking based trust inference system using a k-Nearest Neighbor algorithm was developed and used in Experiment 3 to trigger audio alerts in case of a divergence between system reliability and participant trust in the system. The audio alert was successful in improving trust calibration and contributed to faster trust recovery following a period of low system performance. However, these improvements did not translate into better performance on the target detection task. The findings from this dissertation add to the knowledge base in trust in automation. At a methodological level, a new nonintrusive was developed and validated. At a conceptual level, a better understanding of the effects of variations in the magnitude and duration of system reliability changes on trust and performance was gained. And at an applied level, a candidate countermeasure to trust miscalibration was designed and tested. Ultimately, this research helps prevent catastrophic consequences due to inappropriate reliance on automation, and thus contributes to safer operations in a wide range of application domains.