Theory of Mind in Situated Communication for Collaborative Tasks

Abstract

Theory of mind is an extensively researched topic in psychology and cognitive science. It describes the ability to assign an agent a mental state representing: beliefs, desires, goals, etc., and understanding that another agent's mental state can be different than one's own. Humans acquire theory of mind at an early age which is critical to social interaction. In particular, theory of mind plays a vital role in maintaining common ground during human collaboration and communication.

Ideal integration of autonomous agents in a human world implies they can collaborate on human terms. First, we need a method to measure and infer the belief states of human partners. Second, we need to make sense of those belief states given the task the agents are involved in. Finally, the belief states and their grounding to the task at hand must be used to communicate effectively and make decisions that efficiently achieve the desired goal.

To enable theory of mind modeling in situated interactions between humans and AI agents, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in Minecraft's 3D virtual blocks world. It provides information that captures partners' beliefs of the world and each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for three theory of mind tasks that predict the partner's mental state.

We further incorporate theory-of-mind and collaborative task models for Collaborative Plan Acquisition, where humans and agents strive to learn and communicate with each other to acquire a complete plan for joint tasks. Specifically, we formulate a novel problem for agents to predict the missing task knowledge for themselves and their partners based on rich perceptual and dialogue history. We find that predicting the partner's missing knowledge is a more viable approach than predicting one's own. We show that augmenting with Theory of Mind (i.e., beliefs of others' mental models) produces improved and more stable results than without. These results provide insight for future AI agents that can predict what knowledge their partner is missing and, therefore, can proactively communicate such information to help the partner acquire such missing knowledge toward a common understanding of joint tasks.

To enable knowledge gap mitigation, we developed a model for decision-making, i.e., when to share information with their partner proactively. We define a symmetric role yet disparate knowledge and skill set task where two agents must sort objects based on a set of constraints distributed unevenly between the two partners. We show that an autonomous agent can make a meaningful estimate of their partner's knowledge disparity and effectively communicate missing knowledge to their partner. Furthermore, we also show that when paired with a human subject, behavior incorporating this belief tracking and knowledge mitigation is preferred and is perceived as a more effective collaboration by the human subjects.