Tool-Use Robot Manipulation Tasks for Cooperative and Explainable Operations in Safety-Critical Domains

Abstract

To be effective in assistive tasks, robots need to be capable of performing tasks programmed by non-expert users. Tool-use and assembly tasks are of particular interest as assistive tasks because they present many challenges such as reasoning over interactions between multiple objects and performing complex manipulation behaviors. Considering safety-critical domains further complicates robot reasoning by constraining these manipulation tasks and requiring that robots perform tool-use tasks subject to a wide range of safety considerations.

In this dissertation, we address the problem of reliable autonomous tool manipulation in safety-critical domains. Our goal is to advance planning and execution capabilities in tool-use object manipulation tasks through simple explainable models, enabling robots to engage in dialogue about safety on human-robot teams. We address the following challenges for safely executing tool-use tasks: (1) autonomously composing multi-objective behaviors (actions that satisfy multiple goals); (2) robustly modeling tool grasps and generalizing grasps to novel tools; and (3) reasoning over and engaging in dialogue about safety while performing tasks in different domains.

To perform multi-objective manipulation tasks, we explore reasoning over composable causality in furniture assembly tasks. We expect robots to autonomously compose behaviors to achieve given objectives without solely relying on qualitative observations from expert programmers. To formalize the composable causality of multi-objective actions, we propose a causal control basis. The causal control basis annotates elements of a typical control basis (set of controller behaviors) with causal information describing how a multi-objective action functions in an assembly task. The robot uses the causal control basis to estimate the likelihood that different compositions of behaviors achieve the intended effect. The causal control basis effectively reduces reliance on expert knowledge engineering for performing complex actions, making the execution of these behaviors more explainable.

To further improve dexterous robot manipulation, we explore grasp reflex modeling through tactile servoing for robustly achieving tool grasps in manipulation tasks. We propose a grasp reflex model , a simple explainable model that detects meaningful adjustable states describing the robot’s end-effector pose relative to the tool being grasped. Our trained grasp reflex model identifies statistically significant variables from the end-effector data, and when deployed on the robot, we find that our grasp reflex model achieves one-shot tactile servoing on 6 novel tool instances. Our proposed grasp reflex model is simple enough to be explainable and is reliable and generalizable enough to be trusted in tool manipulation tasks in safety-critical domains.

Towards tool-use and manipulation in safety-critical problem domains, we suggest that humans and robots must challenge each other’s assumptions, minimize overtrust, and characterize risks. To make robots active, trustworthy collaborators, we propose the human-robot red teaming paradigm for safety-aware reasoning. We demonstrate that a human-robot red team can engage in dialogue about safety and improve the team’s understanding of a problem domain. From these interactions, the robot learns to plan to complete tasks safely and mitigate risks during task execution. Safety-aware reasoning allows the robot to reason over and perform tool-use manipulation tasks alongside a human user under varying definitions of safety.

Taken together, our work emphasizes the importance of minimal expert knowledge engineering, interactions with non-expert users, and explainable methodology and models for robot manipulation capabilities. These factors justify the trust human users place in robot systems, and enable robots to reliably perform complex manipulation tasks on human-robot teams in safety-critical problem domains.