Learning Integrated Relational and Continuous Action Models for Continuous Domains.

Abstract

Long-living autonomous agents must be able to learn to perform competently in novel environments. One important aspect of competence is the ability to plan, which entails the ability to learn models of the agent's own actions and their effects on the environment. This thesis describes an approach to learn action models of environments with continuous-valued spatial states and realistic physics consisting of multiple interacting rigid objects. In such environments, we hypothesize that objects exhibit multiple qualitatively distinct behaviors based on their relationships to each other and how they interact. We call these qualitatively distinct behaviors modes. Our approach models individual modes with linear functions. We extend the standard propositional function representation with learned knowledge about the roles of objects in determining the outcomes of functions. Roles are learned as first-order relations using the FOIL algorithm. This allows the functions modeling individual modes to be "instantiated" with different sets of objects, similar to relational rules such as STRIPS operators. We also use FOIL to learn preconditions for each mode consisting of clauses that test spatial relationships between objects. These relational preconditions naturally capture the interaction dynamics of spatial domains and allow faster learning and generalization of the model. The combination of continuous linear functions, relational roles, and relational mode preconditions effectively capture both continuous and relational regularities prominent in spatial domains. This results in faster and more general action modeling in these domains. We evaluate the algorithm on two domains, one involving pushing stacks of boxes against frictional resistance, and one in which a ball interacts with obstacles in a physics simulator. We show that our algorithm learns more accurate models than locally weighted regression in the physics simulator domain. We also show that relational mode preconditions learned with FOIL are more accurate than continuous classifiers learned with support vector machines and k-nearest-neighbor.