Relational learning by observation.

Abstract

In this dissertation, we investigate <italic>learning by observation </italic>, a machine learning approach to create cognitive agents automatically by observing the task-performance behavior of human experts. We argue that the most important challenge of learning by observation is that the internal reasoning of the expert is not available to the learner. As a solution, we propose a framework that uses multiple complex knowledge sources to model the expert more accurately. We describe a <italic>relational learning by observation framework</italic> that uses expert behavior traces and expert goal annotations as the primary input, interprets them in the context of background knowledge, inductively finds patterns in similar expert decisions, and creates an agent program. The background knowledge used to interpret the expert behavior does not only include task and domain knowledge, but also domain independent learning by observation knowledge that models the fixed mental mechanisms of the expert. We explore two learning approaches. In <italic>learning from behavior performances</italic> approach, the main source of information used in learning is behavior traces of expert recorded during actual task performance. In the <italic> learning from diagrammatic behavior specifications</italic> approach, the expert <italic>specifies</italic> behavior using a graphical representation, abstractly depicting the critical situations for the desired behavior. This provides the expert with additional modes of interaction with the learning system; simplifying the learning task at the expense of more expert effort. Both of these approaches are uniformly represented in relational learning by observation framework. Our framework maps learning an agent program problem on to multiple learning problems that can be represented in a supervised concept learning setting. The acquired procedural knowledge is partitioned into a hierarchy of goals and it is represented with first order rules. Using an <italic>inductive logic programming</italic> (ILP) learning component allows our system to combine complex knowledge from multiple sources. These sources include the behavior traces, which are temporally changing relational situations, the expert goal annotations, which are hierarchically organized and provide structured information, and background knowledge, which is represented as relational facts and first order rules. Our learning by observation framework needs to store large amounts of behavior data and access it efficiently during learning. We propose an <italic> episodic database</italic> as a solution, which is an extension of Prolog that improves Prolog by providing efficient and power mechanisms to store and query relational temporal information. We evaluated our framework using both artificially created examples and behavior observation traces generated by AI agents. We developed a general methodology to test relational learning by observation. Our methodology is based on first using a hand-coded agent program as the expert, and then comparing the decision making knowledge of the expert and learned agent programs on observed situations.