Learning General and Correct Procedural Knowledge in a Cognitive Architecture

Abstract

For cognitive architectures that encode procedural knowledge as rules, online procedural learning algorithms exist that can analyze an agent's experiences to acquire additional procedural rule knowledge. Without careful engineering, agents using existing algorithms can learn large numbers of overly-specific rules that apply in limited situations. Moreover, these algorithms have difficulties summarizing some types of agent reasoning, producing rules that generate incorrect behavior. The limited generality of the rules learned, the uncertainty of correct behavior, and the amount of engineering required has limited the effectiveness of online procedural learning algorithms.

To better understand this problem, this work defines the two key qualities of online procedural learning: (1) correctness, whether the procedural knowledge generates the same inferences as the reasoning learned from, and (2) optimal generality, whether the procedural knowledge learned captures the generality of the reasoning learned from. These concepts are used to provide a detailed analysis of the underlying deficiencies of the most advanced procedural learning algorithm currently found in a cognitive architecture, Soar's chunking mechanism.

This work also provides an architecture-agnostic analytical framework to understand issues with online procedural learning. First, it provided four necessary conditions for correct behavior summarization, which can be used to organize the correctness issues of an online procedural learning algorithm into a taxonomy. Second, it provides four architecture-agnostic strategies that can be used to remedy issues with online procedural knowledge learning. Third, this work defines four desiderata for an online procedural learning algorithm that can be used to evaluate systems.

This work also describes a robust implementation of this approach. It describes how the four strategies were applied to Soar's chunking algorithm to create a new implementation of dependency-based procedural learning called explanation-based behavior summarization (EBBS), which includes algorithmic mechanisms that remedy or detect each of the 12 correctness issues identified. This work also describes how EBBS uses a novel unification algorithm called Distributed Identity Graph Unification (DIGU) that captures the generality in the reasoning being summarized, interfaces with the various mechanisms that improve correctness by using the formalism of object ``identity,'' and is uniquely designed to handle the computational demands that arise from online procedural learning in a cognitive architecture.

This work concludes with an evaluation that uses data generated from 14 agents across seven different domains. These experiments show that EBBS learns more optimally general rules that capture reasoning that previously could not be captured, effects correct behavior and improves agent performance without sacrificing agent reactivity.