Semi-automated data analysis using sequence comparison algorithms: Computer-assisted problem-solving with graphs.

Abstract

The use of microcomputers can facilitate the detailed study of student learning processes in a naturalistic classroom context. This study examines the experiences of high school students using microcomputer graphing software to solve problems in scientific data analysis. Anecdotal classroom observations can be supplemented or refocused by the identification of common patterns in behavioral sequences ("traces" of problem solutions) recorded automatically and inobtrusively for a large sample of students during normal classroom activities. Such data demand innovative approaches to analysis and interpretation relying on both analytic computer processing power and holistic human judgment. The use of sequence comparison algorithms and cluster analysis to help make sense of a large volume of trace data is explored. Here the goal of computer-assisted data analysis is to render a complex task more manageable, rather than completely to automate it. Subject matter expertise and first-hand classroom experience on the part of the researcher are shown to be essential to both the appropriate coding of data for input to the quantitative analysis and the interpretation of its output. Sequence comparison methods are found to be useful for quantifying similarities and differences between behavioral trace sequences. An original, "balanced" measure, defined as a composite of a related pair of similarity and dissimilarity measures, performs especially well by the criterion of correspondence to painstaking qualitative judgments in a small-scale (3 problems, n $\approx$ 45) case study. Theoretical and practical implications of the use of various cluster analysis algorithms in conjunction with sequence comparison measures are also discussed. Ex post facto evidence for the validity of these combined methods is then presented through the results of an analysis of a body of data too large (9 problems, n $\approx$ 250) to be treated in its entirety by any strictly qualitative approach. A variety of behavior patterns are described indicative of great inventiveness and persistence on the part of students in the context of "though experiments" facilitated by the ability of computer software quickly to create and modify graphs. A version of the software incorporating "coaching" feedback is associated with the exploration of different major types of graphs before making less important changes. This approach is suggested as one general principle of teaching computer-assisted graphing.