Influence of design decisions on student patterns of use: An example from Seeing Through Chemistry.

Abstract

The importance of instructional software <italic>design</italic> stems primarily from the <italic>teaching and learning activities</italic> it enables. While features can support students in making appropriate connections in their own developing knowledge structures, research has shown that even when software is designed for particular activities, the activities may not occur, and learning may be impeded. It is therefore important to examine students' actual patterns of use during software interaction. Previous research suggested some learners benefit from a hypermedia environment <italic>Seeing Through Chemistry</italic>, but more information was needed about the impact of various design features on student use and the relationship between use and learning outcomes. In this study, I quantify student use through base and revision patterns, examine impact of patterns on explanation quality, and describe how features may influence usage. Participants were 180 volunteers in their first college chemistry course where <italic>Seeing Through Chemistry</italic> was required. The study focused on two modules. Primary data sources were patterns of use extracted from computer-captured log files, and quality scores assigned (post-hoc) to student initial and final explanations for the 16 questions of interest. Consistent with expectations, results indicate that patterns of use were related to performance. Reading content sometimes led to a higher quality initial explanation, and in no situation did ignoring the content result in a better explanation. Revisers significantly improved, and had higher quality final explanations than students who did not revise. Referring to cards sometimes led to higher gain scores. Most students established a fairly consistent base pattern that they used throughout these two modules. However, their application of revision patterns was more variable. This suggests that the tendency to start with cards or questions (base pattern) was perhaps more a characteristic of the student, while the tendency to revise and refer (revision pattern) was more a reflection on the question itself or design features of the software. Students revised more often with harder questions and better model responses. Instructional designers should encourage revision of both right and wrong responses, provide clear feedback, and track student use. Furthermore, designers should encourage learning through moderately hard questions that require students to synthesize.