Experimentation Methodologies for Educational Research with an Emphasis on the Teaching of Statistics.

Abstract

In this thesis, I explore the state of quantitative research in the field of statistics education. First, a content review from several prominent sources in statistics education is conducted. Based on this review, recommendations are made for advancing methodological research in this field.

Next, the design and analysis of a randomized experiment in an introductory statistics course are presented. In this experiment, factorial and crossover designs were used to explore several implementation aspects of ``clickers'', a technology for collecting and displaying, in real time, student responses to questions posed by the instructor during class. One goal was to determine which aspects were most effective in helping improve engagement and learning; another goal was to explore issues involved with implementing a large-scale experiment in an educational setting. The aspects explored were the number of questions asked, the way those questions were incorporated into the material, and whether clicker use was required or monitored. There was little evidence that clicker use increased engagement but some evidence that it improved learning, particularly when a low number of clicker questions were well incorporated into the material (vs. being asked consecutively).

Finally, a strategy for exploiting interactions between design factors and noise variables in the educational context is examined. The objectives of this strategy are: 1) Identify a teaching method that is robust to the effects of uncontrollable sources of variation on the outcome, or 2) Identify when a teaching method should be customized based on a noise variable. Achieving the first objective is desirable when there is heterogeneity in the noise variable within a class, for example, when the noise variable represents characteristics of the students themselves. The second objective involves using information in the interaction to proactively customize a teaching method to particular groups, and is easiest for noise variables measured at the instructor or classroom level.