Using Evidence to Understand and Improve Instruction and Schools: Multiple Methods and Comparative Perspectives

Abstract

This dissertation consists of three papers.

"Online Learning, Offline Outcomes: Online Course Taking and High School Student Performance." This paper uses fixed effects models to estimate differences in contemporaneous and downstream academic outcomes for students who take courses virtually and face-to-face, both for initial attempts and for students taking courses for credit recovery. We find that while contemporaneous outcomes are positive for virtual students in both settings, downstream outcomes vary by attempt type. For first-time course-takers, virtual course-taking is associated with decreases in the likelihood of taking and passing follow-on courses, and of a measure designed to proxy for graduation readiness. For credit recovery students, virtual course-taking is associated with increased likelihood of taking and passing follow-on courses, and of being in line for graduation. Supplemental analyses suggest that selection on unobservables would have to be substantial to render these results null.

"Bridging School Data Use in Principle and in Practice." This paper seeks to better understand the practice of data use for improvement in schools, as well as how well research literature on school data use captures the process as it unfolds in practice. To do so, I first synthesize leading research literature to create a framework reflecting the “in principle” guidance for school data use. This framework is then used to describe and understand the practice of data use in a high-data-use school over a two-year case study. My primary finding is that while the “in principle” guidance for data use captures many important components of the process, it understates the complexity in the relationships between these components. The consequence of this understatement is a bifurcation of the supports required for data use and the process of school data use, belying the reciprocal interdependence that I argue better describes their connectedness.

"Allocations and Expectations: Comparing Teacher Reports of Virtual and Face-to-Face Instruction." This paper relies on a teacher survey asking teachers to describe their experiences teaching both in face-to-face classrooms and in a well-established statewide virtual learning system. The responses are used to create two groups, face-to-face and virtual, made up of the same set of individuals. I then compare the similarities and differences between the two learning modalities. Statistically significant differences are found in the time allocated to several common teaching tasks, in several aspects of the work environment, in expectations for common materials and practices, and in the overall impression of the curriculum and materials. Ultimately, my findings point to a need for more research of this type to better understand the practice of virtual instruction and how it differs from traditional face-to-face instruction. For policy and practice, this work also suggests a need for greater recognition of these differences, and the strengthening of systems to support and prepare teachers and administrators to be effective in virtual instruction.