Evaluating the Impact of Statewide Supports to Reduce Within-School Achievement Gaps: A Mixed Methods Study of Focus Schools in Michigan

Abstract

Until the passage of the Every Student Succeeds Act in 2015, states were required to submit a state-developed accountability plan in exchange for leeway on NCLB accountability provisions. Among the many requirements that had to be addressed in the state-developed accountability plans, state recipients of ESEA waivers had to identify Focus Schools, or the 10% of schools in a state with the largest achievement gap. In this set of papers, I examine whether this accountability-driven, state-led effort to reduce within-school inequality have been successful in Michigan. To do so, I first examine the extant literature on school improvement to provide historical and policy context for school improvement efforts. In Chapter 2, I use a quasi-experimental regression discontinuity design to determine the causal impact of a bundle of services offered to schools in Michigan to narrow the within-school achievement gap in student achievement. Findings demonstrate that supports provided in Michigan likely did not reduce within-school inequality or raise average school achievement during the first two years. However, one caveat for interpreting these findings is that the study is underpowered and therefore may not be able to detect real effects at the school level. In Chapter 3, I use a qualitative case comparison design to determine what organizational features of school supports were influential in the implementation of the treatment. Findings indicate that the interventions detailed above comprise a loose bundle of solutions to the problem of within-school inequality. Taken together, the treatment is relatively weak in its prescription to ameliorate achievement gaps. Consequently, its success depends in large part on the capacity of the organizations and individuals tasked with executing the interventions. This study shows that it is not only capacity but coordinated capacity—a systemic effort to harness individual and organizational capacity—that is important to achieving gains in student achievement for the bottom 30% of students. Chapter 4 turns to the puzzle of how that coordinated capacity can be developed and harnessed to achieve the policy goals set forth in ESEA legislation. In this study, I examine one potential solution to this capacity problem, a Networked Improvement Community approach to school improvement in Focus Schools. I explore the potential of improvement science methods to achieve the aims of Focus School identification across two mid-sized cities in Michigan. This case study examines how such a community can be initiated and sustained in the context of federally-legislated, standards-based reforms and probes the extent to which such an approach can deliver on the premise of capacity building for school improvement. Taken together, this set of studies asks and answers whether identification of Focus Schools is sufficient to reduce achievement gaps; how coordinated capacity at each level of governance—state, ISD, district, and school—influences policy implementation; and how a Networked Improvement Community approach may provide a potential solution to building this coordinated capacity within states. The findings have implications for the way schools and districts in particular structure improvement efforts, particularly under the auspices of the Every Student Succeeds Act (ESSA), the most recent authorization of the ESEA, which devolves considerable responsibility for implementation of policy goals to states.