Charter School Academic Accountability and Value-Added Measurement of Student Academic Performance

According to the latest data analysis conducted by the Center for Education Reform, the strength of the laws governing a particular state's charter schools is the key factor in determining high student academic success. Not surprisingly, by "strength," the Center means how far state laws are willing to go to hold the schools accountable for their students' performance. The data show conclusively that a higher degree of accountability makes for consistently stronger, higher-performing schools, regardless of population, particular mission, and/or funding levels. The only apparent flaw in the Center's findings is that it has seemed to engage in circular logic: its research began with an assumption that charter schools "have an attitude that doesn't accept mediocrity" (p. 1), accepting students that other schools frankly would rather do without into their ranks and refusing to lower their standards. In short: charter schools are presumed to be self-regulating, holding themselves to higher standards than public schools. Given this assumption, it makes sense that strong state laws bolstering this climate of accountability would be viewed as positive contributors to academic performance.

In fact, the Center also alleges that states like New York enjoy high charter-school success (as measured through standardized test scores) because to them, "accountability means being willing to close schools" (p. 2). Is this kind of accountability what the originators of charter schools had in mind; the closing of schools that for whatever reasons cannot prove student success according to the same measures employed by public schools? Dissenters would argue that closing schools when they do not meet state requirements for student academic performance does little more than boost overall state ratings -- and make other charter schools look even better by comparison -- while the practice short-changes students who might have been thriving in the closed schools (albeit in a fashion not measurable by the state).

The bottom line is that the approach of using traditional assessment instruments to evaluate student academic performance in charter schools is problematic, for two main reasons. First, by their nature, charter schools are non-traditional. Often created to serve a specific population and/or to offer education from a specific curricular or pedagogical stance, charter schools are intrinsically different from "regular" public schools; therefore, assessment instruments need to be changed according to the differences that exist. Second, often these "specific populations" are at-risk students or drop-out recovery students. Such students might well show significant progress in a charter school setting, but on the macro level, the school itself might not show the progress mandated by the state because its students are proportionally comprised of students coming from a deficit perspective.

And so the challenge becomes how to hold charter schools accountable for student academic performance in ways that make sense (and appear fair and balanced) to students, legislators, parents, and the general public. After all, Miron and Nelson note that "at the heart of the charter school concept lies a ‘bargain': schools will receive more autonomy in operations in exchange for being held more accountable than other public schools for student outcomes" (p. 1). Yet, perhaps because of the difficulties inherent in measuring a new form of education with old assessment instruments, out of 38 states with charter mandates, only 8 have provisions for independent, comprehensive, updated evaluations that specifically measure student performance. Moreover, in their meta-analysis of charter school-related studies, the authors noted no studies that employed non-traditional measures of student performance. Indeed: in one of the "strongest" of the studies, student performance was measured using state test scores gains, while no mention was made of alternative assessments.

The impact of this problematic application of traditional measures to non-traditional educational venues can be seen in reports such as the "Ohio Charter School Fact Sheet" published by the Ohio Federation of Teachers in 2005. According to this document, 71 percent of all Ohio charter schools are failing (meaning they fell under either Academic Emergency or Academic Watch), while only 10 percent of public schools have earned this dubious distinction. Upon what measure of academic performance do they base this statement? Standardized test scores; again, not necessarily the best measure of progress in schools with non-traditional curricula, pedagogical approaches, and/or student bodies.

However, others in Ohio seem to be aware of this methodological problem. In a 2005 Progressive Policy report on the state's charter schools, it was noted that while "Ohio charter schools have lower test scores and proficiency rates than Ohio public schools overall....they also serve much higher percentages of disadvantaged and minority students, who are more challenging to educate." To counter this dilemma (affecting not just the educational process itself, but also its evaluation), Ohio will soon begin a practice of what it calls "value-added measures" to the process of assessing student academic performance in charter schools. This technique will statistically account for the often lopsided population aggregates of at-risk and/or drop-out recovery students in charter schools. At present (i.e. until this technique is adopted), the state is using a "growth calculation," which will allow "schools to advance one ratings scale (Academic Emergency to Academic Watch, or Academic Watch to Continuous Improvement) if they have improved PIS by 10 points in past two years and at least three points in past year" (Russo, p. 20). In other words, the state will give credit for actual progress made toward predetermined educational destinations, rather than simply determining whether or not those destinations have been reached and leaving it at that.

This would fall much more in line with what Opp, Hamer, Beltyukova, Barton, and Ogawa call the "common denominator" of all charter schools: the mandate to improve student academic performance. The key concept is "improve," and again, a critical distinction needs to be made between achieving a goal and progressing toward a goal. It is worse than comparing apples to oranges when test scores of public school populations are compared with, for example, those of charter schools specifically designed to cater to those students whom the public schools have already failed. Furthermore, to demand that charter schools simply ensure that all their students reach these goals without any recognition of real progress that is being made toward those goals defeats another underlying commonality among charter schools: a philosophy of justice. Charter schools (many of them) were begun to help students who were unsuccessful in the traditional public school model of education. Different students and different schools require different standards.

This word "different" should not be read to mean "inferior," or "easier." This is not a call to dumb down the expectations for charter school students so as to more easily proclaim them as having met the academic standards of their schools and in this watered-down way release those schools from their accountability to student success. It is, however, to acknowledge that the student who travels from an F to a C is, in many ways, more of a success story than the student who moves from a B+ to an A-, and there needs to be a valid way to measure this movement.

This call is echoed by current researchers, who note the "multi-dimensional nature of the missions of most charter schools" and thus "recommend more nuanced, comprehensive approaches to the assessment of the success of charter schools and their students." Moreover, as the results of this study indicate, it is not only the researchers themselves who are calling for a more varied, relevant approach to measurement, but also the parents and teachers themselves; and if the rhetoric of the move away from mandatory public school education toward school choice is to be believed, it is the parents in particular who are the most critically-regarded constituency. One possible approach, according to the authors, would be a comprehensive, holistic authentic assessment process that could be conducted on a periodic basis, and that would be based upon the concept of talent development and cultivation as opposed to a more "banking" approach to education (in which students are seen as empty boxes in which teachers deposit knowledge -- as for a standardized test, for example).

The primary challenge to holistic assessment techniques is, of course, monetary. One of the primary attractions of charter schools is that they hold the promise of accomplishing at least as much as public schools with at least as little the way of resources. The search for grants and donations to support underfunded charter schools is an often-enough told occurrence that it has entered the realm of common knowledge among educators. So, to add a truly authentic evaluation of student academic performance to its list of fiscal responsibilities seems formidable at best and impossible at worst. However, there do exist cost-effective ways to demonstrate student improvement, such as portfolio construction. In the meantime -- until there exists a uniformly-applied , contextually-relevant approach to the evaluation of charter schools -- states like Ohio can (and should) continue to utilize "growth calculations" and/or "value-added methodologies" when analyzing and interpreting school and student performance data.

References

Miron, G., & Nelson, C. Student academic achievement in charter schools: What we know and why we know so little.

Ohio Federation of Teachers. Ohio charter school fact sheet.

Opp, R.D., Hamer, L.M., Beltyukova, S., Barton, T., & Ogawa, M. Assessing charter school success: A pilot study examining the utility of parent and community involvement and talent development in four Lucas County Ohio charter schools.

Russo, A. Charter schooling in the Buckeye State. Progressive Policy Institute Website.

Strong charter laws produce better results: A special report. Center for Education Reform.