Commentary on Harris and Larsen’s New Orleans OneApp Report, What Schools Do Families Want (And Why)?
On January 15, 2015, Education Research Alliance (ERA) founder Doug Harris and Tulane post-doctoral fellow Matthew Larsen published a study (and technical report) in which they analyze 2013-14 data primarily procured via the New Orleans open enrollment process known as OneApp. The name of the study is What Schools Do Families Want (And Why)?
Here are some observations about their study:
First, in their comparison of school performance scores pre-Katrina to post-Katrina, Harris is aware that the Louisiana Department of Education (LDOE) even awards some schools points for students whose scores are not proficient on state tests.
Consider this statement from the Harris/Larsen OneApp analysis:
After Katrina, the lowest-income families had greater access to schools with high test scores. School bus transportation systems expanded, average test scores increased across the city, and schools with higher test scores were more likely to locate near lower-income neighborhoods. Pre-Katrina public schools zoned for the highest-income neighborhoods were 1.3 letter grades higher than schools zoned for low-income neighborhoods; the difference between the lowest- and highest-income neighborhoods dropped to just a half letter grade considering the nearest schools after Katrina.
It seems that Harris and Larsen are equating higher school performance scores with higher test scores. As noted above, the LDOE incorporation of “bonus points” for non-proficient students boosted school performance scores, and RSD benefited from this practice.
Also, not sure how useful the above pre- to post-Katrina school grade comparison is given that there is no anchor. That is, the “closing if the letter grade gap” could mean that the highest letter grades have fallen. It doesn’t necessarily mean that the highest remained stationary while the lowest rose. Also, the highest-to-lowest income ratios are not necessarily the same pre-Katrina versus post-Katrina.
The degree to which the letter grade “gap closure” is an artifact of the post-Katrina mixture of income levels brought about by open enrollment remains unclear.
Moreover, school letter grades and performance scores serve as a fine example of high-stakes numbers easily gamed. Whereas Harris and Larsen re-scaled performance scores to compare pre-Katrina with post-Katrina school scoring outcomes, since 2011-12, the public has only “seen” the letters A B C D F and not the alterations in scoring that make those letters not directly comparable from one year to the next. Therefore, in 2011-12, a school with a D could have had a C in 2012-13 simply due to changes in calculation. However, the public “sees” the grade as “improved.” A deception.
Additionally, Harris and Larsen comment that “very-low-income families also have greater access to schools with high average test scores.” However, even with inflated school performance scores, most RSD schools continue to be rated as C, D, or F, the definition of a “failing school” by the original Louisiana voucher standard. The schools that have consistently been “high average test score” schools are those that were not taken over by the state post-Katrina and continue to be with the Orleans Parish School Board (OPSB). General “access” to “higher average test score” schools might be “greater,” but it remains limited.
Next, Harris and Larsen note that “practical considerations” prevent parents from choosing higher-test-score schools. Indeed, it could be that so few A and B schools are available for parents to “choose,” especially given that many of these are selective-admissions schools, that the limited choice of a C school over a D school does not entice parents to choose to a greater degree based on academics.
That noted, Harris and Larsen also hit on a point that is worth mentioning: They state that the parents’ income plays into those “practical considerations” that influence choice of schools. Thus, one might offer the “choice” of attending a school with a higher school letter grade; however, family income is a systems issue in which a single choice (school attendance) must fit the greater functional environment of the family.
In the OneApp analysis, it is interesting that lower-income families value the social aspects of school more so than do the higher-income families. It could be that lower-income families depend upon school to provide more social opportunities for the students as well as for the family and community than do wealthier families.
Harris and Larsen touch on social importance in cautioning the interpretation of thier results:
Supporters and critics of the New Orleans reforms may latch on to some findings over others, but these results should not be oversimplified. The fact that families have “more choices” and seem to actively exercise choice does not necessarily mean that communities are better off. Education fulfills social goals that go beyond what “consumers” might want. Also, while surveys suggest that the average parent is satisfied with the choice-based system, some parents may prefer the types of neighborhood schools that no longer exist with the elimination of neighborhood attendance zones. [Emphasis added.]
In their next statements, Harris and Larsen are a bit too trusting that what they are reporting reflects positive outcomes:
At the same time, while very-low-income families are less likely than moderate-income families to choose schools for their academic outcomes, very-low-income families are not necessarily worse off academically. First, there is some evidence that average academic quality has improved and become more equally distributed across the city. Second, the reforms allow schools to develop specialized programs that attract like-minded families and teachers and may help build an engaging school culture—and higher achievement. Third, our evidence suggests that some parents have strong preferences for academics and these parents could influence the market in a way that improves academics for all students.
A distinction must be made here between “academic quality” and increased test scores. If Harris and Larsen are determining “academic quality” based on high test scores, they should consider that schools could be doing little more than test drill in an effort to raise test scores. Furthermore, “equal distribution of academic quality” could be an accepted regiment of test drill throughout RSD charters.
Test drill might raise test scores, but it does not promote a well-rounded education.
Academic quality is also influenced by school stability, which is influenced, in turn, by staff stability.
Given the dependence of RSD charters on temporary, inexperienced, testing-outcome-focused staff as is provided by Teach for America (TFA), and given the charter operator turnover that is signature of the current national under-regulated charter sector, research into the details of such school and faculty stability is indispensable to competently assessing the academic quality of New Orleans charter schools.
As to “specialized programs attracting like-minded families” and parents “influencing the market,” one must assess the degree to which such programs serve to segregate a community into “haves” and “have nots.” Selective admissions charters in particular should be regularly audited to see the degree to which their admissions policies (including the advertising of open seats) purposely excludes certain subgroups of students.
As it stands, RSD charters are long overdue for an audit, and that includes a publicized detailing of charter school closures and reopenings and the reasons for such closures/reopenings, and faculty turnover, also including reasons.
Then there is the issue of school data readily available to researchers (not just LDOE-approved researchers) and to the public in general. Moreover, even as I write this, Louisiana State Superintendent John White continues to withhold Louisiana schools’ class of 2014 ACT scores from public view. Thus, data that might inform the public regarding the “college and career readiness” that has been emphasized ad nauseam of late is intentionally kept from that public. Withholding of information is an underhanded means of manipulating “choice” and of preventing from happening any informed accountability of educational policy that drives such “choice.”
I appreciate what Harris and Larsen have tried to offer in this OneApp study. Nevertheless, the real meat of RSD research has yet to happen. It must include examination of the quality of education offered by charters as such is expressed beyond the narrow prescription of test scores and test-dependent indices. It must include regular audits of finances and of educational offerings. It must include examination of student attrition and school discipline policies, of charter school and faculty turnover. And it must include true transparency via ready availability of de-identified data for any researcher properly filing a public records request.
Harris and Larsen close their study with the following statement:
Choice is not enough and it is only real when parents are well informed and can readily access the schools they prefer.
If New Orleans parents want to keep their community schools and “choose” charters from there, they have no choice. None.
Is “choice” really “choice” when it is a system imposed from outside without community empowerment from the outset? Hardly.
Whether Harris’ et al. work gets to the meat of the matter remains to be seen.