Louisiana School Letter Grades: The Games Reformers Play

Let me begin this post by stating that I am against the use of letter grades as a measure of school performance. Assigning a single letter grade to a school in order to determine school worth bespeaks the disguised agenda of privatization– nothing more.

Nevertheless, the grading of public schools abounds. Reformers insist that letter grades offer the public a quality identifier that is “intuitively understood” (straight from Jeb Bush’s Florida and officially introduced into the American Legislative Exchange Council [ALEC] playbook of so-called model legislation in 2010).

School Letter Grades: “Intuitively” Fooling the Public

For privatizers, the beauty of letter grades rests in the fact that no matter how the calculations behind the grades change– and no matter how the instruments used to measure “achievement change– and no matter how the numeric scaling of the letters change– the letters themselves remain the same. A, B, C. D. F. And since the letters themselves do not change, the public assumes that these grades hold a constant (and comparable) meaning: An “A” this year is “the same” as an “A” next year. A “D” last year and a “C” this year means that the grade “went up,” and that automatically indicates “improvement.” A “B” this year from an “A” last year means that the school must not have “performed” as well this year when compared to last.

Not so by a long shot.

There is no way to compare one year’s school letter grades to those of another year unless the instrumentation, the grading scale, and the formulas have all been held constant. No changes. No “tweeking.” Otherwise, comparisons from one year to the next are a lie.

The Best Shell Game in Town: John White and His Letter Grades

In his releasing two sets of letter grades this year– one calculated using the 2012 formula and scale and the other, using the “new” 2013 formula and scale– White inadvertently enables the public to see that the “new” 2013 school letter grades favor “moving” school grades to the middle. That is, the higher letter grades tend to “move” lower (in the direction of a “C”) from 2012 to 2013, and the lower letter grades tend to “move” higher (again, in the direction of a “C”). I discuss this false “movement” in my previous posts on the subject.

In 2013 Louisiana school letter grades, any “change” that contradicts use of the same formula, scaling and instrumentation as were used for the 2012 calculation is nothing more than an artifact of applying a “new” system.

The High School Grade Shell Game

Consider this graph below, composed by my colleague Herb Bassett. (Forgive the quality of the photo; Herb’s graph and explanation can be found here.) This graph is of schools designated as high schools. From left to right, the three sets of bar graphs correspond to 1) the 2012 high school letter grade distribution (i.e., the number of “A’s” through “F’s”; 2) the 2013 high school letter grade distribution if grades had been calculated using the same rules and scale in 2013 as in 2012, and 3) the 2013 high school letter grade distribution based upon White’s/LDOE’s “new” calculations and scale. Notice how few “A’s” the “new” 2013 high school distribution yields (the blue bar) compared to the proportion of schools graded “A” had the same rules applied in 2013 and in 2012. Notice also how “full” the middle of the 2013 “new” distribution is (consider the height of the orange, yellow, and green bars on the right as compared to the same color bars in the middle graph).

Imagine what it took to produce the third set of bars by transforming the middle set of bars– imagine compressing both the “A” and (to a lesser extent) “F” categories and “forcing” the middle categories to “rise.” This is White’s game. It reminds me of squeezing a closed toothpaste tube at both ends and forcing a bulge at the center of the tube. The amount of toothpaste hasn’t changed, just the pressure applied to the extremes. Yet White will not only  publicly boast of an obvious increase in toothpaste; he will attempt to sell his “toothpaste miracle” to school districts nationwide.

One means of White’s/LDOE’s “squeezing the extremes” of the high school toothpaste tube involves a trick of converting End-of-Course (EOC) test results into a point-value system designed to produce lower high school scores. Another “extreme squeeze” involves the “new” grading scale, which is biased in favor of lower-scoring schools. (For more details on the current rule changes, see Bassett’s details in this post.)

White/LDOE also prevented 11 out of 15 would-be (under 2012 rules) “A” schools from receiving Top Gains status and the associated financial rewards. In short, the “A” schools needed to increase their scores while being subjected to a scoring system biased against “A” school score increases. (After all, increasing an “A” school score defies the 2013 system bias of scores tending toward the middle [“C”]).

Schools other than the above would-be “A” schools were also penalized for not receiving higher grades based on White’s/LDOE’s illusion that the former 2012 and “new” 2013 letter grades can be compared in order to determine “progress” or “decline.”

Thus, White/LDOE made it possible for high schools to actually improve their scores yet be penalized for “decline” solely because White/LDOE changed the measurement rules.

And this capriciousness the public is supposed to “intuitively understand”?

Another Farce: K-8 “Bonus Points”

One means that White/LDOE employed to produce this “pressure at the extremes”– this lowering of the number of “A’s” and “F’s” and raising of the numbers of “B’s,” “C’s,” and “D’s”– involves the application of a “selective extra credit” maneuver known as “bonus points.”

White/LDOE have decided to award “bonus points’ to K-8 schools that make “significant progress with students who are below grade level.” Based upon the actual awarding of these bonus points in 2013, it seems that the best place for low-proficiency students is in what were the “D” and “F” schools under the former 2012 school letter grade rules.

Herb Bassett captures this Wonderland foolishness beautifully when he writes,

D and F schools are the best place for non-proficient students.  School choice and vouchers should not be used to take students out of these schools, because LDOE’s data indicate that these are where students are most likely to make significant progress toward proficiency.

However, Bassett is quick to offer this more reasonable conclusion:

LDOE has manipulated the Bonus Point system to skew the School Performance Scores through its three rule revisions within the last year.  The chart shows that schools with low SPSs calculated under the old rules are given a much greater boost through the Bonus Points.  This helps to explain why the new formula yielded fewer D’s and F’s.

Here is the chart Bassett references:

The red bars above represent percentages of K-8 schools receiving any number of “bonus points” at each “old” 2012 school letter grade level. The blue bars represent what is supposed to be the maximum number of bonus points a school can earn toward the 2013 school performance score (ten points). (Note: The percentage markers on the left side of the graph rise by tens. Sorry I cut them off. See a clearer graph here.)

In this reformer game, the “D” and “F” schools clearly benefit. However, with all of the reformer cries about “failing” schools, one might wonder why White/LDOE would obviously bias school performance scoring criteria in favor of “failing” schools.

I have three letters for you….

John White’s Favorite Brand of Toothpaste: RSD

Here is my hypothesis regarding the reasons behind the 2013 scoring bias illustrated in this post. First, the bias against “A” schools from the 2012 to the 2013 school letter grade calculation and scaling is designed to dull the shine on these schools. After all, state-run RSD in New Orleans– the supposed “miracle” worthy of nationwide emulation– looks bad by comparison. The RSD dream of surpassing those non-state-run school districts is dust. It has been eight years since the post-Katrina takeover. Eight years and RSD is no “A-B” district. Therefore, systematic reduction in the number of “A” schools serves to make RSD appear better.

That’s the squeeze on one end of the tube.

There is a second maneuver in the 2013 school score calculation and scaling, and it is necessary in order to give RSD the appearance of progress: Artificially raise the 2013 scores of schools rated “D” and “F” in 2012. Since such low-graded schools abound in RSD, the “new” 2013 scoring rules cannot help but benefit it.

That’s the squeeze on the other end.

The result: We have the likes of John White peddling his RSD Miracle to any unsuspecting district that doesn’t bother to examine the situation closely enough. So what if other schools and districts are casualties in this letter grading lie.

It’s every RSD for itself.

Remember to floss.