The True Mis-measurement of Education

On Friday, June 19, 2015, I spent the day attending the Educational Research Alliance (ERA) conference in New Orleans.

The title of the conference is The Urban Education Future? Lessons from New Orleans Ten Years After Katrina.

ERA Director Doug Harris, who I have written about more than once and with varied degrees of trust and acceptance, graciously allowed me to attend via media credentials.

My first knowledge of Harris came when I learned that he was the researcher who wrote the value-added model (VAM) textbook (2011) for which American Federation of Teachers (AFT) President Randi Weingarten wrote the foreword.

So, Harris and his fellow ERA researchers are into VAM.

They call it “measuring growth.”

One of the ERA conference sessions I attended was entitled, Measuring (and Mis-measuring) School Performance.

It was a presentation about adding a VAM component to school performance scores (SPS). I agree that SPS does not accurately capture student growth. However, I have yet to read any convincing empirical evidence that adding VAM into any high-stakes cocktail would improve (let’s just nebulously capture it in two words) “measurement outcomes.”

Nevertheless, in the panel discussion that followed (which included Margaret Raymond of Stanford University’s CREDO; ERA researcher Robert Santillano; New Orleans College Prep charter management founder/CEO Ben Kleban; Algiers Charter School Association charter management CEO Adrian Morgan, and Georgetown University education policy analyst Thomas Toch), it was clear that additional quantification was the preferred means of “improving” upon the “mis-measurement of education.”

As I listened to the discussion, I thought of specific students whom I have taught and whose learning outcomes, when quantified (not “if,” but “when”) would likely do my “quantified teacher value” no favors.

I have taught for over two decades, and thinking of students in terms of their “quantified value” in supporting my career is an issue that was not even a remote slice of my reality in 1991 when I began my teaching career.

But it is here now: the high-stakes testing standoff. Test-score-driven reality versus the ethical high road. “Me first” or my kids.

I choose my kids.

I will not refuse to teach a student who may not “pay off” for me. But each time I choose the student over potential consequences to my career, I am aware of the stakes that have been placed upon my career by those who presume that I can morally and ethically control my students’ test score outcomes.

Teaching has become high-stakes numbers, like casino gambling. Some people will go to great lengths to win.

There is a reason why casino owners invest heavily in security:

They know that when the stakes are high, the system will be gamed.

Even as I was sitting in that ERA session and hearing the conversation drift back to That Which Can (Must!) Be Quantified, a story was breaking in New Orleans.

It was about gaming the testing system in order to ensure a high-stakes outcome: funding.

Marta Jewson of The Lens posted the following on June 19, 2015:

ReNEW CEO Encouraged Retesting of Students So Network Could Get Grant

The head of a five-school charter network in New Orleans put pressure on school leaders to increase test scores — even encouraging them to retest struggling students and offering to pay for inducements for the kids  — so the organization could earn money from a national nonprofit, according to emails released Thursday.

“If you did have the students take it and they we not motivated to do their best PLEASE regive it next week,” ReNEW Schools CEO Gary Robichaux wrote in a May 8 email, complete with emoticon. “I will pay for any incentives you would like to give the kids for doing their best on this FINAL test!! :-)”

The exam at issue, the STAR assessment, is not one that’s used as part of the state accountability program. It was chosen by the network to measure growth, with the test given at the beginning and end of the academic year.

Last month, the top two officials at ReNEW’s SciTech Academy abruptly resigned, andRobichaux said at the time it was because of testing irregularities — including students retaking the test. Head of School Tim Hearin and Principal Alex Perez also let students take the test for one another and take the tests at home, Robichaux said then. Neither Hearin nor Perez could be reached for comment today.

Reached today by phone, Robicheaux now says having the students retake the test is acceptable. However, he said because of the other irregularities, the network would not rely on or submit the results from the test to support a grant request.

“But obviously I’m not going to give it to them now with what happened at SciTech,” he said.

Likewise, ReNEW board President Brian Weimer wasn’t troubled by the retesting.

“My understanding is that with STAR, it is permissible to retest students,” he said. “The board takes all these issues very seriously and we’re aware and cooperating with the state and will do what we need to do to resolve any problems and issues.”

In the email, released Thursday evening by ReNEW, Robichaux explains why improved test scores are critical.

“Leaders, in order to secure funding from Charter School Growth Fund we need to present them our STAR data later this month,” he writes. “Of course they will be looking for growth in this nationally normed test over this year.”

The Charter School Growth Fund is a nonprofit based in Colorado that supports charter schools with underserved students. The organization released this statement in response to a question about whether the test irregularities would affect ReNEW’s eligibility for funding:

“We look at many factors when considering applicants, including achievement results. We ask applicants to take a nationally normed test to measure student growth, and we expect them to adhere to certain recognized standards around test administration.”

Robichaux’s email was passed along to school leaders and also includes this line from Sumeet Goil, the network’s director of data and assessment:

“It’s even more important now considering this data factors into the new teacher compensation system.”

Eventually Hearin passes along the email to two people. ReNEW has redacted one of the names in accordance with their whistleblower policy.

It reads:

“check out this email, and the attached doc, and please get it done. remember that all kids need to grow significantly. whatever it takes bro. delete this message. hit me up with questions in person. failure is not an option!”

In an interview two weeks ago, Robichaux said grant money was not at stake. He said the Charter School Growth Fund would look at PARCC scores, a new state test, but the email released Thursday contradicts that statement.

Though ReNEW provided a sheaf of emails to The Lens in response to a June 1 public-records request, they noted they were withholding one out of concerns about revealing a whistleblower. When The Lens asked the network to cite the legal justification, ReNEW Chief of Staff Colleen Mackay instead sent over this email thread.

Renaissance Learning, the company that authors the STAR exam, did not return a request for comment.

The network alerted the Louisiana Department of Education after The Lens published its first story. A representative sent this today in response to questions:

“The Department continues to work with ReNEW Schools to gather information related to concerns over assessments and will soon determine if further inquiries are warranted. At this time, the Department has not issued any notices to ReNEW Schools.”

High-stakes, test-centered “accountability”:

The True Mis-measurement of Education.

________________________________________________

Schneider is a southern Louisiana native, career teacher, trained researcher, and author of the ed reform whistle blower, A Chronicle of Echoes: Who’s Who In the Implosion of American Public Education.

She also has a second book, Common Core Dilemma: Who Owns Our Schools?, newly published on June 12, 2015.