Cheating As We Worship: The Almighty Standardized Test
Contemporary America worships the standardized test.
Certainly corporate reform worships standardized tests because of the money to be made doing so. (Corporate reform ultimately worships financial profit.) Also, the American media so often swallows whole any test scores and publishes such as indisputable truth. And the American public–well– let’s just say that many apply no critical thinking skills when reading/hearing about how “well” or “badly” an institution scored on Some Standardized Measure Guaranteed to Broadcast Truth.
I had the idea to write this post while on my Tuesday morning duty in our high school gym. It’s a quiet duty spot; my goal is to make sure students do no loiter in the gym on their way from the parking lot into the main building. I am able to read while sitting there, and I have been reading Diane Ravitch’s Death and Life of the Great American School System. I was reading the section in which Ravitch discusses the numerous ways that teachers and administrators can “game the system” to increase test scores despite no evidence of actual student learning.
I also thought about two instances in my own life when a standardized test or the lack of one has directly influenced my career as a teacher. More on that to come.
One of my students asked me yesterday about how I was doing with my blog. I told him and the rest of the class about my idea for this post. Another of my students asked, “You mean how students cheat?” I said, “No, how teachers and administrators cheat.” She was surprised at my response.
Indeed, the contemporary problem of cheating on standardized tests has little to do with students trying to manipulate outcomes. That kind of “old fashioned cheating” predates the high-stakes testing movement. No, no, no. We have advanced in our cheating. Now, those on the testing war front lines are the teachers and administrators, not the students. Corporate reform has all but absolved parents and students of the responsibility for their test scores. If a score is low, it must be the teacher. It must be the administration.
In the end, I realize that the students and parents do lose when faculty are fired and schools are closed. But those currently facing at the standardized guillotine are the school administrators and teachers.
America’s relationship with the standardized test score baffles me. Perhaps my response is due to my training in measurement and statistics. I have seen the underside of the magic trick, so to speak. The numbers that are revered by so many because of their holographic infallibility are no mystery to me.
The great irony regarding my combined experience as a Ph.D. in statistics and a public school teacher is that I am not allowed to teach statistics in Louisiana public schools. The reason, as I heard straight from LDOE, is that Louisiana has no certification test for me to take in statistics, and since there is no test, then there is no certification. Never mind my Ph.D. Never mind my dissertation, or my blind-reviewed publications. Never mind my full-time, five-year university teaching history. Never mind my years as a reviewer of stats textbooks or my associate editor of research position with a national-level, blind review journal.
Never mind that I am able to teach the teachers of high school stats courses.
Never mind all of the rich experience that I could bring to a high school statistics classroom.
As for my teaching statistics in the Louisiana public schools, nothing matters in the absence of a standardized certification test.
I have a second story regarding certification tests.
I earned a minor in German in my undergraduate studies at LSU. I moved to Georgia in 1993 and accepted a position teaching middle and high school German for a year under the condition that in order to continue, I needed to pass Georgia’s certification test in German.
I could not pass the exam.
What happened in my classroom contradicted my inability to pass the exam. I was hired to replace a teacher whose personal problems resulted in her German I students only learning half of the German I curriculum. I was to teach these students German II. On my first day, I was surprised to encounter a classroomful of frightened students who pleaded with me not to set them up to fail. They told me that they were not prepared; that their teacher did not finish teaching the German I material. I assured these students that we would agree upon where their German I education ended and that I would begin there. I also told them that by the end of the year, we would cover all of the German II curriculum, as well.
That is exactly what we did.
In my German classroom, I had a double-sided sign that said, “English” on one side and “Deutsch” on the other. I spoke only the language designated by the visible side of the sign. I did this in both my German I and II classes. Most of the time, the sign said “Deutsch.” The kids moaned, and they struggled, but in the end, they learned German.
How do I know that my teaching was successful despite my failing the certification test? First, I had a group of German II students who chose to continue to the optional German III class. I later learned that my students who continued on to German III (with another teacher) were successful. One wrote to me to thank me for the preparation; she told me that the transition to German III was smooth.
Second, I had a German I student who won first place in a tri-state German poetry declamation competition. I asked my student, Lindsay, “Have you had any other German teachers?” She answered, “No. Only you.” I felt vindicated for my failed exam. I responded, “Good.”
But what of that German certification exam?
In my efforts to improve and pass the exam, I contacted the writer of the exam. He was a local German professor. He admitted that not as much time and money was allocated to the German exam as to other, more popular exams. It were as though the German exam were a sort of “necessary afterthought.”
I took the exam three times. Never was I offered any specific information on where I might focus my efforts for improvement. I was simply given an overall score and several subscale scores. And I never knew who determined the cutoff score or the rationale used for such a determination.
This ETS document notes the importance of being able to justify a cutoff score:
… There is no purely objective way to set standards. All methods of setting standards depend upon some type of subjective judgment. … You should be prepared to explain the reasons for the use of a passing score…. You should try to anticipate any harm that might be caused by the use of a passing score.
I lost my job.
I battled the stigma.
Given that I could successfully teach German courses and yet not pass the certification exam, I wondered whether anyone had bothered to connect the Georgia German certification exam with the abilities one must possess in order to teach high-school-level German. Did the man who designed the exam ever teach high school German? Did his university students have some “edge” based upon the fact that he wrote the exam that they would have to pass for certification?
I want to be clear here: I am not against certification exams, nor am I against entry-level standards for professionals. I think that entry-level professionals should demonstrate their suitability for their profession. However, I am against exams being hailed as the infallible, “end-all-be-all” evidence. I am against the subjugation of expert judgment to a numeric exam score.
I believe that high-stakes tests should be rarely used at all and never in isolation from other, unstandardized, subjective assessments, such as expert observation and judgment.
However, the reality is that we live in an age of mega-high-stakes assessment ever looming over the heads of public school administrators and teachers.
A psychometric vulture circling and waiting for the wounded to die.
Let the cheating begin.
One way that teachers and administrators cheat is the erasure method. That is, teachers or administrators take the students’ completed standardized tests, erase the incorrect answers, and bubble in the correct answers. Teachers could even hold weekend pizza parties while they alter student answers. Scores are sure to skyrocket.
The teacher might also save time and feed the correct answers directly to the students as they take the test. Or teachers could point out incorrect answers. Or teachers could produce study guides that include verbatim questions from the actual exam. These are messier alternatives since they leave a lot of witnesses.
Consider this USA Today excerpt:
[Seipelt Elementary School in Ohio’s] gains and losses are typical of a pattern uncovered by a USA TODAY investigation of the standardized tests of millions of students in six states and the District of Columbia. The newspaper identified 1,610 examples of anomalies in which public school classes — a school’s entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests.
Such anomalies surfaced in Washington, D.C., and each of the states — Arizona, California, Colorado, Florida, Michigan and Ohio — where USA TODAY analyzed test scores. For each state, the newspaper obtained three to seven years’ worth of scores. There were another 317 examples of equally large, year-to-year declines in an entire grade’s scores.
The higher the standard deviation, the rarer that improvement is. In dozens of cases, USA TODAY found 5, 6 and even 7 standard deviations, making those gains even more exceptional. (Note: A standard deviation of 3 is considered “high.”)
Large year-to-year jumps in test scores by an entire grade should raise red flags, especially if scores drop in later grades, says Brian Jacob, director of the Center on Local, State and Urban Policy at the University of Michigan. Such fluctuations by themselves do not prove there was cheating, but Jacob says they offer “a reasonable way to identify suspicious things” that should be investigated. [Emphasis and commentary added.]
Another general means of cheating is the less-direct method of altering the group of test takers. Teachers or administrators might, for example, alter the classification of certain students, such as special education students, prior to a high stakes test in order to maximize scores. Consider this recent Louisiana story:
Dr. Gary Jones, former Superintendent of Rapides Parish schools and current Assistant Superintendent with the Louisiana Department of Education explains in [a] letter … that districts should carefully consider whether they reclassify their students as disabled but currently classified as able to take the state mandated ACT tests and graduate as suddenly too severely disabled to take state mandated tests and unable to complete graduation requirements. …. … A recent e-mail, which also alludes to a conference call from Superintendent White, implies that schools and districts should re-examine all of their students and consider changing their placements from LAA2 tested, to LAA1, untested. [Emphasis added.]
Included in altering the test-taker group is the strategy of suspending low scorers on the day of a high-stakes test:
“Introduction of high-stakes testing to improve school accountability has apparently led [some Florida] schools to disproportionately punish low-performing students during the testing period to try to ‘game the system,’” [economist David] Figlio said.
With lengthy suspensions, students are more apt to miss the examination and its make-up dates, thus raising the school’s overall average, he said. [Emphasis added.]
As to student selection, some schools declared as “public” for receiving public school funds are not strictly public in that they are able to exercise an admission requirement. Such is true of charter schools. Charters are generally better able to “skim” better students from a school system, community, or region, thereby enabling the charter to “achieve” higher test scores.
Is “skimming” cheating? I say yes, if the school boasts of its high test scores without also revealing its admission, retention, and discipline practices. As Ravitch notes in Death and Life:
Whenever there is competition for admission, canny principals have learned how to spot the kids who will diminish their scores and how to exclude them without appearing to do so. [Emphasis added.]
Yet another fine way to cheat (manipulate? play?) the test score system is to lower the standard by which “success” is measured. This altering of a standard of success is a frequent occurrence of late in Louisiana education in general. In June 2012, John White/BESE has lowered the graduation rate necessary for high schools to earn “bonus points”; White/BESE has increased the percentage of teachers to be declared “highly effective” from 10% to 20% (thereby lowering the standard without facing the real problem, use of value-added modeling [VAM] to pigeonhole teachers); 2013 school score calculations will focus on students achieving “basic” instead of the previous focus on students scoring “advanced.” (I have the pdf file on this.)
In short, a superintendent and rubber-stamp state board of education (or a mayor, or a governor) can progressively lower the scoring standard and produce the illusion of scoring gains on standardized tests. In order to successfully promote the lie of score improvement, a district must downplay the drop in the scoring standard from one year to the next while focusing on the percentages of students “improving” from one year to the next.
The final means of cheating is also indirect, and it is well-accepted and encouraged by many school officials: Teaching to the test. The high-stakes nature of standardized testing has removed the testing itself from its original purpose: To improve instruction and promote student learning. As Ravitch notes in Death and Life:
Excessive test preparation distorts the very purpose of tests, which is to assess learning and knowledge, not just to produce higher test scores. [Emphasis added.]
In “teaching to the test,” students are becoming increasingly able to master the testing format without possessing the actual knowledge tested. Thus, students who have been coached to score well could actually lack the knowledge that should be evident based on their high test scores. And students are resisting:
… several studies have renewed questions about the impact of high-stakes testing on teaching and learning. Research suggests that high-stakes testing leads to constant test preparation or “teaching to the test” that severely narrows the curriculum and “de-skills” teachers….
In theory, the alignment of state curriculum with the tests would ensure that teaching to the test is teaching the curriculum. However, teaching to the test tends to inflate scores at the cost of in-depth classroom instruction. It is exactly the memorization exercises, test preparation and worksheets to which students have objected most in their protests against high-stakes testing. …
As the T-shirts of the protesting [Texas] students attest, high-stakes testing is de-personalizing education for many students, leaving them feeling objectified. [Emphasis added.]
Students are feeling used by the process. They are correct. In corporate reform, students become a means to an end, nothing more than a test score that will move the school average up or down.
The test is worshiped; the student, cheated.
Test or student?
All adult stakeholders are choosing a side based upon their behavior.