Skip to content

Cheating As We Worship: The Almighty Standardized Test

March 2, 2013

Contemporary America worships the standardized test.

Certainly corporate reform worships standardized tests because of the money to be made doing so. (Corporate reform ultimately worships financial profit.) Also, the American media so often swallows whole any test scores and publishes such as indisputable truth. And the American public–well– let’s just say that many apply no critical thinking skills when reading/hearing about how “well” or “badly” an institution scored on Some Standardized Measure Guaranteed to Broadcast Truth.

I had the idea to write this post while on my Tuesday morning duty in our high school gym.  It’s a quiet duty spot; my goal is to make sure students do no loiter in the gym on their way from the parking lot into the main building.  I am able to read while sitting there, and I have been reading Diane Ravitch’s Death and Life of the Great American School System. I was reading the section in which Ravitch discusses the numerous ways that teachers and administrators can “game the system” to increase test scores despite no evidence of actual student learning.

I also thought about two instances in my own life when a standardized test or the lack of one has directly influenced my career as a teacher.  More on that to come.

One of my students asked me yesterday about how I was doing with my blog.  I told him and the rest of the class about my idea for this post. Another of my students asked, “You mean how students cheat?”  I said, “No, how teachers and administrators cheat.”  She was surprised at my response.

Indeed, the contemporary problem of cheating on standardized tests has little to do with students trying to manipulate outcomes. That kind of “old fashioned cheating” predates the high-stakes testing movement. No, no, no.  We have advanced in our cheating. Now, those on the testing war front lines are the teachers and administrators, not the students. Corporate reform has all but absolved parents and students of the responsibility for their test scores. If a score is low, it must be the teacher.  It must be the administration.

In the end, I realize that the students and parents do lose when faculty are fired and schools are closed. But those currently facing at the standardized guillotine are the school administrators and teachers.

America’s relationship with the standardized test score baffles me. Perhaps my response is due to my training in measurement and statistics. I have seen the underside of the magic trick, so to speak.  The numbers that are revered by so many because of their holographic infallibility are no mystery to me.

The great irony regarding my combined experience as a Ph.D. in statistics and a public school teacher is that I am not allowed to teach statistics in Louisiana public schools. The reason, as I heard straight from LDOE, is that Louisiana has no certification test for me to take in statistics, and since there is no test, then there is no certification.  Never mind my Ph.D. Never mind my dissertation, or my blind-reviewed publications. Never mind my full-time, five-year university teaching history. Never mind my years as a reviewer of stats textbooks or my associate editor of research position with a national-level, blind review journal.

Never mind that I am able to teach the teachers of high school stats courses.

Never mind all of the rich experience that I could bring to a high school statistics classroom.

As for my teaching statistics in the Louisiana public schools, nothing matters in the absence of a standardized certification test.

I have a second story regarding certification tests.

I earned a minor in German in my undergraduate studies at LSU.  I moved to Georgia in 1993 and accepted a position teaching middle and high school German for a year under the condition that in order to continue, I needed to pass Georgia’s certification test in German.

I could not pass the exam.

What happened in my classroom contradicted my inability to pass the exam.  I was hired to replace a teacher whose personal problems resulted in her German I students only learning half of the German I curriculum. I was to teach these students German II. On my first day, I was surprised to encounter a classroomful of frightened students who pleaded with me not to set them up to fail.  They told me that they were not prepared; that their teacher did not finish teaching the German I material.  I assured these students that we would agree upon where their German I education ended and that I would begin there.  I also told them that by the end of the year, we would cover all of the German II curriculum, as well.

That is exactly what we did.

In my German classroom, I had a double-sided sign that said, “English” on one side and “Deutsch” on the other.  I spoke only the language designated by the visible side of the sign.  I did this in both my German I and II classes. Most of the time, the sign said “Deutsch.” The kids moaned, and they struggled, but in the end, they learned German.

How do I know that my teaching was successful despite my failing the certification test?  First, I had a group of German II students who chose to continue to the optional German III class.  I later learned that my students who continued on to German III (with another teacher) were successful.  One wrote to me to thank me for the preparation; she told me that the transition to German III was smooth.

Second, I had a German I student who won first place in a tri-state German poetry declamation competition.  I asked my student, Lindsay, “Have you had any other German teachers?” She answered, “No. Only you.”  I felt vindicated for my failed exam. I responded, “Good.”

But what of that German certification exam?

In my efforts to improve and pass the exam, I contacted the writer of the exam. He was a local German professor. He admitted that not as much time and money was allocated to the German exam as to other, more popular exams.  It were as though the German exam were a sort of “necessary afterthought.”

I took the exam three times. Never was I offered any specific information on where I might focus my efforts for improvement. I was simply given an overall score and several subscale scores. And I never knew who determined the cutoff score or the rationale used for such a determination.

This ETS document notes the importance of being able to justify a cutoff score:

… There is no purely objective way to set standards. All methods of setting standards depend upon some type of subjective judgment. … You should be prepared to explain the reasons for the use of a passing score…. You should try to anticipate any harm that might be caused by the use of  a passing score.

I lost my job.

I battled the stigma.

Given that I could successfully teach German courses and yet not pass the certification exam, I wondered whether anyone had bothered to connect the Georgia German certification exam with the abilities one must possess in order to teach high-school-level German. Did the man who designed the exam ever teach high school German? Did his university students have some “edge” based upon the fact that he wrote the exam that they would have to pass for certification?

I want to be clear here: I am not against certification exams, nor am I against entry-level standards for professionals.  I think that entry-level professionals should demonstrate their suitability for their profession.  However, I am against exams being hailed as the infallible, “end-all-be-all” evidence.  I am against the subjugation of expert judgment to a numeric exam score.

I believe that high-stakes tests should be rarely used at all and never in isolation from other, unstandardized, subjective assessments, such as expert observation and judgment.

However, the reality is that we live in an age of mega-high-stakes assessment ever looming over the heads of public school administrators and teachers.

A psychometric vulture circling and waiting for the wounded to die.

Let the cheating begin.

One way that teachers and administrators cheat is the erasure method.  That is, teachers or administrators take the students’ completed standardized tests, erase the incorrect answers, and bubble in the correct answers. Teachers could even hold weekend pizza parties while they alter student answers.  Scores are sure to skyrocket.

The teacher might also save time and feed the correct answers directly to the students as they take the test. Or teachers could point out incorrect answers.  Or teachers could produce study guides that include verbatim questions from the actual exam. These are messier alternatives since they leave a lot of witnesses.

A major problem with the above approaches is the ensuing investigation, particularly once the scores drop when those allegedly encouraging the cheating are promoted.

Consider this USA Today excerpt:

[Seipelt Elementary School in Ohio’s] gains and losses are typical of a pattern uncovered by a USA TODAY investigation of the standardized tests of millions of students in six states and the District of Columbia. The newspaper identified 1,610 examples of anomalies in which public school classes — a school’s entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests.

Such anomalies surfaced in Washington, D.C., and each of the states — Arizona, California, Colorado, Florida, Michigan and Ohio — where USA TODAY analyzed test scores. For each state, the newspaper obtained three to seven years’ worth of scores. There were another 317 examples of equally large, year-to-year declines in an entire grade’s scores.

The higher the standard deviation, the rarer that improvement is. In dozens of cases, USA TODAY found 5, 6 and even 7 standard deviations, making those gains even more exceptional. (Note: A standard deviation of 3 is considered “high.”)

Large year-to-year jumps in test scores by an entire grade should raise red flags, especially if scores drop in later grades, says Brian Jacob, director of the Center on Local, State and Urban Policy at the University of Michigan. Such fluctuations by themselves do not prove there was cheating, but Jacob says they offer “a reasonable way to identify suspicious things” that should be investigated.  [Emphasis and commentary added.]

Another general means of cheating is the less-direct method of altering the group of test takers. Teachers or administrators might, for example, alter the classification of certain students, such as special education students, prior to a high stakes test in order to maximize scores. Consider this recent Louisiana story:

Dr. Gary Jones, former Superintendent of Rapides Parish schools and current Assistant Superintendent with the Louisiana Department of Education explains in [a] letter … that districts should carefully consider whether they reclassify their students as disabled but currently classified as able to take the state mandated ACT tests and graduate as suddenly too severely disabled to take state mandated tests and unable to complete graduation requirements. …. … A recent e-mail, which also alludes to a conference call from Superintendent White, implies that schools and districts should re-examine all of their students and consider changing their placements from LAA2 tested, to LAA1, untested. [Emphasis added.]

Included in altering the test-taker group is the strategy of suspending low scorers on the day of a high-stakes test:

“Introduction of high-stakes testing to improve school accountability has apparently led [some Florida] schools to disproportionately punish low-performing students during the testing period to try to ‘game the system,’” [economist David] Figlio said.

With lengthy suspensions, students are more apt to miss the examination and its make-up dates, thus raising the school’s overall average, he said.  [Emphasis added.]

As to student selection, some schools declared as “public” for receiving public school funds are not strictly public in that they are able to exercise an admission requirement.  Such is true of charter schools.  Charters are generally better able to “skim” better students from a school system, community, or region, thereby enabling the charter to “achieve” higher test scores.

Is “skimming” cheating?  I say yes, if the school boasts of its high test scores without also revealing its admission, retention, and discipline practices.  As Ravitch notes in Death and Life:

Whenever there is competition for admission, canny principals have learned how to spot the kids who will diminish their scores and how to exclude them without appearing to do so[Emphasis added.]

Yet another fine way to cheat (manipulate? play?) the test score system is to lower the standard by which “success” is measured.  This altering of a standard of success is a frequent occurrence of late in Louisiana education in general. In June 2012, John White/BESE has lowered the graduation rate necessary for high schools to earn “bonus points”; White/BESE has increased the percentage of teachers to be declared “highly effective” from 10% to 20% (thereby lowering the standard without facing the real problem, use of value-added modeling [VAM] to pigeonhole teachers); 2013 school score calculations will focus on students achieving “basic” instead of the previous focus on students scoring “advanced.” (I have the pdf file on this.)

In short, a superintendent and rubber-stamp state board of education (or a mayor, or a governor) can progressively lower the scoring standard and produce the illusion of scoring gains on standardized tests.  In order to successfully promote the lie of score improvement, a district must downplay the drop in the scoring standard from one year to the next while focusing on the percentages of students “improving” from one year to the next.

The final means of cheating is also indirect, and it is well-accepted and encouraged by many school officials:  Teaching to the test.  The high-stakes nature of standardized testing has removed the testing itself from its original purpose:  To improve instruction and promote student learning.  As Ravitch notes in Death and Life:

Excessive test preparation distorts the very purpose of tests, which is to assess learning and knowledge, not just to produce higher test scores[Emphasis added.]

In “teaching to the test,” students are becoming increasingly able to master the testing format without possessing the actual knowledge tested.  Thus, students who have been coached to score well could actually lack the knowledge that should be evident based on their high test scores. And students are resisting:

… several studies have renewed questions about the impact of high-stakes testing on teaching and learning. Research suggests that high-stakes testing leads to constant test preparation or “teaching to the test” that severely narrows the curriculum and “de-skills” teachers….

In theory, the alignment of state curriculum with the tests would ensure that teaching to the test is teaching the curriculum. However, teaching to the test tends to inflate scores at the cost of in-depth classroom instruction. It is exactly the memorization exercises, test preparation and worksheets to which students have objected most in their protests against high-stakes testing.  …

As the T-shirts of the protesting [Texas] students attest, high-stakes testing is de-personalizing education for many students, leaving them feeling objectified. [Emphasis added.]

Students are feeling used by the process. They are correct.  In corporate reform, students become a means to an end, nothing more than a test score that will move the school average up or down.

The test is worshiped; the student, cheated.

Test or student?

All adult stakeholders are choosing a side based upon their behavior.

7 Comments
  1. Mercedes–

    Holy Cow! I was working on collecting the various bits and pieces that will make up the March 3 entry for BROOB when your story hit my inbox via email. Dropped everything and headed over here to post…

    Your story is maddening. How could anyone in charge of teacher certification policy in any state read this heartbreaking tale and NOT get that their bureaucratic mindset is DESTROYING, not strengthening, teacher quality in their state? I saw the same thing happen to far too many colleagues who taught and coached in Kansas when that state enacted something similar to the idiocy that you got caught up in. I said at the time that Kansas had decided to self-inflict a master teacher genocide, the effects of which can still be seen there even now. Ridiculous…

    Your story reminds me of an argument I often make to friends who, like you, can’t quite let go of the idea that those certification tests have some merit in our K-12 system. Brothers and sisters, how can you support a system in which SHAKESPEARE HIMSELF couldn’t get a high school gig teaching his own works? How can you support a system in which Harlan Ellison, one of the most celebrated (if not THE most) short story specialists ever to practice the craft couldn’t join a high school faculty to teach creative writing or literature courses? Walt Whitman would be laughed out of a principal’s office if he came in looking for a gig teaching poetry (and not just because of the beard). We could play this game until the heat death of the universe arrives, couldn’t we? How many of our greatest writers, artists, musicians, scientists, etc. would we turn away from a job teaching our children because of some stupid test? If your answer is > 0 (Spoiler Alert: It will be.), then I win (shocker, I know). And I won on the merits. I’ll have that debate gladly in any forum, in front of any audience, in opposition to whatever pro-certification test advocate they can scrounge up who is foolish enough to want to get clowned in public (if you think I’m good in written form, you really should try to catch me live sometime; get a good seat near the front).

    Has there ever been a society in human history which has done more to belittle, and cripple, and demoralize the noblest profession of them all than ours is doing at the moment? My reverence for Socrates, arguably the best of us all, knows no bounds, and yes, I am familiar with his argument about why he chose the hemlock. But I will choose a different path. Nothing less than all-out war against this testing-based certification madness will do, sorry. If this is not one one of those things Diane Ravitch rightly attacks as “zombie ideas,” I don’t know what is.

    And don’t even get me started on the National Council on Teacher Quality. Whoops! Too late. I plan on raining a firestorm of rhetoric and ridicule on them that will make the firebombing of Dresden look like a pillow fight.

    A national “bar exam” for teachers? Every time I catch Randi Weingarten broaching that notion in public or in print, a bottle rocket bearing the greeting “From Atlantic, With Love” written in my blood will be headed her way. Stay sharp, Randi, they’re small and hard to see until they’re right up on you…

    I’m going to stop now, before Death by Conniption ensues, but as they say in every movie and TV show EVER: “This isn’t over.”

    Shu

    • Shu, I appreciate your upset on my behalf, and it is true that certification tests and procedures sometimes exclude good teachers. However, I want my readers to also be encouraged by my story; we all experience unfairness and loss, but we should not allow it to smother our future. Instead, by publicizing my struggles, I hope that others in similar situations will know that they are not alone and will press on.

      As a follower of Christ, I stake my life on His words that He will “turn all things to good” if I trust Him. It may not look or feel “good” while I am going through, but “through” is the key word: I will emerge on the beneficiary’s end, and I can take others with me by relaying my experience.

  2. Linda permalink

    Another proposed trick here in CT (a well known “reformer”) is to put all students, whether they qualify or not, in the free and reduced lunch category thereby boosting the scores for that subgroup.

    • Good point, Linda. In Louisiana, John White proposed giving all students free meals. Of course this could be manipulated to promote the view that socioeconomics is not an influence on test scores– there would be no way to sort the students, and all could technically be called “students on subsidized meal plans.”

    • Thanks for the additional comment. My wife, who is still teaching full time (I’m retired), when I mentioned your comment to her said, “Oh, yeah, it’s being done here.” Great. Perfect. Just what we need. So News Corp et al will know that a lot of kids don’t do a lot of homework, while judging them on standardized testing which explains nothing in this world of testing and test-prep “options.”

  3. Bonnie permalink

    Another thing that happens–administrators and their testing chairs have unfettered access to test materials after hours; they are the foxes guarding the hen house. No matter how well-designed the security procedures when the test is administered, the next level up has the greatest opportunity and incentive to cheat.

Trackbacks & Pingbacks

  1. Worshiping the BS Test | A View From The Edge

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: