Skip to content

The AP and Gallup Polls: A Commentary

August 21, 2013

The theme of this past several days appears to be the national education poll and how much one should (or should not) trust the results. On August 17, the Associated Press (AP) released a twisted version of what they would like for parents to want for their children– high-stakes tests– except that the term “high stakes” does not appear once on AP’s 45-page survey results. I challenged AP on August 18 in this post for both its selective reporting and shaping of survey outcomes.

On August 19, Valerie Strauss of the Washington Post blog, the Answer Sheet, contacted AP for a response to my post. Here is what she received, as reported on her blog on August 21:

I asked the Associated Press to respond to her (Mercedes Schneider’s) analysis, and spokesman Paul Colford said that the AP stands by its poll and accompanying story. He said the AP “had editorial control over the design of the survey and ultimately the interpretation of the results.” [Emphasis added.]

Let’s promote a colossal lie to satisfy the AP poll “major funding” Joyce Foundation and stand by the lie in the face of indisputable truth.

In refusing an explanation, AP spokesman Paul Colford advances the opinion that the AP farce speaks for itself.

So it does.

And what else “speaks” is the timing of the AP press release for its debut the weekend before release of the annual PDK/Gallup Poll of the Public’s Attitudes Toward the Public Schools. You see, the Gallup result was not presented with the reformer-purchased slant as was the AP release. So, AP had to get there first, so to speak, in order to mark its territory by telling the American public what it wants. Never mind the truth.

The contradiction between what is observed and what the AP survey “found” is common sense to political consultant Jason Stanford, who said as much on his “Back to School” spot on Fox 7 in Austin, Texas, last night. His words echo this comment by education historian Diane Ravitch:

It is not too strong to say that the AP story is propaganda for the corporate reform agenda. Where are the rallies of parents saying, “Please keep testing our kids! Test them more! Fire their  teachers!” I haven’t seen any, have you?  [Emphasis added.]

AP needed the edge on Gallup in order to promote a lie that its reformers might later cite as “research evidence” for their established desire to kill public education. The four days between the AP article release and the publicizing of the Gallup results allowed for AP to have the stage in popularizing its false results.

Yes, Paul Colford, I am once again challenging AP’s lack of integrity and obvious ulterior motives. Feel free to respond.

But what of that Gallup poll?  I now turn my attention to Gallup, with comparison commentary on the AP survey interspersed.

Gallup annually polls a national sample of adults (those 18 years and older) on its opinions regarding educational issues. The Gallup sample differs from the AP sample; the AP sample is comprised of adults who are also parents of school-age children. Thus, Gallup respondents need not be parents. This said, Gallup does offer results of its parent subsample for a number of questions.

Since the AP sample was comprised of parents, AP could tailor all of its questions to issues faced by parents of school-age children. As I noted in my previous AP post, the parents in the AP survey believe their children are receiving a better education than they did, from teachers who care yet who are seriously underpaid. Therefore, it is not that the AP survey did not have positive findings; it is that the AP report capitalized on a couple of items in which the questions were tailored to produce a result that suited AP’s reform funding source.

The Gallup result also reflects the public faith in teachers. To the question, “Do you have trust and confidence in the men and women who are teaching children in the public schools?”, 72% responded “yes.” When a similar question was asked regarding principals, 65% said “yes.”

As to public/parent awareness of the Common Core, both the Gallup and AP results indicate that adult awareness of Common Core is low; Gallup has two-thirds “not ever hearing of the Common Core’; AP has over half “hearing only a little or nothing at all” of Common Core.

Now comes an important distinction between the Gallup and AP surveys regarding Common Core: Gallup restricts its follow-up questions regarding Common Core to the subset of individuals who say they have heard of it, whereas AP asks its subsequent Common Core question of all respondents, including the majority who just noted little to no knowledge of the standards. Thus, the AP follow-up question on Common Core betrays the predetermined agenda of those funding the survey.

Neither the Gallup nor the AP survey asks respondents whether they have read the Common Core State Standards. This means that even the minority who perceive themselves as “knowledgeable” about Common Core could well have formed this perception from merely hearing the term used, even erroneously.

As to the frequency and usage of standardized tests, the AP survey leads its respondents with a vague definition of standardized tests, one that might easily be confused in a parent’s mind as a “fair test for students”:

Standardized tests are used to evaluate students using a consistent test so that each student is scored in the same way. [Emphasis added.]

Notice no mention of grading the teacher, just of “being fair” to students.

This portrayal of standardized tests as “fair tests for students” completely alters the interpretation of AP survey outcomes. For example, asking parents if they think students “take too many standardized tests” could be interpreted as, “Do you think students in your child’s school take too many fair tests?”

Right there is where AP gets its 61% for the number of standardized tests at “about right.” The same parents who believe that their child’s teachers are “good” also believe that those “good” teachers are administering “fair” tests at a pace that it “about right.”

Another example: AP results include 60% of parents believing that “standardized tests should be used to evaluate teacher quality.” Given that the definition offered read like a “fair test,” parents could have been responding to, “fair tests for students should be used to evaluate teacher quality.” To call this support for “high stakes testing” is blatant deception.

In contrast to the AP questions on standardized test frequency and usage, the Gallup questions are clear and to the point:

Over the last decade, there has been a significant increase in testing in the public schools to measure academic achievement. Just your impression or what you may have heard or read, has increased testing helped, hurt, or made no difference in the performance of the local public schools?

Some states require that teacher evaluations include how well a teacher’s students perform on standardized tests. Do you favor or oppose this requirement?

These Gallup questions do not include faulty definitions or language designed to cloud the issue or promote a given response. As a result, the public responses are more reliable. Gallup has found that the public perception that increased standardized testing “hurts” the performance of local public schools has risen from 2007 (28%) to 2013 (36%). The percentage of individuals who “oppose” the use of student test scores in teacher evaluations has also risen from 2007 (47%) to 2013 (58%).

Unlike the AP survey, the Gallup survey asks public perceptions of “new styles of schooling.” However, here, a number of Gallup questions pose problems due to wording. Below are two examples. I highlight terminology that might bias public responses not only to these exact questions but to follow-up questions, as well:

As you may know, charter schools operate under a charter or contract that frees them from many of the state regulations imposed on public schools and permits them to operate independently. Do you favor or oppose the idea of charter schools?

There are increasing opportunities for high school students to earn college credits online over the Internet. Do you favor or oppose this practice?

I am not surprised that the public favors charters based upon the definition provided (68% “favor” in 2013). After all, charters have been “freed” from the constraints of bureaucracy; the public schools have not.  Therefore, in choosing charters, respondents likely are choosing a school’s “freedom” from rules and regulations. With “freedom” in mind, consider a subsequent charter question and a rewording with “freedom” substitution:

The original

Would you support new public charter schools in your community?

reads like

Would you support new schools freed from the bureaucracy faced by public schools in your community?

If Gallup presented information regarding the turnover of under-regulated charter schools and the ramifications, including the possibility that students could earn a meaningless diploma if charter accreditation is revoked, the public favor for “schools freed from imposed state regulations” would likely decrease.

As for the “opportunities” presented by online learning, the word “opportunities” bespeaks that which is good and desirable. The Gallup poll includes four “opportunities” questions in the “new styles of schooling” section (even the section heading could sway respondents). In all four questions, a clear majority of respondents have selected “favor.” What one cannot know is whether participants truly favor these options or whether they are responding to the positive term, “opportunities.”

A likely reason that the Gallup voucher question does not reflect the response favor of the charter and “opportunity” questions (2013: 70% “oppose”) concerns the clarity of voucher question wording. The voucher is defined not as a “freedom from public school constraints” or an “opportunity” but as the use of public funds for private schools:

Do you favor or oppose allowing students and parents to choose a private school to attend at public expense?

If Gallup reworded its charter and online education questions to parallel its voucher question, the public would see that charters, vouchers, and online education all draw funding away from community public schools and send it to entities that are not publicly operated.

A final example of a misleading Gallup question involves school closures:

To save money, some school districts are closing neighborhood schools because of declining student enrollment, causing students to attend another public school. Do you favor or oppose this process? [Emphasis added.]

School closure is too complex an issue to be captured in a single question. First of all, charter competition has contributed to “declining student enrollment” in some districts. Respondents do not get to know this. Second, “attending another public school” offers nothing to respondents regarding the difficulties of locating a new school and of transportation and safety issues associated with being forced to attend distant schools. Finally, the question presumes that public school closure is not intentionally done in order to promote privatization. The public might be completely unaware of any and all such issues.

Which brings me back to Valerie Strauss:

It isn’t easy for people who are not familiar with polling methodology — which includes me and nearly everybody else on the planet — to understand the difference between polls that have some real validity and those that don’t. The bottom line is to be very careful about competing claims from this and that poll. They aren’t all alike. [Emphasis added.]

Reading polling results requires critical thought and a healthy skepticism.

A PhD in statistics and research doesn’t hurt, either.

 

5 Comments
  1. Bridget permalink

    http://us4.campaign-archive2.com/?u=b4ad2ece093459cbf2afb759f&id=687f5e27b4&e=337bb88832

    Dr. Schneider, on a similar note, this is an interesting report out of Colorado by NEPC about reports of NOLA charter school success in Louisiana and how the information presented to the public is misleading.

  2. patriciahale permalink

    The incompetent don’t know they’re incompetent. (Dunning-Kruger). So logically speaking, people have opinions on things they know nothing about.

  3. La Edu Watcher permalink

    The old PR adage – ” He who gets there first defines the truth.” – it’s what these ed deformers and supporters have been so good at the last few years. They get out in front with their messaging, which makes the real data, information, story and its tellers look like they are on the defensive. Unfortunately, most folks will only see the headline of the AP/Gallup poll, not the blog that refutes its claims.

Trackbacks & Pingbacks

  1. Mercedes Schneider Compares AP and Gallup Polls | Diane Ravitch's blog
  2. Schneider to David Brooks: Common Core Does Not Encourage Opinion Writing Like Yours (UPDATE) | Diane Ravitch's blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: