Skip to content

Hart, Weingarten, and Power Point Deception

May 30, 2013

Randi Weingarten really wants to promote the illusion that teachers have bought into the Common Core State Standards (CCSS). She hired Hart Research Associates to conduct a survey of AFT membership regarding perceptions about CCSS, and Hart did so March 27 – 30, 2013. Weingarten has used this survey as a platform to proclaim that “75% of AFT teachers surveyed support the Common Core.” But it is a lie, and I can prove it using a single Power Point slide composed by Hart for AFT and promoted by AFT and comparing that slide to the actual survey question.

I have written about other issues related to the Hart/AFT survey, but the gross manipulation of the survey result I will show in this post trumps all.

Proper Reporting of Survey Results

The National Council on Public Polls (NCPP) was established in 1969 to “set the highest professional standards for public opinion pollsters.” Its membership includes 27 polling organizations, including ABC News, Annenberg Public Policy Center, CBS News, Gallup, Los Angeles Times, and NBC News.

Hart Research Associates is not a member. At least not currently.

NCPP members agree to adhere to certain Principles of Disclosure, which are designated as three levels, the first being what survey reports must include:

Level 1 Disclosure: All reports of survey findings issued for public release by a member organization will include the following information:

  • Sponsorship of the survey
  • Fieldwork provider (if applicable)
  • Dates of interviewing
  • Sampling method employed (for example, random-digit dialed telephone sample, list-based telephone sample, area probability sample, probability mail sample, other probability sample, opt-in internet panel, non-probability convenience sample, use of any oversampling)
  • Population that was sampled (for example, general population; registered voters; likely voters; or any specific population group defined by gender, race, age, occupation or any other characteristic)
  • Size of the sample that serves as the primary basis of the survey report
  • Size and description of the subsample, if the survey report relies primarily on less than the total sample
  • Margin of sampling error (if a probability sample)
  • Survey mode (for example, telephone/interviewer, telephone/automated, mail, internet, fax, e-mail)
  • Complete wording and ordering of questions mentioned in or upon which the release is based
  • Percentage results of all questions reported

Member organizations reporting results will endeavor to have print and broadcast media include the above items in their news stories.

Member organizations conducting privately commissioned surveys should make clear to their clients that the client has the right to maintain the confidentiality of survey findings. However, in the event the results of a privately commissioned poll are made public by the survey organization the above items should be disclosed.

In the event the results of a privately commissioned poll are made public by the client, the survey organization (a) shall make the information outlined above available to the public upon request and (b) shall have the responsibility to release the information above and other pertinent information necessary to put the client’s release into the proper context if such a release has misrepresented the survey’s findings. [Emphasis added.]

The goal of such stipulations is to ensure unbiased reporting of survey results.  I have bolded the sections in which the Hart/AFT Power Point info I will share has violated such disclosure.

Now, Hart is not currently a member of this organization and is therefore not subject to the hearing procedures outlined at the end of the Principles of Disclosure section.  Even though Hart is no longer a member of NCPP, the guidelines set by NCPP are a standard for assessing unbiased reporting of polling results.

Hart’s non-member status in NPCC does not excuse it from honest, straightforward reporting of survey results. In fact, Peter Hart used to sit on the NCPP board of trustees; his name is on this 1988 letter sent to a non-member polling organization with the following admonition:

As you are not a member of NCPP, obviously you are not bound by our release standards. However, had more of the pertinent information on methodology been released it would have enhanced the credibility of your efforts. … Shortcomings in the release of survey results from any source are of concern to us, as we all suffer when professional standards are breached. [Emphasis added.]

Thus, as a trustee member of NCPP, Peter Hart has been associated with admonishing non-member pollsters of the need for clear, open, above-board release of survey details.

Many of the Principles of Disclosure are common sense. If I conduct a poll and do not clearly and accurately disclose polling questions and actual results fully and according to the responses asked of participants, I do not have the public’s best interest in mind, for I am choosing to withhold clear information. Furthermore, if I reword questions and reduce reported categories, and if I withhold numbers on graphics supposed to represent the result, then I am manipulating the public via limited release of information.

The NCPP Principles of Disclosure include two additional levels. Level Two involves what survey organizations should provide to the public upon request. It is noted in Level Two that “complete wording of questions” is a Level One requirement and becomes a level Two requirement only if the survey has been translated into another language.

Hart released the survey instrument not as part of this original Power Point but upon the request of one of my readers: Hart AFT survey

At a later date, the same reader requested complete response counts. Those counts are yet to be received.

Level Three includes criteria that NCPP member organizations are “strongly encouraged” to meet, including public release of raw data sets (minus participant identifying information) and public release to a website of “complete wording, ordering, and percentage results of all publicly released survey questions… for a minimum of two weeks.”

The goal is… dare I write it… transparency.

The Power Point Slide In Question

Actually, a number of slides in the Hart/AFT survey evidence reporting problems. However, Weingarten’s position in support of CCSS is the hinge for the rest of the survey questions presented in this Power Point. In order to argue that teachers want more CCSS assistance, they first must be shown to support CCSS, period.  Otherwise, the issue needing addressing is not more CCSS assistance but the lack of teacher investment in CCSS.

The teachers in this AFT poll are not invested in CCSS.

The slide in question is page 3 of the Power Point above.

AFT slide 3

AFT 75%

The first deceptive issue with this slide is its title: Teachers Overwhelmingly Approve of Common Core State Standards. This leads readers to “see” what they are being told to in the slide– not only do “teachers” (not AFT teachers, but teachers in general) “approve,” but by a landslide they approve.  The clearly-intended message: Teachers in general really are for this Common Core. Readers are then told that teachers were asked this question:

 Based on what you know about the Common Core State Standards and the expectations they set for children, do you approve or disapprove of your state’s decision to adopt them?

Yet this was not the survey question.  The actual survey question was as follows:

Q3 Based on what you know about these standards and the expectations they set for children, do you strongly approve, somewhat approve, somewhat disapprove, or strongly disapprove of your state’s decision to adopt the Common Core State Standards?

Strongly approve ……………………………………………………………………….. 1

Somewhat approve …………………………………………………………………….. 2

Somewhat disapprove ………………………………………………………………… 3

Strongly disapprove ……………………………………………………………………. 4

Not sure ………………………………………………………………………………….. 5

Notice that a primary difference between the Power Point question and the actual survey question involves the gradation of the responses. This is key. In the Power Point, Hart/AFT have collapsed categories in order to present the illusion of “overwhelming approval.”  Thus, “strongly approve” and “somewhat approve” have been reduced to “approve,” and “strongly disapprove and somewhat disapprove” have been reduced to “disapprove.”  “Not sure” is not accounted for using a bar.

“Somewhat approve” is not “overwhelming approval.” It is hesitant approval. It is approval with reservation.  But saying that a notable number of teachers “approved with reservation” doesn’t sell the “teachers ‘overwhelmingly’ approve” point-of-view. So let’s manipulate the categories in reporting to support what we want to promote.

Notice also on the page 3 Power Point that only two bars are shown on a three-dimensional drawing of a bar graph.  Three-dimensional bar graphs are misleading; they can be manipulated because there is no straight, obvious “zero” line for readers to measure the graph against. And taking what should be a five-bar graph (one for each of the possible responses to the original survey question) and reducing it to two bars gives the illusion that only two options matter. Such presentation can easily disguise the subtleties of the muliple responses.

Why ask for people to respond using multiple responses then collapse catgories in reporting? Simple: The results didn’t turn out the way the researchers had hoped and require some doctoring to say “what we need them to.”

AFT took Gates money, and Gates supports (and bankrolls) CCSS.

Is this too far-fetched?

Back to bar-graph deception:

Hart chooses a two-bar graph and for each bar uses two different colors, supposedly to represent the multiple possible response categories.  But again, if each category is not clearly presented as its own two-dimensional bar against a clear zero line, the the reader must work to tease out the subtleties in response numbers/percentages.

Notice that the first bar on page 3 is tall and has “75%” at the top. But it is two-toned.  These are the response categories, unlabeled, with actual counts/percentages per category also absent.

This graph is the only one missing any kind of response category labeling.  Based upon subsequent graphs in this presentation, I see that the bottom color in each bar is equivalent to the “strongly approve” (in this slide, dark blue) or “strongly disapprove” group (dark red), and the color on top is the “somewhat approve” (light blue) and “somewhat disapprove” (light red) group.

Notice that the largest area in this graph is light blue– “somewhat approve.”

Most of the respondents were not “sold out” on CCSS. They had reservations.  They “somewhat approve” of CCSS.

This is not “overwhelming approval” for CCSS.

I am sorry that I am unable to report actual numbers at this time. If Hart sends the requested actual counts to my blog reader who requested them I will post the actual survey questions (all of them) with the actual response counts.

As for what we have before us now, on page 3 of this Hart/AFT Power Point, the title is deceptive, the question is misreported, the graph is confusing, and the percentages reported are misleading.

There is also an “aside” on this slide: “79% are very/fairly familiar with CCSS.” Again with the limited reporting and collapsing of categories.

Notice how much more detail is devoted to the reporting on subsequent slides.  They are still weak for their two-bar graphs, but there is less effort to conceal information.  The sale has been made: “75% of AFT teachers surveyed ‘overwhelmingly approve’ of CCSS.” These are now the “bring it home” slides to promote a moratorium on an issue that AFT teachers aren’t sold on but that Hart and Weingarten have shaped reporting to “prove” that they are.

Other Survey Questions

Once I receive the actual results for the survey, I will know how teachers in this survey responded to these two questions:

Q12a Have you read the Common Core standards for mathematics and English Language Arts, or have you not had an opportunity to read the standards?

Have read the standards …………………………………………………………….. 1

Have not had an opportunity to read the standards ………………………… 2

Not sure ………………………………………………………………………………….. 3

Q12b Have you received a hard copy of the Common Core standards, or have you not received a hard copy?

Have received a hard copy ………………………………………………………….. 1

Have not received a hard copy ……………………………………………………. 2

Not sure ………………………………………………………………………………….. 3

It seems that these would be questions asked at the outset of the survey. It does make sense that one would want to check to see if those who say they are or are not familiar with these standards have seen or read them. Instead, Hart/AFT ask and report on this perception question:

Q2 How familiar are you with the Common Core State Standards–very familiar, fairly familiar, just somewhat familiar, or not familiar?

Very familiar ………………………………………………………………………………. 1

Fairly familiar …………………………………………………………………………….. 2

Just somewhat familiar ……………………………………………………………….. 3

Not familiar ……………………………………………………………………………….. 4

Not sure ………………………………………………………………………………….. 5

And even Q3, the focus of the infamous page 3 of the Power Point,  is based upon perception (Based on what you know about these standards…).

But that is for another post.

A Challenge

Hart is a major research company. Weingarten leads a major teachers union.

To them I write:

You have deceived the public with your information manipulation.

You have done what NCPP tries to guard against. You have contributed tarnish to the image of the polling and survey research fields.

Rebut this article, folks.  I can’t wait to read it.

29 Comments
  1. Ellen Lubic permalink

    Excellent analysis and education for folks who do not understand polling. Thanks Mercedes.

    Ellen

  2. TeacherGirl permalink

    Not only were we not supplied with a hard copy of the ELA CCSS IN MY district but we were told the transitional curriculum was too long for them to print out and we’d have to print our own.

  3. Linda permalink

    Wow!

    Randi, do you have a response? We are waiting.

  4. Scott Holder permalink

    I agree with TeacherGirl. I was not given either, although I did work on the transitional curriculum for our district. I was able to see most of the CCSS material at our workshop. We all seemed to generally agree that it was not necessarily something in the best interest of the students.

  5. John Young permalink

    Reblogged this on Transparent Christina.

  6. I’m concerned about the absence of data for those Not sure… Most people unfamiliar with the details would opt out for “not sure.”

    So if 90 percent of the respondents were not sure and this was then ignored, washed out, one could have remaining 1% strongly support, 5.35% somewhat approve, and 0% somewhat disagree, and 3.65% strongly disagree and still get the exact same results as the graph displayed….

    One could truthfully say when comparing those supporting over those against the rate of those supporting was an overwhelmingly 75%…

    However more truthfully stated in this scenario would be that 99% of all teachers surveyed do not strongly approve of Common Core Standards.

  7. Mercedes, if you find a graphic you want to paste, make it as big on your screen as possible, hit the screen shot button on your keyboard and paste it into Paint (standard on pc) or a photo editing program (most will open a page the size of whatever you last copied to the clipboard), crop out what you don’t want and save as a png or jpg to import into your doc. Or, copy the cropped image again (whether you save or not) and see if you can paste that into your blog.

    Excellent reporting. It’s wonderful to see someone dissecting bad reporting, exposing it and the shysters who perpetuate it. Good for you…good for the public. Thank you for all you do!

    • Thanks, Katherine. Security settings were problematic on my old laptop. It would not allow me to paste the image.

      • Linda permalink

        I second the thank you for all you do comment. I recently watched your two videos explaining the RSD fiasco/hoax and I can’t wait for your book. I hope to meet you someday. Love, Linda

      • Hi, Linda. I will be at the Network for Public Education conference in Austin on March 1 and 2, 2014.

    • Katherine, I have new technology and was able to insert the notorious slide 3.

  8. Fred, I have noted in my Hart/Weingarten posts that neither Hart nor AFT has publicized complete information regarding survey and participants.

    Yesterday one of my friends tweeted Weingarten and asked for public posting of 1) AFT demographic info and 2) complete survey results.

    No response.

    Feel free to tweet her and ask for the same.

Trackbacks & Pingbacks

  1. @rweingarten has some explaining to do re: #AFT #commoncore survey – @ THE CHALK FACE knows SCHOOLS MATTER
  2. Hart, Weingarten, and Power Point Deception | deutsch29 ← NPE News Briefs
  3. Sunday reads. | Fred Klonsky
  4. 20x the Testing with #CCSS: Are Teachers Being Told the TRUTH About Common Core? Is the #AFT Poll More “Junk Science”? Will #CCSS be the Most Devastating Ed Policy in US History? | Teachers' Letters to Bill Gates
  5. This is what we’ve been talking about re: the @aftunion “survey” on #commoncore @rweingarten – SCHOOLS MATTER @ THE CHALK FACE
  6. Panelist Defends Common Core at Center For American Progress Forum: 'The Children Belong to All of Us'
  7. Pastor Mikes Report | Panelist Defends Common Core at Center For American Progress Forum: ‘The Children Belong to All of Us’
  8. Common Core Panalist: “the children belong to all of us” | Agenda 21 Radio
  9. Panelist Defends Common Core at Center For American Progress Forum: 'The Children Belong to All of Us' | Sharing Liberty
  10. Panelist Defends Common Core at Center For American Progress Forum: ‘The Children Belong to All of Us’ | Political Thrill
  11. Bill Gates: Common Core Opponents 'Shrouded in Myths'
  12. Pastor Mikes Report | Bill Gates: Common Core Opponents ‘Shrouded in Myths’
  13. NewsSprocket- Bill Gates Asks Teachers to Explain Common Core to Parents | NewsSprocket
  14. Bill Gates Asks Teachers to Explain Common Core to Parents
  15. Bill Gates Asks Teachers to Explain Common Core to ParentsThe Daily News Source | The Daily News Source
  16. Bill Gates Shows How Stupid He Really is About Education | Agenda 21 Radio
  17. Poll Favoring Common Core Funded by Gates Foundation Grantee | Riverside Daily Digest

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: