Skip to content

Study: New SAT Is Biased Against Females– and College Board Is Not Addressing the Issue

October 24, 2017

The piece featured below is an important one given our nation’s over-sized, under-questioned reliance on standardized testing as The neutral, unbiased means of tagging student potential and worth.

At the forefront of the testing game are the ACT and SAT. School systems across the country are pouring unprecedented amounts of money in paying for their high school students to not only take the tests but also in prep programs with the aim of bolstering student scores.

The secondary school focus on ACT and SAT outcomes is often prompted by the fear of insufficient marks on the psychometrically-invalid school and district grading that plagues our country’s K12 schools.

Now, as one can read below, the “new” SAT yields evidence that the test is biased against females. College Board president David Coleman appears more interested in concealing issues of SAT bias in favor of promoting his SAT product and wrangling a notable slice of the college-testing market from the SAT’s chief rival, the ACT.

Thus, the report of SAT bias comes from outside of the College Board when it should be the Coleman’s College Board that produces such research on its own testing product– preferably before the faulty test is marketed.

The piece below is entitled, “The Disadvantaging of Female Testers: A Reproof of College Board,” and is written by Harvard graduate and Compass Education Group founder, Art Sawyer, who holds the distinction of perfect scores on both ACT and SAT tests.

College Board’s avoidance of subgroup performance differences.

In the report “How the New SAT has Disadvantaged Female Testers,” I look at the impact the new SAT has had on its largest subgroup. That full report is not a quick read. It examines the changes on the new SAT piece by piece and looks at the interplay of those components. It’s meant to be as comprehensive as the current data allow. It seemed to demand a companion piece that was less sober and was willing to enumerate the mistakes and evasions College Board has made in this area. It is not meant as a Cliffs Notes for the in-depth report—which I would be delighted for you to read. It does explain, though, why the full report needed to be written—because College Board seemed to have no interest in doing so.

The research shows that high-performing female test-takers, in particular, were disadvantaged by the new SAT in comparison to the old SAT. This statement is quite intentional in its phrasing. No matter what one thought of the old SAT’s starting point—pro or con—the proportion of female students in top scoring ranges has declined. Moreover, this result could not have been a surprise to those developing the new exam.

College Board should state the policy it took on subgroup score differences in designing the new SAT.

College Board either violated the policy it once held of ensuring that subgroup “gaps that exist on the current test do not widen,” or it chose to abandon that policy. It is quick to use the shibboleths of “opportunity” and “equity” but slow to provide the research to back up those generalities.

College Board should share the data it had both before, during, and after the creation of the new SAT.

College Board has not gone on record with its findings on the shift in gender inequities on the new SAT and has remained similarly mum—beyond misleading press releases—on how observed results have changed among other subgroups.

College Board should explain why it stopped publishing key information.

College Board removed almost all data on gender and income from its annual report on college-bound seniors. It has not published—or at least not made them available on its website—supplemental data tables that are crucial to understanding SAT performance such as percentile tables by gender, race, and ethnicity. It last published those tables in 2015 and has now removed them from its site. Compass has reconstructed the data in order to show the change from old SAT to new.

The further disadvantaging of female students was a foreseeable consequence of the new SAT’s change in structure and scoring.

What was treated by David Coleman, president of College Board, as an applause line in 2014—the return of the prodigal 1600!—would not be viewed as enthusiastically by the thousands of female students now left out of the top score ranges because of the exam’s reconfiguration. How the various components contributed to the impact on female test-takers is summarized in the chart below. The full report provides the data and context.

sat-summary-of-male-female-changes-300x204@2x

College Board (and ACT) should commit to providing more information—at all score levels—of subgroup differences.

Mean score differences understate the impact on female students. Gaps that existed on the old SAT at higher score ranges have widened further. This widening occurred on an exam that was supposed to better align with the “the work students do in school.” Yet it is precisely such schoolwork where females outperform males. Why the additional disparity?

sat-gap-total-scores

The difference in observed results for male and female testers must not be accepted as an inevitable result of standardized testing. It is not.

The SAT continues to lag ACT when comparing standardized mean differences of males and females. In fact, ACT scores for the class of 2017 showed females outperforming males at the mean, at least. There are still many concerns about the ACT as one moves away from the mean, but College Board should do more to address its own test’s disparity—even if only to dispute the claim.

The SAT and PSAT are increasingly government funded, and decisions regarding them are a matter of public policy.

For the class of 2018, approximately 1 million students will have their SATs paid for by state or local governments, and millions more will receive subsidized PSATs. Has College Board been upfront with those entities on the impact the new SAT has had on female students?

The invisible inequity of minimum scores, thresholds, and “cut scores.”

The SAT is not used solely for admission purposes. “Cut scores” at some colleges can determine merit aid and eligibility for honors colleges. The new SAT almost certainly exacerbated—without debate or publicly available research—female underrepresentation in these areas.

The mission creep of the SAT and PSAT has extended the import of score result differences into new terrain.

A popular justification for the PSAT is its use in predicting AP performance. College Board publishes “expectancy tables” by cut score and offers its judgment—on PSAT reports—of a student’s likelihood of success in 21 different AP courses. College Board is cautious in its phrasing of how this information is to be used, but the euphemisms of “Has Potential” and “Not Yet Demonstrating Potential” should not mask the PSAT’s own potential for gender-based steering. College Board has not addressed how the PSAT 10 and PSAT 8/9 may suffer from some of the same subgroup differences shown on the new SAT.

The research on score gaps should not inform individual student decisions.

Female students should no more ignore the SAT as an option as they should ignore STEM careers because of underrepresentation in those fields. Differences in population-level scores are not a verdict on individual performance. A student should try to identify the admission test that best displays her strengths.

Compass’ own research on this topic should be unnecessary, because it is properly the College Board’s responsibility.

College Board was afforded an immense amount of trust by having an entirely new test immediately accepted by every college in the country. It should repay that trust with a proper accounting of how the test performs. College Board has devalued internal research on the SAT and hobbled external research.

A concern is that the increasingly winner-take-all stakes in state-funded testing has made College Board more circumspect about attacking hard problems—and remediating subgroup differences is among the hardest—lest it provide an opening for the ACT. There seems to be an increasing trend toward a tightly controlled narrative and less expansive research.

My hope is that this piece and the full report are widely shared and, perhaps, provoke a response. Even an evisceration of my logic would mean that the College Board is paying attention to the issue and to its responsibility in addressing subgroup performance.

unbalanced seesaw 2

__________________________________________________________________________________________

Want to read about the history of charter schools and vouchers?

School Choice: The End of Public Education? 

school choice cover  (Click image to enlarge)

Schneider is a southern Louisiana native, career teacher, trained researcher, and author of two other books: A Chronicle of Echoes: Who’s Who In the Implosion of American Public Education and Common Core Dilemma: Who Owns Our Schools?. You should buy these books. They’re great. No, really.

both books

Don’t care to buy from Amazon? Purchase my books from Powell’s City of Books instead.

 

9 Comments
  1. Duane E Swacker permalink

    Comparing two onto-epistemologically invalid tests (as proven by Noel Wilson in his 1997 dissertation “Educational Standards and the Problem of Error”)-the old SAT with the new SAT-can result only in invalid comparisons. Mental masturbation all of it. Eliminate the SAT and ACT for the error and falsehood filled malpractices that give false results and we don’t have to worry about the supposed effect of those invalidities on anyone.

    • While earning my social sciences (master’s) degree, too often my experience became an argument between me and whoever was teaching that presuming a locus or “center” for human views was not only narrowly culturalist, but in our nation’s case deprecatingly patriarchal.

  2. By seeking to reduce the gap between ‘subgroups’, College Board reveals the major flaw in its entire operation, and the Testing Industry, in general.

    They start with an assumption (males and females think alike) and then adjust the tests so that assumption is reflected in their test results. Funny, though, they only pick certain groups. (for example, why not, ‘rich kids and poor kids are equally intelligent? But that’s another issue, and not as important as the main fallacy).

    The fallacy is that we ‘know’ who is ‘bright’ and who isn’t. The tests are created and modified to make sure that it gives the ‘right’ result. Thus, the tail wags the dog. The fact that ‘adjustments’ need to happen proves that the tests are B******t.

    Famously, these tests can predict college success no better that High School grades (before they were forced to include test scores such as these). But, what does ‘college success’ actually mean to the happiness of an individual 50 years later, or, for that matter, the happiness of a society?

    An ‘educator’ has the responsibility to ‘draw out’ a student, to make them think in different ways, to help them stand in another’s shoes. A portion of this might involve teaching a technique, so students can see more deeply and in a different way. However, the goal is the seeing, not the technique.

    Tests (particularly those that label students ‘good’ or ‘bad’) are dangerous. Even simple math tests.
    As diagnostics, I suppose they are a tool, one that judges both the student and the teacher. However when we ‘label’ a student, the test and the future outcome become so entwined that it is impossible to disentangle the result. Teachers should be free to teach, and not pollute the psyche of students by ‘evaluating’ them.

    I know, that sounds unrealistic. Our ‘system’ forces teachers to ‘evaluate’ and ‘label’. But, does that ‘system’ produce the greatest good for the greatest number?

  3. I see a contradiction in the summary chart

    In the first line, it is stated that the decreased proportion of females is due to the increased weight of math and decreased weight of writing.

    But in the same chart we see that women are doing better in math (which now has increased weight) and are doing worse in writing (which now has decreased weight).

    While I understand that there is a problem with the decreased proportion of women among the top scorers in the new SAT, I could not detect any comprehensible explanation for this in the long and convoluted article by Sawyer. Can you help?

    • The effects are complex, as Sawyer notes in the summary I posted and elaborates on in the full report. Scores for females are higher on the individual subsections of the new math test, but these actually translate into a lower overall math score on the new SAT. Sawyer notes that the math subsections on the new SAT lack scoring coherence.

      Such a contradiction in scoring (from subscores to overall score) begs for additional research– which College Board has not delivered.

Trackbacks & Pingbacks

  1. New Study: New SAT is Biased Against Females | Diane Ravitch's blog
  2. College Board Botches the Scoring of the June 2018 SAT; Affected Test Takers Petition for Rescore | deutsch29

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s