Raw Scores, Scaled Scores, and Louisiana’s PARCC-ish Tests
A number of Louisiana parents are insisting that the Louisiana Department of Education (LDOE) release raw score information on the PARCC-sort-of test that it administered in spring 2015 to students in grades 3 through 8.
I agree that parents have the right to know their children’s raw scores. Yet these raw scores must accompany scaled scores in order to be understood.
The reason for this is that standardized testing utilizes various forms of a test. Sometimes various forms are given in the same test administration. Other times, the forms are altered as items are changed out in order to avoid unfair advantage to test takers in future administrations.
For any standardized test, a raw score is available for each test taker. However, some versions of a test are easier than other versions. Therefore, the raw score could be biased in favor of students taking the easier test. In other words, two students– one who gets more items correct on an easier version of a test and another who takes a more difficult version yet gets fewer items correct– might actually be “equal” in their performance when the outcome is adjusted for differences in test difficulty.
Scaled scores account for the difficulty of tests and adjust the score to account for such differences in test form difficulty. They do so via a method called “equating.”
For more on raw scores, scaled scores, and equating, see this brief, 2011 article by ETS.
Now, let’s turn our attention for a moment to Louisiana and its PARCC-ish test.
All of the other states that purported to give PARCC tests in 2015 contracted with Pearson, the vendor of PARCC tests. Louisiana gave its PARCC-styled tests via an old contract with Data Recognition Corp (DRC).
So, for all other states that called themselves “PARCC states” in spring 2015, Pearson is handling the scoring (including scaling and equating).
DRC appears to be doing so for LDOE. (The PARCC website for Louisiana notes that Louisiana “included items developed through the PARCC process.” For all other states, like Arkansas, the PARCC website notes that the states “administered the PARCC assessment.”)
With the conclusion of performance level setting last month, the recommendations of these 200 or so educators went to the PARCC state education commissioners/superintendents. Together, they approved final threshold scores within the range recommended by the educators. Student scores will range from 650 to 850, with a 700 representing the threshold of Level 2, 725 representing the threshold of Level 3, and 750 representing the threshold of Level 4. The threshold score for Level 5 will vary slightly by test and will be approximately 800. Each state makes its own decisions about possible additional uses of the score results and each state will release results on its own timeline.
According to a September 17, 2015, Advocate article, White is planning to scale the Louisiana PARCC-ish test on the same scale as are the legitimate PARCC states: 650 to 850. (Note that the Advocate article writes about Louisiana as though Louisiana was a legitimate PARCC state in spring 2015. It was not. No contract with Pearson.)
The Advocate article says that “the state’s vendor” is converting raw to scaled scores. That vendor is DRC.
According to the Advocate, Louisiana will also use five scoring levels– just like the Pearson PARCC. But White is going with different labels for those five levels: minimal command, partial command, moderate command, strong command and distinguished command.
The PARCC website will use the following five performance level labels on its score reports:
• Level 1: Did not yet meet expectations
• Level 2: Partially met expectations
• Level 3: Approached expectations
• Level 4: Met expectations
• Level 5: Exceeded expectations
Whether White and PARCC will try to compare Louisiana’s DRC-vended, partial-PARCC test to that of the Pearson-vended PARCC tests is unclear. However, it looks like he is setting Louisiana up for such a comparison.
I assume White will use the PARCC cut scores as noted in the PARCC “Setting Cut Scores” excerpt above.
According to the Advocate, the middle category will be considered “basic” in Louisiana. Thus, the top three categories will be “passing.” Still, know that many (most?) students will not score in those top three PARCC-ish categories.
But back to Louisiana and LDOE’s amazingly clear communications about what is transpiring with its PARCC knockoff:
Note that in this post, the details on Louisiana’s PARCC-ish test I draw from an Advocate article.
Nevertheless, let us keep this November 2015 PARCC-ish test release in perspective. Whether or not White is purposely delaying the PARCC-ish test result release until after both the October 2015 BESE election and gubernatorial election, these elections will happen– the results of which could substantially alter the emphasis of Louisiana public education on tests, tests, tests.