Skip to content

California’s Smarter Balanced Results Capture Poverty

September 10, 2015

On September 09, 2015, California State Superintendent of Public Instruction Tom Torlakson released California’s Smarter Balanced results, which comprise the largest component of the California Assessment of Student Performance and Progress (CASPP).

As San Jose University professor Roxana Mariachi notes, a major issue with California’s Smarter Balanced results involves the apparent inability of the California Department of Education (CDE) to produce evidence that the Smarter Balanced tests are reliable and valid:

At the September 2nd, 2015 State Board of Education meeting, you heard public comment from Dr. Doug McRae, a retired test and measurement expert who has for the past five years communicated directly and specifically to the Board about validity problems with the new assessments.  He has submitted the following written comments for Item #1 [CAASPP Update] at the latest meeting and spoke again about the lack of evidence for validity, reliability, and fairness of the new assessments.

“My name is Doug McRae, a retired testing specialist from Monterey.

“The big question for Smarter Balanced test results is not the delay in release of the scores, or the relationships to old STAR data on the CDE website, but rather the quality of the Smarter Balanced scores now being provided to local districts and schools. These scores should be valid reliable and fair, as required by California statute as well as professional standards for large scale K-12 assessments. When I made a Public Records Request to the CDE last winter for documentation of validity reliability and fairness information for Smarter Balanced tests, either in CDE files or obtainable from the Smarter Balanced consortium, the reply letter in January said CDE had no such information in their files. I provided a copy of this interchange to the State Board at your January meeting. There has been no documentation for the validity, reliability, or fairness for Smarter Balanced tests released by Smarter Balanced, UCLA, or CDE since January, as far as I know.

“Statewide test results should not be released in the absence of documented validity reliability and fairness of scores. Individual student reports should not be shared with parents or students before the technical quality of the scores is documented.” [Emphasis removed.]

The absence of evidence that Smarter Balanced tests are reliable and valid means that there is no evidence that the tests measure what they purport to measure (i.e., “college and career readiness”), nor is there any evidence that the tests measure anything consistently.

I wrote about Smarter Balanced in November 2014. At that time, the Smarter Balanced Assessment Consortium (SBAC) had decided that “college ready” meant “Common Core content ready,” and it had no clue how to operationalize “career ready.”

That was only two months prior to McRae’s request for reliability and validity evidence, which does not seem to exist.

Still, Torlakson and the CDE are pitching Smarter Balanced results as though they are reliable and valid. From the September 09, 2015, CDE press release:

SACRAMENTO—State Superintendent of Public Instruction Tom Torlakson today unveiled the results of new online assessments administered to about 3.2 million students last spring to gauge their progress in learning new, more rigorous academic standards designed to prepare them for college and careers in the 21st century. …

CAASPP includes a number of assessments, but the most widely given are the Smarter Balanced Summative Assessments, which evaluate student progress on the California standards in mathematics and English language arts/literacy, often referred to as the Common Core.

“The results show our starting point as a state, a window into where California students are in meeting tougher academic standards that emphasize critical thinking, problem solving, and analytical writing,” Torlakson said. “California’s new standards and tests are challenging for schools to teach and for students to learn, so I am encouraged that many students are at or near achievement standards. However, just as we expected, many students need to make more progress. Our job is to support students, teachers, and schools as they do.”

Now, it’s not that the Smarter Balanced tests are necessarily capturing nothing. It’s just that neither the Smarter Balanced Assessment Consortium (SBAC) nor the CDE has bothered to establish what exactly the Smarter Balanced tests are capturing, or with what degree of consistency.

I would like to point out one comparison reported in the CDE press release, which states that 3.2 million students took what amounts to the mystery Smarter Balanced tests:

Almost 1.9 million students were classed as “economically disadvantaged” (defined by the CDE as “students eligible for the free and reduced priced meal program, FRPM, foster youth, homeless students, migrant students, and students for whom neither parent is a high school graduate”).

The remaining 1.3 million were classed as “not economically disadvantaged.”

The CDE divided the test results by percentage into four categories, which it termed “exceeding standards,” “meeting standards,” “nearly meeting standards,” and “not meeting standards.”

From “exceeding” to “not meeting, the economically disadvantaged percentages were 8, 23, 28, 41– which means that the economically disadvantaged tended toward lower scores.

Compare the above to the “not economically disadvantaged,” whose corresponding scoring percentages were 29, 35, 21, 15– which means that students who were not economically disadvantaged tended toward higher scores.

Given the results above, the onus rests upon the CDE and SBAC to present to the public empirical evidence demonstrating that the Smarter Balanced tests do more than simply mirror the socioeconomic status of test takers.

In addition, it is the responsibility of CDE and SBAC to demonstrate that the shaky operationalizing of “college and career readiness” as referenced in the November 2014 SBAC information above has some solid empirical backing. If no such evidence is available, then it is reasonable to assume that both CDE and SBAC are knowingly perpetrating a colossal fraud with the supposed “college and career ready” SBAC assessment product.

california poverty

______________________________________________________

Schneider is a southern Louisiana native, career teacher, trained researcher, and author of the ed reform whistle blower, A Chronicle of Echoes: Who’s Who In the Implosion of American Public Education.

She also has a second book, Common Core Dilemma: Who Owns Our Schools?, published on June 12, 2015.

both books

7 Comments
  1. Another vast scheme of testing of uncertain value, sold publicly as if great value for things we have no simolexway of measuring….ahh the follies of data driven reform when in reality we dont have good enough data, and data is not always whats most important.

  2. 2old2tch permalink

    I have a feeling that if this post could be hung on Torlakson’s office door, for the next year, there would still be no answers to the above requests much less satisfactory ones. When highly trained professionals raise reasonable concerns that are totally ignored, what alternatives do reasonable people have? Suggestions?

  3. LA Educator permalink

    Come on, teachers & students, you’re not failing well enough. Let’s change the test!

  4. Jill S permalink

    In CA we have some of the worst schools and some of the best schools in the country. We have 6.2 million students in K-12 school, more bodies than over half of the country has in their entire state. Some of these students are smart, some are not, some are poor and some are rich, some are high achievers and some will barely get by. Like everywhere else. These tests are an expensive, time consuming way to point out the obvious. Luckily CA has very strong protections for those of us who opt out!

  5. Smarter Balanced clearly states they have no “operational definition” of “college and career readiness”. Dated November 14, 2014, their document, “Interpretation and Use of Scores and Achievement Levels” ends with this sentence: “Smarter Balanced does not yet have a parallel operational definition and framework for career readiness.”

Trackbacks & Pingbacks

  1. This Just In: New CA Smarter Balanced Tests Measure Parent’s Affluence… and Nothing Else | Network Schools - Wayne Gersen

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: