Guest Post: Deb Herbage on American Institutes for Research (AIR)

  Deb Herbage

Deb Herbage (@DebHerbage) was born and raised in Massachusetts and worked for a Fortune 100 company for 20 years as a business analyst.  She relocated to Trinity, Florida in 2007 with her husband and daughter. Deb is an outspoken advocate and active member of the Opt Out Florida Network and over 50 Opt Out groups across the country.

The following is a guest post by Herbage, who has been investigating American Institutes for Research (AIR).

_______________________________________________________________

Who is Really Benefiting from the State Assessment Contracts?

by Deb Herbage

With its inception in 1946, the American Institutes for Research is world famous for their social science and behavioral and psychological research.  Partnering with the US DOD (Department of Defense), the US DOH (Department of Health), the US COC (Chamber of Commerce), and the US DOE (Department of Education), AIR is a non-profit company based in Washington, DC, and is the recipient of many contracts from the various states for their Common Core “state assessments,” whether it be signing a contract directly with a state Department of Education (like the case in Florida, North Dakota, Delaware, Washington, Oregon, Maine, Michigan and Arizona)or through the Smarter Balanced Assessment Consortium (SBAC) (established in December 2010 for $175,649,539with the funds provided by the Federally funded Race To The Top, RTT).

On September 6, 2012 – AIR announced its partnership with the Smarter Balanced Assessment Consortium to create an online test delivery system for the states.

Here is an excerpt from the SBAC Memorandum of Understanding touting the “services” it is providing to their membership states (after they pay the fees of course) (click on image to enlarge):

SBAC also announced that it chose AIR to partner with the consortium to deliver the initial tests to millions of students in 2013 and 2014.  AIR stated it would make available an “open-source testing platform to enable all states and others to deliver cutting edge tests.”  On December 2, 2014, AIR announced they were teaming up with Microsoft and AssistX to bring students across the United States the most secure online assessment.  AIR has many affiliates under its influence, such as Learning Point Associates,  Association for Education Finance and Policy, College Measures and International Development, Evaluation and Research Program, and many partnerships, such as with Data Recognition Corp. (DRC), the Bill and Melinda Gates Foundation, Council of Chief State School Officers (CCSS) (one of two groups that brought us the Common Core State Standards), Wested, and Google.

When Florida’s Commissioner of Education, Pamela Stewart, signed the contract with AIR (which outsourced the contract to DRC on June 4, 2014), it agreed to $59,933,754 for Year 1, $58,301,988 for Year 2 and $56,954,413 for Year 3 which totals $175,190,155 for the initial 3 year contract. (Click images to enlarge.)

Should FL choose to renew their contract – the grand total will be $322,799,459.  For that kind of money one would expect a flawless test, and one would expect the FL DOE to hold AIR to the terms of the contract. One would also expect AIR would fulfill its promises and obligations in the contract, such as “must provide empirical evidence of psychometric validity.” AIR stated in numerous places in the contract it signed with the FL DOE that the test was psychometrically valid and reliable.  Here are some excerpts from the executed contract (number 14-652):

AIR’s Assessment Program offers psychometric and statistical services that stand alone in terms of quality and innovation. The integration of psychometrics with statistics and sampling sets AIR apart from the competition.

…State testing programs can undercut the best psychometrics, leading to volatile test results from year to year and inaccurate classification of examinees. AIR combines expertise in sampling and psychometrics; all of our samples are optimized, and our statistics accurately reflect the complexities of the sample designs. (page 165)

In testimony to the Florida Senate on March 4, 2015, Commissioner Stewart stated that the test “was absolutely reviewed and is psychometrically valid and reliable.” This is not a true statement because if it were, then Florida could have saved the taxpayers the $594,000 we paid for the validity study to be conducted.  In fact, Utah’s Department of Education sent a letter dated March 23, 2015, to Senator Simmons, stating they did not have psychometric and reliability data for their SAGE test, another AIR test. AIR used the Utah SAGE questions on the Florida Standards Assessment (FSA).  Why would the FL DOE allow questions from a test that was not psychometrically valid or reliable be used on our test?

Why is this important?  As we know, SBAC is one of the two organizations that make up the Common Core “testing consortia” (with PARCC, Partnership for Assessment of Readiness for College and Careers, being the other one). Whether states signed the contract with AIR or with SBAC (essentially the same thing), the SBAC Memorandum of Understanding (MOU) clearly states that each “member state” is receiving psychometric services (see two excerpts from the SBAC MOU below).

The MOU clearly states SBAC will conduct validity studies as part of the “Assessment and Item Design” when states purchase the “Assessment Package.”  (Note that the two MOU’s from PARCC and SBAC include the similar language; just the state name and number of students tested varies.)

The SBAC/AIR charge “member states” fees  for services, such as a previously-alluded-to annual membership fee, as well as administrative and support service fees (which include “psychometric” services) and a “projected testing numbers” fee.  They are charging the member states based on number of students tested.  Every member state MUST comply with the confidentiality and the governing board. This “taking the lead” in forming Common Core testing consortia is obviously influenced by the Council of Chief State School Officers (CCSSO).

For not being recognized as a legal “federal entity,” CCSSO sure has a lot of power of persuasion, as does the other Common Core license holder, NGA (National Governors Association).

All the Common Core testing consortia contracts I have looked at all maintain that their tests are “psychometrically valid and reliable.”

(Here is some summary information on state testing contracts and MOUs I have and have examined for Oregon, North Dakota, California, Ohio, West Virginia, Colorado, and Delaware, which have a combined price tag in excess of $150 million, often for three years of testing.)

Based upon all those costs noted in the state contracts and membership MOUs, AIR, SBAC, PARCC, Measured Progress, Data Recognition Corporation are all drawing massive profits.

It doesn’t really matter if a state signed with SBAC or PARCC.  Considering the faulty rollout of the state assessments across the US, after paying the exorbitant taxpayer money for the state assessments, one would think that these tests would be the ultimate display of perfection or at least live up to the bragging in the contracts.  Our kids certainly are not benefiting from these tests– tests they cannot log into, tests that shut down on them, tests that cannot determine academic performance or inform instruction – all the expectations outlined in the state contracts.

And considering we had the hotly-debated “validity study” here in Florida– and due to the fact that SBAC and AIR joined forces (back in 2012) and AIR notes repeatedly they wrote the test for SBAC and also wrote the special adaptive algorithm for the state assessments and according to the below excerpt from the California SBAC MOU– would it be a reasonable assumption to say that states are being charged for psychometric services that are not being provided?

One thing is for sure: Our kids are NOT benefiting from these tests.

But I know a lot of people who are.