Will PARCC and Smarter Balanced Become One? Details from PARCC’s Request to Help Chart Its Future
On February 17, 2016, the Partnership for Assessment of Readiness for College and Careers (PARCC) issued a Request for Information (RFI) to organizations that wished to help advise PARCC on the future of PARCC. As a Common Core testing consortium, PARCC has not garnered a notable number of states that have agreed to administer PARCC tests; as such, in November 2015, PARCC began offering “options” to states, including the vending of PARCC items (rather than the usage of entire PARCC tests) to drum up interest in the PARCC consortium. (Much of this information one can find in the RFI linked above.)
In its RFI, PARCC was looking for advice about 1) stability, 2) sustainability, 3) comparability, and 4) economies of scale.
PARCC intends to use the information gleaned from the RFIs to inform a Request for Proposal (RFP) regarding “qualified vendor(s) for providing [PARCC’s future (as detailed in the RFI)] products and services.”
Note also that the PARCC nonprofit, PARCC, Inc., is currently the organization that manages the PARCC Consortium. However, its future as PARCC manager is not a given; the PARCC, Inc., contract to manage PARCC expires in June 2017. It, too, responded to the PARCC RFI.
The RFI itself includes the following statement about public access to the RFI responses that PARCC receives (as well as the included disclaimer):
On our public forum call on Tuesday, February 16, Secretary Skandera said that responses to the RFI will be held as confidential. While responses to the RFI will not be publicly posted, they will be subject to the District of Columbia’s Freedom of Information Act, or FOIA, DC Code §§ 2-531-539. Please identify any sections in your response containing confidential proprietary information that could result in substantial harm to your competitive position if publicly disclosed, and, if warranted, the appropriate FOIA exemptions will be applied in the event of a FOIA request. This is a non-competitive, non-binding RFI. This RFI is issued for information and planning purposes only and does not constitute a solicitation. Replies to the RFI will not be considered official offers, nor will replies result in an award of services.
Apparently EdWeek’s Catherine Gewertz noticed that FOIA possibility; on August 01, 2016, Gewertz submitted a FOIA request to Washington, DC’s, Office of the State Superintendent of Education (OSSE) for the PARCC RFI responses.
The OSSE response (all 128 pages of it) can be found at the end of Gewertz’s August 10, 2016 EdWeek article. Gewertz hits the highlights in her article and invites readers to peruse the FOIA for themselves.
This post includes highlights from my own reading of the results of PARCC’s RFI, which involves responses from 10 entities:
- Council for Aid to Education (CAE)
- Data Recognition Corp (DRC)
- Illinois Higher Ed
- Educational Testing Service (ETS)
- Measured Progress
- PARCC, Inc.
- Smarter Balanced Assessment Consortium
It’s a long post. You might want to refresh your beverage before settling in for the read.
In its response, ACT alludes to “turning around the PARCC organization” (page 5). I find it humorous that a phrase popularized by test-driven ed reform is being used to describe a Common Core testing consortium.
ACT also acknowledges that the vendor selected to manage PARCC will have to pump loads of cash into the consortium. Thus:
The investment needed to provide for profitability and sustainability of PARCC coupled with the principle of comparability and the desire for a “neutral, third party partner” may prove difficult, especially for for-profit vendors. (page 5)
ACT continues to note that the profit motive and need for item security could create “significant barriers to overcome including likely financial barriers” when one considers item ownership and offering an open item bank to states (page 6).
On page 8, ACT notes that any future RFP needs to include a means to verify that the number of available items is sufficient to meet state demands, and the associated fee schedule needs to be clear.
ACT also suggests “third party yearly audit of the item bank” (page 8).
Council for Aid to Education (CAE)
New York-based Council for Aid to Education (CAE) (a research and assessment nonprofit; see its 2014 tax form here) suggests that PARCC be overseen (and test content owned) by an independent nonprofit (page 12), which it refers to as “the Structure”:
CAE suggests that the most efficient model would be one whereby the Structure has exclusive rights in and to all content, whether originating from PARCC or being newly developed by the Structure or states. … Co-ownership of content will create more problems than it solves. (page 13)
The question of the nonprofit, PARCC, Inc., as being the owner of PARCC items has come up before. PARCC, Inc., has stated that it holds the rights to PARCC items. I asked PARCC, Inc., to produce the document stating that it was the owner of PARCC items. To date, it has produced no such document.
Note that to date, no nonprofit is permanently installed as overseeing the PARCC consortium– which makes nonprofit ownership of PARCC items unrealistic (and currently, unlikely).
Back to CAE:
CAE also suggests that in order to incentivize states to contribute items, states could be offered “credits (use rights) for items they develop” (page 13).
Regarding states’ use of RFP to contract with testing vendors: CAE suggests requiring the state-contracted testing vendor to use PARCC items (as such are owned by the Structure). This arrangement requires a state testing vendor to subcontract with the CAE-proposed, PARCC-item-owning nonprofit.
For a quick read on what assessments CAE is actually involved with and in what states (e.g., Indiana, Mississippi, Texas, New York), see page 25 – 29. For info on some of CAE’s staff, see pages 30 – 35).
Data Recognition Corporation (DRC)
In its RFI letter (page 36), DRC brags about its service regarding “collaborating with PARCC in 2015 in delivering the Louisiana statewide assessment program.” Note that in 2015, PARCC delivery was supposed to happen via PARCC vendor, Pearson, with all other states being required to contract with Pearson for PARCC assessments. In 2015, states were supposed to contract with Pearson for PARCC in order to use PARCC assessments. However, the Louisiana Department of Education made a deal that was kept from the public eye by having DRC arrange to somehow procure PARCC items. The content of the DRC test was never clarified. (Those interested can read here about Louisiana’s shady PARCC history.)
Now, in its 2016 RFI, DRC tries to present its so-called 2015 PARCC delivery as legitimate. Indeed, Louisiana’s using DRC to make an undisclosed arrangement with Pearson and/or PARCC did inspire PARCC consortium members to vend PARCC items in 2016 (as opposed to requiring states to use entire PARCC assessments).
It seems that DRC asked to redact all but the above cover letter; so, the public does not get to know what details DRC has in mind regarding suggestions about the future of PARCC.
I wonder if DRC’s redaction is connected to its undisclosed arrangement regarding Louisiana’s 2015 so-called PARCC assessments. Wouldn’t want any FOIA to let the public know what really transpired.
Illinois Higher Ed Response
The next response comes from Dr. James L. Applegate, executive director, Illinois Board of Higher Education, and Richard Freeland, Distinguished Professor and Professor Emeritus, Northwestern University and former director of the Massachusetts Department of Higher Education.
They want PARCC’s ability to measure college readiness to be a core design feature of PARCC restructuring. They note:
We didn’t see this college-ready design element, nor the participation of higher education, adequately addressed in the RFI. …
We feel it is imperative for PARCC’s governing board to insist that a clear threshold be established and met in order for states to make claims about students’ college readiness. (pages 36-37)
The truth of the matter is that there is no single profile of a successful high school graduate that could be scripted for success at one of the thousands of diverse colleges across the US. There are hundreds, maybe thousands of profiles. Doesn’t a profile or definition of college readiness depend upon…
- what high school the student graduates from;
- what career path the student is interested in pursuing;
- what goals the student has for his or her college path; and
- what college the student attends.
If somehow we think that Common Core Standards and new national assessments based on the standards will help us define college readiness, I think we are being overly optimistic or quite naive. …
Isn’t it true that persistence and motivation are much more important to success in life (maybe college) than whether we have scored high on a standardized test? We all know this is true. So why do we continue to let our better judgments be swayed by organizations like PARCC, the federal government, and other educational organizations that continue to promote this idea of “scoring high on tests” leads to success. Certainly, students who score high on standardized tests have a core set of skills that will help them in college; however, students who don’t score as high on standardized tests, but have perseverance and motivation, also have important skills that will help them be successful.
College readiness cannot be represented by a “clear threshold” on the PARCC test. If it could, PARCC would have already produced such thresholds for each of its tests; validated these with several truly-independent research studies, and used the outcomes to advertise the superiority of PARCC assessments.
Educational Testing Service (ETS)
One of the tidbits from the ETS RFI is that Louisiana superintendent John White is apparently working with “heads of testing companies” to have a single entity overseeing both PARCC and the other Common Core testing consortium, Smarter Balanced:
…It is important to consider whether the Structure exists only to support the goals of the PARCC Governing Board, or whether it may also include Smarter Balanced– and perhaps other groups of states. We know that the PARCC Governing Board is aware that Louisiana Superintendent of Education John White had been speaking with the heads of testing companies about a similar set of goals concerning content from both Smarter Balanced and PARCC. (page 41)
ETS continues with noting that having a Structure that acts as an item/test form “library” should preclude any testing company from being the Structure since the Structure should not also sell competing tests/items.
In its RFI, ETS also suggests that the PARCC Governing Board, “should it continue to exist,” would not control the very Structure it is trying to establish to oversee PARCC, “except to the extent that members of the PARCC Governing Board also serve on the Structure’s Board of Advisors.” (page 42) (ETS does not think that states should have members on the board of directors– that such would be a conflict of interest.) ETS returns to this ownership idea on page 46, where it notes that the consortium’s trying to limit Structure ownership would be “problematic.” ETS suggests that what PARCC is actually seeking is a Structure that would be a “steward” and not an owner– which “would further restrict the number of entities seeking to become the Structure.” (page 47)
And in order for PARCC to be sustained, it must offer its products at a lower cost “than states would be charged were they to purchase such content directly from established vendors.” (page 43)
ETS also notes that the initial advantage of the Race to the Top funding for PARCC will “diminish over time” and that “the cost advantage of federal funding needs to be replaced with the cost advantage of volume purchasing, which will require the Structure to develop and maintain a large customer base of states.” (page 43)
ETS also suggests that PARCC sell its items to testing companies so that the testing companies might, in turn, sell them as part of an off-the-shelf product to the states– which might also allow for more comparability of scores and results across competing products.” (page 44). What this also means is that states might be administering PARCC without the public knowing it.
The Structure itself would need to develop content, which seems to defeat the initial purpose of a consortium of states supposedly being in charge of running their own assessment show. The economics of the situation seems to dictate as much.
Such a Structure would have all of the power– and just imagine if that Structure could rope in Smarter Balanced and other states, as well. Looks like we’re heading toward a monopoly, when corporate reformers are supposedly against monopolies. Then again, it is not a monopoly if we call it taking a corporate reform idea “to scale”:
While we agree that achieving economies of scale is critical to the viability of the new Structure, relying on state contributions of content is unlikely to be an effective means of supporting this principle. (page 46)
Even contributing items to PARCC would not mean the states have ultimate control over those items:
…While according to the RFI, states would continue to have rights to use contributed items for free, they would in some very real sense need to lose control of those items. They could not, for example, decide to release items that might be used in secured tests in other states. (page 46).
ETS is apparently concerned about conflicts of interest that the PARCC consortium dismissed:
The RFI states that the Structure will have a Board of Directors. This is not a problem, though it does assume that the Structure will not be a privately-held entity, not all of which have such Boards. The RFI also outlines that affiliated states would have some number of seats on this Board. While not inherently a problem, we believe that the authors of the RFI should consider certain conflicts of interest that this arrangement might induce. States will be both customers of the Structure, and members of a governing board. In the former role, they are likely to be concerned with the lowest short-term price for services. This may conflict or appear to conflict with the role a board of directors plays, since directors must concern themselves with the long-term health of the Structure, which may be at odds with short-term prices. Although the Governing Board provided answers to questions on March 7th which indicated that the Board does not believe this to be a conflict of interest, we are still concerned that it might be. (page 51)
ETS suggests that states belong to a board of advisors, which adds another layer but prevents the conflict of interest noted above by states’ having the control of a board of directors. (page 52)
It seems that ETS really likes the idea of Smarter Balanced becoming the overseer of PARCC. Of course, this could well lead to PARCC’s being subsumed into Smarter Balanced:
Another thing to consider is the issue which Louisiana Superintendent of Education John White has raised with testing companies about making the repository of excellent test content broader than just PARCC content. We would recommend that the Structure not be limited to only PARCC IP, but also include Smarter Balance IP. In this case, the PARCC Governing Board would not have the same amount of influence over the Structure as the RFI suggests.
Given states’ interest in seeking an entity that might offer both PARCC and Smarter Balanced IP, the Governing Board may wish to consider the Smarter Balanced consortium as a candidate for your Structure. If selected to play this role, Smarter Balanced could lease states and vendors IP from PARCC, Smarter Balanced, or perhaps a combination thereof. Of course, if a single entity were to serve as the steward of the IP from both consortia and broker for the development of new content, it is possible that the unique structure and nature of the two bodies of content may become increasingly less distinguishable over time. It is not clear whether the Governing Board would see this possible development in a positive, negative, or neutral light.
It would also appear that [the] Smarter Balanced team has the requisite technical expertise, state relationships, and organizational experience and bandwidth to support the Governing Board’s goals. Furthermore, the organization also appears well-suited to avoid the myriad potential conflicts of interest described in our responses above. Finally, combining both consortia’s IP management into a single Structure would significantly enhance sustainability by eliminating redundant staff and operational expense, and would provide more items for building future forms for each consortium, thus saving investment dollars for keeping the content fresh. (pages 51-52)
And that is how ETS ends it– with the suggestion to follow John White and create one Common Core testing consortium.
Measured Progress offers its RFI on pages 57-70. Some highlights include Measured Progress’ supporting the idea that PARCC Governing Board members be included as members of the Structure board of directors (page 61). Measured Progress also suggests an option for PARCC to have a Third Party Coordinator (TPC) to act as adviser/mediator between the PARCC Governing Board and a Content Management Contractor (CMC), with the CMC and TPC being part of the Structure along with a board of directors.
The rest of the Measured Progress RFI offers a detailed breakdown of additional questions/clarifications PARCC should consider in its restructuring. What PARCC wants to do with its different test-item usage options and varied levels of state involvement make it sound like PARCC did not really consider what it was getting itself into in offering such options to states. Then again, like much that is associated with corporate ed reform, the decision to “do” PARCC in the first place (via those enticing Race to the Top assessment grants) appears to have occurred in the absence of any notable consideration beforehand of what “doing”– and maintaining— would realistically require.
PARCC, Inc., is currently the nonprofit overseeing PARCC. In its cover letter, it makes it clear that it was created to oversee PARCC and that it wants to continue to have the right to do so. (pages 71-72)
It is the only entity whose continued existence depends upon being selected as the Structure after its June 2017 contract expires.
PARCC, Inc., references a licensing model matrix (page 78) that has apparently been deleted from the FOIA. The matrix “seeks to clarify the role of the Structure, states, and testing vendors”– with PARCC, Inc., already in the Structure role.
On page 79, PARCC, Inc. states that “having the Structure own all PARCC assessment items and ancillary materials is the right course of action. (page 79) PARCC, Inc., writes that it has previously owned all items after it and PARCC states consulted with the legal counsel of Arnold and Porter. However, PARCC, Inc., has not produced such a document that details how, exactly, a temporary entity “owns” PARCC items.
PARCC, Inc., is opposed to state co-ownership of items with the Structure (i.e., itself) because states could basically do what they want with the items they co-own despite what PARCC, Inc., thinks about the issue. Test security could easily become a problem. That noted, the states in the Common Core testing consortia supposedly were the owners of the items they created. However, PARCC, Inc, notes:
It is legally possible for states to co-own content with the Structure. For example, content that is contributed by a state [and] that meets the standards of quality and need, as determined by the Structure, could be co-owned by that state and the Structure. However, while it is possible, it may not be desirable, as it introduces added cost complexity without much additional benefit. Co-ownership presents the following challenges:
- If items in the bank have different combinations of owners, they could require different approvals and conditions;
- It will open IP disclosure risks that would threaten test security and sustainability; and
- If departing states were to take their content, it may deplete the bank and create risk for the remaining states.
Assuming the goals of co-ownership are the ability (1) to say that the state is “in control of its testing program and (2) for the state to use content even after it withdraws from its affiliation with the Structure, these goals can be achieved through licensing and use agreements even if the Structure holds and owns all content in the PARCC item bank. While the idea of IP being co-owned may be attractive for political reasons, the cost and time spent managing co-ownership would not be insignificant for the Structure or the state, and the total cost would be higher than a single-owner model. (page 81)
Know when such RFI considerations should have happened? Prior to the Obama/Duncan Race to the Top Assessment Consortium contest announced in June 2009 at the National Governors Association annual meeting.
Regarding the details of PARCC, Inc.’s proposed model for moving states from conceptualization to implementation of PARCC assessments has been deleted from the FOIA. (page 85)
Throughout the remainder of its RFI (entire RFI, pages 74-94), PARCC, Inc., is clearly interviewing for the job of continuing in its role as overseer of the PARCC assessments. It notes that there have been issues with the “multilayered decision making process” (page 89) and with state chiefs disagreeing and it hints that “tensions” have “paralyzed” the decision making process. (page 90) Thus, PARCC, Inc., suggests that the Structure (which, of course, means PARCC, Inc.) be the key decision maker. (page 90)
The short of it is that PARCC, Inc., thinks it has done a fine job and that it should be retained as PARCC overseer and given broader decision making power, and that the real problem has been due to the state leaders on the PARCC Governing Board.
However, PARCC, Inc., has been overseeing PARCC since 2014, and during that time, PARCC state membership has dwindled. According to the logic of market-driven reform, that makes PARCC, Inc., a “failing” nonprofit.
Pearson’s cover letter is missing; cover-letter-less its RFI response begins on page 95.
I wonder if the Pearson cover letter told off PARCC for its failure to deliver a juicy state membership to Pearson, the once-chosen official vendor of PARCC tests. Part of that failure included allowing Louisiana to deliver so-called PARCC assessments in 2014-15 without having a contract with Pearson to do so, which was supposed to be the arrangement, and which further contributed to a reduced number of states contracting with Pearson for PARCC assessments. Of course, their also might have been some clandestine arrangement to reimburse Pearson for not getting a Louisiana contract.
The point is, the public does not get to know.
Pearson suggests that states retain ownership of their items and license the items to the PARCC consortium. (page 98)
Pearson notes that PARCC has struggled with “adequate field test participation” and adds:
Without assurances for field test participation or other such mechanisms to provide adequate content with compatible statistical data, the structure may lack the control necessary to maintain sufficient content in some areas to fulfill assessment blueprints. (page 104)
Pearson adds that the ability to adequately field test full PARCC tests is threatened by states opting to utilize PARCC items rather than entire PARCC tests.
Pearson believes that the PARCC consortium could become little more than an item vendor of items created years ago under the federal money:
…We suggested that the Partnership states might consider the full preservation of the full PARCC summative assessments as a priority. Without a critical mass of affiliate states that are committed to and providing oversight over a common assessment, there is a danger that the structure will soon be maintaining nothing more than an item bank of content that was established by the Race to the Top funding and the initial operational years of the PARCC assessment. (page 110)
Pearson suggests that PARCC consider subcontracting its item development and item banking. (page 111)
Some highlights from the Questar RFI, which begins on page 113, include this observation about the Structure’s owning PARCC content:
A clear benefit of the structure owning all of the content is the degree to which an arrangement would make licensing more straightforward and easy to implement, as the entity will have carte blanche to configure licensing terms and protect IP.
A challenge created by the structure’s ownership of all content, though, is the degree to which the model might inhibit the states’ willingness to contribute items. (page 116)
Questar’s concerns about co-ownership of items has been previously noted in this post (for example, under the PARCC, Inc., section), and has led Questar to suggest that the Structure be “owner of all content.” (page 117)
Regarding PARCC as offering to be both full-test provider as well as item vendor, Questar has this to offer:
We consider the differentiated levels of access to be a well-devised feature of the model, given the feature’s potential to motivate states to contribute content. Such a system may be necessary to overcome states’ objections to relinquishing the rights they associate with traditional models of ownership related to the test assets in which they invest.
The disadvantage of this aspect of the model may be the challenge of mandating the volume of content development that states must agree to generate in order to achieve a high level of item access. As with any such comparable model, the participants will tend to act according to their own self-interest, consuming more and contributing less than necessary, unless required to do so otherwise. (page 118)
Questar then offers its idea of model adjustments, which are blacked out. (page 118) However, whatever the new model is, it offers the Structure “the advantage… or potentially tapping into the power of a competitive marketplace, as organizations like Questar (and those we compete with) devise innovative ways to use structure content and add value in the normal course of striving to surpass the competition.” (page 119) And then the next sentence is blacked out.
Questar’s governance model includes an advisory group, but most of the Questar details on governance are also blacked out. (page 119)
The Smarter Balanced Assessment Consortium is the last entity on the list of those responding to PARCC’s RFI (page 122).
Smarter Balanced points out that it is not facing the membership and financial difficulties faced by PARCC:
The Smarter Balanced Consortium currently has 17 members, including 15 states, the US Virgin Islands, and the Bureau of Indian Education. We are financially self-sustaining, with a reserve fund sufficient to support core operations for 12 months without additional revenue. Currently, we have only two active foundation grants comprising less than 2 percent of our revenue and we receive no money from the US Department of Education. Consortium members have access to a complete assessment system, including adaptive, summative assessments; more than 90 interim assessments, with additional interim assessments under development; and a Digital Library including more than 2,500 instructional and professional development resources that incorporate the formative assessment process. (page 124)
One of Smarter Balanced decisions involves “making available packages of items to the community of vendors,” where states choose their own vendors. (page 125)
Smarter Balanced also charges states a membership fee. Non-member states must get board approval to access Smarter Balanced content– and must pay the same fees “as would otherwise be required for membership.” (page 125)
What Smarter Balanced has that PARCC does not is a larger number of states willing to belong to the consortium– and to remain in the consortium. It also seems that the leadership of Smarter Balanced member states are able to get along better than those in PARCC, which appears to include the ability to hash out disagreements. (page 127)
The University of California (UC) oversees the Smarter Balanced materials:
The University of California (UC) owns the Smarter Balanced intellectual property; however… should the consortium seek to terminate the relationship with UC, the University agrees in the MOU (memorandum of understanding) to transfer all assets to a third party designated by the members. Members receive a license to use the Consortium’s resources… in exchange for membership dues. (page 127)
In its RFI, Smarter Balanced makes no offer to oversee PARCC. It simply details what it has done. The idea that Smarter Balanced oversee PARCC is an ETS suggestion apparently originating with Louisiana superintendent John White’s behind-the-scenes maneuvering with testing company execs to somehow bring both PARCC and Smarter Balanced under the same Structure.
Given the length of this post, I’ll keep this closing brief:
PARCC now has its RFI suggestions. It needs to decide what it wants to do, and it has until the end of the 2016-17 school year (June 2017, to be precise) before its contract with PARCC, Inc., expires.
We’ll see if PARCC hands itself over to Smarter Balanced– or if Smarter Balanced even wants oversight of PARCC.
I would not be surprised if some PARCC states are already considering jumping the PARCC ship in favor of seemingly more stable Smarter Balanced. But those states need to be willing to pay the Smarter Balanced membership fee.
Frankly, I don’t see PARCC recovering from this situation as a consortium.