Skip to content

CREDO Evaluates Federal Grant for Scaling Charters (NOLA and TN): *Ambitious Targets Not Met*

March 27, 2018

Neither New Orleans’ Recovery School District (RSD) nor Tennessee’s Achievement School District (ASD) is particularly impressive. Of course, the goal was not just to take over traditional public schools but to replace those schools with charter schools.

This replacement is called the “charter restart model” (CRM).

In short: No jaw-dropping, stellar, see-we-told-you-so results.

And those charter schools were supposed to outperform by leaps and bounds the traditional public schools they replaced.

Well.

The Center for Research on Education Outcomes (CREDO) composed the federally-required evaluation report, which is featured on the CREDO website.

Below are some excerpts.

student desk This is a “high quality seat.” Just go with it.

From CREDO:

The CRM was predicated upon a particular Theory of Action: investments made in charter management organizations (CMOs) with proven track records of success in improving performance for academically disadvantaged students would increase systemwide capacity to turnaround low-performing schools. This, in turn, would increase the number of high quality seats available to students within the CRM ecosystem.

To implement this Theory of Action, the CRM comprised three overarching goals. First, NSNO (New Schools for New Orleans) and RSD would build local capacity to incubate and expand charter restart operators. Second, NSNO and RSD would create sustainable infrastructure to sustain the charter restart model in perpetuity. Finally, NSNO and RSD would demonstrate the scalability of the CRM by codifying the model and replicating it in Tennessee via partnership with ASD.

Toward these ends, NSNO and RSD in New Orleans and ASD in Tennessee granted i3 (Us Department of Education “Investing in Innovation”) funds to CMOs in yearly cohorts to turn around lowest-performing schools, with the explicit aim of moving schools from the bottom 5% of performance to the top 33% in New Orleans or the top 25% in Tennessee. …

We observed wide variation in school performance. School performance was found to associate significantly with a number of systems-level factors. In particular, despite a rigorous and fine-tuned design for selection processes to identify charter operators for turnaround schools, the selection process as implemented in New Orleans suffered a loss of integrity for a period of time. The resulting turnaround operators from that period did not have the necessary skills, experience, and capacities to perform their roles successfully. To their credit, NSNO and RSD revised their approach before the end of the grant period and selected stronger partners to conduct the difficult work of school turnaround in later cohorts. …

The most publically visible shortcoming within Goal 2 concerned community engagement. None of the program partners, (NSNO, RSD, or ASD) ever successfully managed stakeholder engagement as a core commitment, as per the original CRM Theory of Action. …

A critical portion of the i3 grant was to establish the feasibility of scaling the CRM to other communities that had similar legislative and regulatory foundations. …

However, scaling the CRM to Tennessee did not entirely succeed. …

The decentralization upon which the CRM is predicated replaces a single point source management (district) with a multi-point source management structure. The CRM further creates a set of levers (citywide school choice, CMO- and school-level autonomy, decentralized student supports) which in turn drive school quality in the absence of a single central oversight authority. This decentralization, coupled with the faulty assumption that the CRM would move schools from closure to successful schools in one try, created externalities which the CRM system-level partners did not anticipate and/or were not equipped to resolve. Systems-level partners missed key opportunities to intervene early in the study period and as such left CMOs and schools vulnerable to exogenous and endogenous shocks.

CREDO then states that the New Orleans OneApp “vastly resolved issues of equitable access to schools.” However, even Enroll NOLA’s OneApp FAQ sheet notes that some students have “higher priority” than others.

Some more from CREDO, specifically, about teacher and admin turnover:

The limitations of human capital pipelines in both New Orleans and Tennessee impacted every CRM school. Principals struggled to find teachers who both fit their schools’ culture and who also could produce student results. Teachers reported consistent frustration in accessing professional development resources. Because teacher turnover was so high, and the teaching corps in both Memphis and New Orleans so inexperienced, professional development focused on basics year after year in support of new teachers, instead of progressing to more sophisticated pedagogical topics to support the development of more experienced – but still hardly expert –teachers. … By the middle years of the study, principals reported that they relied as frequently on teachers quitting as they did on professional development to bolster the quality of their teaching corps. Indeed, teacher quality was unilaterally considered by schools to be a fundamental challenge to the success of the CRM.

Principal turnover also plagued CRM schools. Twelve of the CRM schools had at least one school leader turnover during the study period, and one school had as many as five leadership transitions. Leadership turnover created inevitable disruptions to the continued maturation of school operations, as new leaders learned systems, rebuilt relationships with staff, and often introduced operational and/or pedagogical approaches that differed from their predecessor. Overall, both teacher and leader turnover – even when moving to superior talent – impacted cohesion and smooth operations. …

And about those lofty transformation goals:

The Charter Restart Model envisioned the creation of schools that would transform from lowest 5 percent in their local school performance distributions to the top 33 and top 25 percent of schools in New Orleans and Tennessee, respectively. To reach this ambitious goal, students in CRM schools would have needed to outperform their peers every year by very large margins in order to move up in the distribution. The necessary trajectory of gains provides the motivation for the Student impact analysis: if the CRM was successful, students in the restart schools would grow academically faster than their peers in non-CRM schools. …

The full set of findings appears in the Student Impact Report; here, we present only the most salient results. Over all schools and all years of study, the student academic progress in CRM schools did not differ from that observed in the non-CRM schools in their local ecosystems. These results …show that in the aggregate, CRM students posted academic gains that were not statistically different from their peers in either reading or math. This finding holds when the schools are disaggregated by geography: in New Orleans and in Tennessee, the state-level findings show no significant gains in reading or math compared to their non-CRM peers. Based on these topline results, it is clear that the CRM’s ambitious performance targets were not met.

Deeper analysis of the impact of the CRM on student progress was more revealing. When we look at school-level academic growth, three of the 13 schools in New Orleans did achieve this target in reading; and two of 13 New Orleans schools achieved this target in math. No CRM schools in Tennessee achieved the performance target in either subject. …

While these aggregate findings of the Impact evaluation suggest little progress by the CRM overall, the aggregate findings obscure additional important insights, particularly regarding the model of intervention implemented by any given operator.

So. The charter restart model was not the “phenomenon of disruption” it was billed. The CREDO report alludes to “incremental improvement” and “given operator” progress.

And according to CREDO, almost-all-charter New Orleans “continues to evolve”:

It is important to note here that as the CRM schools stabilized, matured, and built resilience, the system was simultaneously maturing around them. In the CRM’s first two years, NSNO and RSD stood back while CMOs (charter management organizations) were expected to solve, ignore, or supersede systems-level barriers that they were, in fact, ill equipped to impact. But as the system evolved, NSNO and RSD recalibrated their activity to build the connective tissue – the system – that operates above and between CMOs and schools. Community support for the system also evolved in this time, as evidenced by the yearly Cowen Institute public opinion polls, the reduction in community protests, and the nascent engagement of families with their children’s schools and CMOs.

Of course, there is the added layer of all RSD charters “returning” to the Orleans Parish School Board (OPSB) by 2019, which has the leverage of revoking a charter, say, if a charter operator tries to go it on the cheap and give city bus tokens to small children instead of providing yellow school buses. However, it seems in the face of having its charter revoked (and the charter operator replaced), the charter operator for Einstein charter schools has apparently come around to the idea of providing those yellow buses.

But I digress.

I’ll leave interested readers to peruse the rest of the CREDO report for themselves.

student desk Hmm. It seems it’s just a desk. My bad.

______________________________________________________________________________________

Want to read about the history of charter schools and vouchers?

School Choice: The End of Public Education? 

school choice cover  (Click image to enlarge)

Schneider is a southern Louisiana native, career teacher, trained researcher, and author of two other books: A Chronicle of Echoes: Who’s Who In the Implosion of American Public Education and Common Core Dilemma: Who Owns Our Schools?. You should buy these books. They’re great. No, really.

both books

Don’t care to buy from Amazon? Purchase my books from Powell’s City of Books instead.

From → Charters, RSD

3 Comments
  1. Lance Hill permalink

    CREDO has been glorifying the New Orleans category five charter disaster for more than ten years. This study is just another attempt to excuse their selective use of data. There are some kiss-and-tell moments, including that Charters found the fastest way to improve teaching quality was to get rid of low evaluated teachers–just like they eliminate low-evaluated students to raise test scores.

  2. Laura H. Chapman permalink

    I was out of town and did not see your report on the CREDO study until today (April 8)—after I had done some highlighting of my own from the Executive Report focused onkeeping the “New Orleans Charter Restart Model” alive and available for “scale-up” in spite of massive failures and wrong assumptions about what matters in education—a relentless focus on student academic performance, meaning test scores, and iron-fisted management of schools. In the end, the researchers seem to throw up their hands and give up.
    Here are some excerpts:

    “Principals struggled to find teachers who both fit their schools’ culture and who also could produce student results. Teachers reported consistent frustration in accessing professional development resources. Because teacher turnover was so high, and the teaching corps in both Memphis and New Orleans so inexperienced, professional development focused on basics year after year in support of new teachers, instead of progressing to more sophisticated pedagogical topics to support the development of more experienced – but still hardly expert – teachers” (p. 10).

    “…New Orleans Charter Restart Model systems-level partners will need to strategically and proactively plan for low teacher and principal retention in coming years” (p. 9).

    …”We find on average that New Orleans Charter Restart Model schools struggle with core functions through their third, fourth, and fifth years of operation across multiple domains “(p.11).

    “In total, the findings of the Implementation study suggest that the required runway for school turnaround is years longer than anticipated and requires multi-dimensional supports and capacities”(p.12).

    “Based on these topline results (in the Student Impact study) it is clear that the New Orleans Charter Restart Model’s ambitious performance targets were not met” (p.12).

    “Taken in aggregate, the findings…indicate that the New Orleans Charter Restart Model was not entirely successful…. (and) …”may take years to get right, but without which large scale education reform cannot succeed” (p. 17). “At its core, the New Orleans Charter Restart Model posits that Charter Management Organizations (CMOs) are the primary lever for educational quality to improve” (p. 18). WHAT? WHAT?

    CMOs entered some neighborhoods that the CMOs said: “had not had a high performing school in living memory. These CMOs “ faced the critical task of not only installing a functional school, but also educating neighbors and parents about what to expect from such a school (p.22). Notice that a school is to be “installed” in a community. That is one of many dubious constructs in the “restart model.” You can install a portable classroom, a portable toilet, but a school is not so easily dropped on a community, ready to use. The CMOs seemed to be surprised that members of some communities cherished their bands and football programs and did not think an extended school day was a great idea.

    “Tension between community members and alumni who mourned the loss of school bands and football teams …and New Orleans Charter Restart Model operators looking to extend instructional time often resulted in compromise solutions that allowed schools to protect beloved cultural institutions, but in doing so forced a diversion from their most obvious path to increasing academic rigor “(p. 22). …In some cases, we find New Orleans Charter Restart Model schools are unable to achieve the balance of academics and enrichment observed in their Flagships due in part to how their unique microcontexts behaved” (p. 22).
    Wow—that really is a revealing statement, likely an understatement of the arrogance of CMOs intent on shoving their view of a proper education into a community.

    CREDO researchers think of “tool kits” and “arsenals of” system-level reforms (intervention strategies) to make charter schools succeed, if not on the first attempt then on the “second, third, and fourth attempts” (p. 25). One of these, mentioned favorably three times in the report is the engagement of the Relay Graduate School of Education “to strengthen the human capital pipeline (p. 6).” WEAKEN is a more accurate word if you care a wit about professionals in education.

    Here is the most remarkable assertion:
    The Executive summary ends with the idea that “system-level barriers” (which refers the whole architecture of the New Orleans Charter Restart model) might better be dumped or “delicately bypassed” in favor of “thoughtfully applied instructional strategies such as targeted mentorship, place-based learning, blended learning and other instructional options (p. 26) for “reforming learning itself,” unencumbered by “human capital constraints, facilities challenges, integrated holistic services, et cetera” (p. 26).

    That last statement reads like a repudiation of the whole New Orleans Charter Restart Model with only a random array of slogans like “targeted mentorship,” and “place-based learning” as alternatives.

    http://nolai3eval.stanford.edu/sites/default/files/documents/NEW ORLEANS CHARTER RESTART MODEL%20Executive%20Summary.pdf

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: